How to Calculate Intraclass Correlation Coefficient in Excel

This article will show you how to calculate Intraclass Correlation Coefficient (ICC) in Excel.

Table of Contents hide

Download Workbook

What is the Intraclass Correlation Coefficient?

How to Calculate Intraclass Correlation Coefficient (ICC) with Example

Conclusion

Download Workbook

You can download the free practice Excel workbook from here.

Calculate Interclass Correlation Coefficient (ICC).xlsx

What is the Intraclass Correlation Coefficient?

The Intraclass Correlation Coefficient (ICC) determines the reliability of ratings for multiple items where there are multiple raters involved. The ratings are measured by comparing the variability of different ratings of the same item (or subject) rated reliably by different raters.
The ratings are quantitative. The value of an ICC can range from 0 to 1, where 0 means no reliability and 1 means perfect reliability among raters.
Based on the following three factors, the result of the various versions of an ICC is calculated.

Factors	Description
Model	One-Way Random Effects: This model indicates that each item is rated by a different group of randomly chosen raters. The raters are considered the source of random effects while using this model. As the same group of raters usually rate each item, so this model is rarely used in practice. Two-Way Random Effects: This model indicates that each item is rated by a number of randomly selected raters from a population. Both the raters and the subjects are considered the sources of random effects while using this model. When we want to generalize the findings to any raters similar to the raters used in the study, then this model is used. Two-Way Mixed Effects: This model indicates that each item is rated by a number of randomly selected raters from a population. The chosen group of raters are the only raters of interest for this model. So, we don’t want to generalize the findings to any other raters who share similar characteristics as the raters used in the study.
Type of Relationship	Consistency: The systematic differences between the ratings of judges. Absolute Agreement: The absolute differences between the ratings of judges.
Unit	Single Rater: When only a single rater is the base for the measurement. The Mean of Raters: When the mean of ratings from all the judges is the base for the measurement.

How to Calculate Intraclass Correlation Coefficient (ICC) with Example

From this section, you will learn the steps for calculating ICC with an easy example.
Suppose four different judges were asked to rate the quality of 10 different products. The ratings given by them are shown below (and the example dataset for this article).

Now let’s learn the steps of calculating the ICC for this rating variability in Excel.

Steps:

Firstly, click the tab Data -> Data Analysis.

Secondly, a Data Analysis pop-up window will appear. Then, select Anova: Two-Factor Without Replication.
Later, click OK.

After that, another Anova: Two-Factor Without Replication pop-up window will come up. At first, in the Input Range box, drag through the whole range from your dataset that contains the rating values. For our case, the range is C5:F14.
Then, in the Output Range box, click on the cell in your dataset that you want as the starting cell of the output result. We want cell H4 as the starting cell of the output range, so we selected cell H4.
Lastly, click OK.

Finally, you will get the outcome of the Anova: Two-Factor Without Replication in your worksheet. Accordingly, you will be able to see the whole Summary of the calculations along with the individual Count, Sum, Average and Variance of the ratings by the Judges for the Products.

Then, to measure the ICC based on those rating variations, you need the bottom Anova table for input values.
The formula to calculate the Intraclass Correlation Coefficient (ICC) for the extracted output is:

=(K26-K28)/(K26+J27*K28+(J27+1)*(K27-K28)/(J26+1))

Subsequently, the Intraclass Correlation Coefficient (ICC) for our dataset is 0.08840678.

The interpretation of the value of an Intraclass Correlation Coefficient (ICC) is as follows:

Less than 0.50 -> Poor reliability
Between 0.5 and 0.75 -> Moderate reliability
Between 0.75 and 0.9 -> Good reliability
Greater than 0.9 -> Excellent reliability

As a result, we can conclude by saying that the calculated ICC of 0.08840678 refers that the products being rated with “poor reliability” by different raters.

Conclusion

To conclude, this article showed you how to calculate Intraclass Correlation Coefficient (ICC) in Excel. Therefore, I hope this article has been very beneficial to you. Furthermore, feel free to ask if you have any questions regarding the topic.