On various intraclass correlation reliability coefficients.

Abstract

Bartko (1966, 1974) has presented some analysis of variance (ANOVA) intraclass correlation reliability coefficients that avoid some serious deficiencies not uncommonly found in reliability measures. In his second edition, Winer (1971, pp, 289-296) presented some intraclass correlation results which appear to have deficiencies. His so-called for anchor points approach will produce an intraclass correlation reliability coefficient of unity (as expected) for the case in which the judges (raters) agree perfectly about a group of subjects. However, the method will also yield an intraclass correlation of unity for the case in which the judges display a constant additive bias. In general with Winer's approach, any adjustment of original rating data that leaves the rater's variance-covariance matrix unaltered will produce the same intraclass correlation coefficient, and thus numerous variations (of which additive bias is a subset) of the original data set can and will yield the same intraclass correlation. Bias and Unity Reliability As a first illustration on a more elementary level, the phenomenon discussed above can be observed with the product-moment correlation, which is a sometimes used but not recommended measure of reliability. Consider

Keywords

Related Publications