Abstract
To test the agreement between two observers who categorize a number of objects when the categories have not been specified in advance, Brennan & Light (1974) developed a statistic A ′ and suggested a normal approximation for its distribution. In this paper it is shown that this approximation is inadequate particularly when one, or both, of the observers place a fairly equal number of objects in all of their categories. A chi‐squared approximation to the distribution of A ′ is developed and is shown to work well in a variety of situations. The relative powers of A ′ and the ordinary X 2 test for association are dependent on the type of ‘agreement between the observers’ that is assumed. However a simulation for a fairly general type of agreement indicates that the X 2 test is more powerful. As the X 2 test is also much easier to apply, it would seem preferable in most situations.
Keywords
MathematicsCategorizationTest (biology)Variety (cybernetics)Test statisticStatisticType (biology)Distribution (mathematics)StatisticsStatistical hypothesis testingComputer scienceMathematical analysisArtificial intelligence
Related Publications
MEASURING AGREEMENT WHEN TWO OBSERVERS CLASSIFY PEOPLE INTO CATEGORIES NOT DEFINED IN ADVANCE
Basic to many psychological investigations is the question of agreement between observers who independently categorize people. Several recent studies have proposed measures of a...
Nominal scale response agreement as a generalized correlation
A further discussion of Brennan & Light's (1974) measure of nominal scale response agreement between two raters is given. Specifically, a monotonic function of Brennan &...
Understanding interobserver agreement: the kappa statistic.
Items such as physical exam findings, radiographic interpretations, or other diagnostic tests often rely on some degree of subjective interpretation by observers. Studies that m...
Bootstrap Methods in Econometrics: Theory and Numerical Performance
The bootstrap is a method for estimating the distribution of an estimator or test statistic by resampling one's data. It amounts to treating the data as if they were the populat...
EDF Statistics for Goodness of Fit and Some Comparisons
Abstract This article offers a practical guide to goodness-of-fit tests using statistics based on the empirical distribution function (EDF). Five of the leading statistics are e...
Publication Info
- Year
- 1984
- Type
- article
- Volume
- 37
- Issue
- 2
- Pages
- 271-282
- Citations
- 48
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
48
OpenAlex
Cite This
Richard Brook,
W. Douglas Stirling
(1984).
Agreement between observers when the categories are not specified in advance.
British Journal of Mathematical and Statistical Psychology
, 37
(2)
, 271-282.
https://doi.org/10.1111/j.2044-8317.1984.tb00805.x
Identifiers
- DOI
- 10.1111/j.2044-8317.1984.tb00805.x