Abstract
Receiver operating characteristic (ROC) curves are used to describe and compare the performance of diagnostic technology and diagnostic algorithms. This paper refines the statistical comparison of the areas under two ROC curves derived from the same set of patients by taking into account the correlation between the areas that is induced by the paired nature of the data. The correspondence between the area under an ROC curve and the Wilcoxon statistic is used and underlying Gaussian distributions (binormal) are assumed to provide a table that converts the observed correlations in paired ratings of images into a correlation between the two ROC areas. This between-area correlation can be used to reduce the standard error (uncertainty) about the observed difference in areas. This correction for pairing, analogous to that used in the paired t-test, can produce a considerable increase in the statistical sensitivity (power) of the comparison. For studies involving multiple readers, this method provides a measure of a component of the sampling variation that is otherwise difficult to obtain.
Keywords
Receiver operating characteristicStatisticsCorrelationWilcoxon signed-rank testStatisticGaussianSensitivity (control systems)MedicinePattern recognition (psychology)Statistical powerSampling (signal processing)Correlation coefficientArtificial intelligenceMathematicsComputer science
Related Publications
An Evaluation of Methods for Estimating the Area Under the Receiver Operating Characteristic (ROC) Curve
The area under the receiver operating characteristic (ROC) curve serves as one means for evaluating the performance of diagnostic and predictive test systems. The most commonly ...
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
A representation and interpretation of the area under a receiver operating characteristic (ROC) curve obtained by the "rating" method, or by mathematical predictions based on pa...
A Computer Program for Rapid Generation of Receiver Operating Characteristic Curves and Likelihood Ratios in the Evaluation of Diagnostic Tests
We describe a MUMPS program to facilitate the evaluation of the diagnostic effectiveness and efficiency of laboratory tests using receiver operating characteristic (ROC) curves ...
Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach
Methods of evaluating and comparing the performance of diagnostic tests are of increasing importance as new tests are developed and marketed. When a test is based on an observed...
Analyzing a Portion of the ROC Curve
The area under the ROC curve is a common index summarizing the information contained in the curve. When comparing two ROC curves, though, problems arise when interest does not l...
Publication Info
- Year
- 1983
- Type
- article
- Volume
- 148
- Issue
- 3
- Pages
- 839-843
- Citations
- 7003
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
7003
OpenAlex
Cite This
J A Hanley,
Barbara J. McNeil
(1983).
A method of comparing the areas under receiver operating characteristic curves derived from the same cases..
Radiology
, 148
(3)
, 839-843.
https://doi.org/10.1148/radiology.148.3.6878708
Identifiers
- DOI
- 10.1148/radiology.148.3.6878708