ESTIMATION OF RECEIVER OPERATING CHARACTERISTIC (ROC) CURVE PARAMETERS: SMALL SAMPLE PROPERTIES OF ESTIMATORS.

Persistent Link:
http://hdl.handle.net/10150/184127
Title:
ESTIMATION OF RECEIVER OPERATING CHARACTERISTIC (ROC) CURVE PARAMETERS: SMALL SAMPLE PROPERTIES OF ESTIMATORS.
Author:
BORGSTROM, MARK CRAIG.
Issue Date:
1987
Publisher:
The University of Arizona.
Rights:
Copyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.
Abstract:
When studying detection systems, parameters associated with the Receiver Operating Characteristic (ROC) curve are often estimated to assess system performance. In some applied settings it is often not possible to test the detection system with large numbers of stimuli. The resulting small sample statistics many have undesirable properties. The characteristics of these small sample ROC estimators were examined in a Monte Carlo simulation. Three popular ROC parameters were chosen for study. One of the parameters was a single parameter index of system performance, Area under the ROC curve. The other parameters, ROC intercept and slope, were considered as a pair. ROC intercept and slope were varied along with sample size and points on the certainty rating scale to form a four way factorial design. Several types of estimators were examined. For the parameter, Area under the curve, Maximum Likelihood (ML), three types of Least Squares (LS), and Distribution Free (DF) estimators were considered. Except for the DF estimator, the same estimators were considered for the parameters, intercept and slope. These estimators were compared with respect to three characteristics: bias, efficiency, and consistency. For Area under the curve, the ML estimator was the least biased. The DF estimator was the most efficient, and all the estimators except the DF estimator appeared to be consistent. For intercept and slope the LS estimator that minimized vertical error of the points from the ROC curve (line) was the least biased for both estimators. This LS estimator was also the most efficient. This estimator along with the ML estimator also appeared to be the most consistent. The other two estimators had no significant trend toward consistency. These results along with other findings, illustrate that different estimators may be "best" for different sample sizes and for different parameters. Therefore, researchers should carefully consider the characteristics of ROC estimators before using them as indices of system performance.
Type:
text; Dissertation-Reproduction (electronic)
Keywords:
Signal detection (Psychology); Parameter estimation.
Degree Name:
Ph.D.
Degree Level:
doctoral
Degree Program:
Educational Foundations and Administration; Graduate College
Degree Grantor:
University of Arizona

Full metadata record

DC FieldValue Language
dc.language.isoenen_US
dc.titleESTIMATION OF RECEIVER OPERATING CHARACTERISTIC (ROC) CURVE PARAMETERS: SMALL SAMPLE PROPERTIES OF ESTIMATORS.en_US
dc.creatorBORGSTROM, MARK CRAIG.en_US
dc.contributor.authorBORGSTROM, MARK CRAIG.en_US
dc.date.issued1987en_US
dc.publisherThe University of Arizona.en_US
dc.rightsCopyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.en_US
dc.description.abstractWhen studying detection systems, parameters associated with the Receiver Operating Characteristic (ROC) curve are often estimated to assess system performance. In some applied settings it is often not possible to test the detection system with large numbers of stimuli. The resulting small sample statistics many have undesirable properties. The characteristics of these small sample ROC estimators were examined in a Monte Carlo simulation. Three popular ROC parameters were chosen for study. One of the parameters was a single parameter index of system performance, Area under the ROC curve. The other parameters, ROC intercept and slope, were considered as a pair. ROC intercept and slope were varied along with sample size and points on the certainty rating scale to form a four way factorial design. Several types of estimators were examined. For the parameter, Area under the curve, Maximum Likelihood (ML), three types of Least Squares (LS), and Distribution Free (DF) estimators were considered. Except for the DF estimator, the same estimators were considered for the parameters, intercept and slope. These estimators were compared with respect to three characteristics: bias, efficiency, and consistency. For Area under the curve, the ML estimator was the least biased. The DF estimator was the most efficient, and all the estimators except the DF estimator appeared to be consistent. For intercept and slope the LS estimator that minimized vertical error of the points from the ROC curve (line) was the least biased for both estimators. This LS estimator was also the most efficient. This estimator along with the ML estimator also appeared to be the most consistent. The other two estimators had no significant trend toward consistency. These results along with other findings, illustrate that different estimators may be "best" for different sample sizes and for different parameters. Therefore, researchers should carefully consider the characteristics of ROC estimators before using them as indices of system performance.en_US
dc.typetexten_US
dc.typeDissertation-Reproduction (electronic)en_US
dc.subjectSignal detection (Psychology)en_US
dc.subjectParameter estimation.en_US
thesis.degree.namePh.D.en_US
thesis.degree.leveldoctoralen_US
thesis.degree.disciplineEducational Foundations and Administrationen_US
thesis.degree.disciplineGraduate Collegeen_US
thesis.degree.grantorUniversity of Arizonaen_US
dc.identifier.proquest8716347en_US
dc.identifier.oclc698483837en_US
All Items in UA Campus Repository are protected by copyright, with all rights reserved, unless otherwise indicated.