DETECTABILITY OF CORRECTNESS: A SIGNAL DETECTION MEASURE OF KNOWING THAT ONE KNOWS

Persistent Link:
http://hdl.handle.net/10150/290556
Title:
DETECTABILITY OF CORRECTNESS: A SIGNAL DETECTION MEASURE OF KNOWING THAT ONE KNOWS
Author:
Hosseini-Ardehali, Jinoos
Issue Date:
1981
Publisher:
The University of Arizona.
Rights:
Copyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.
Abstract:
To decide wisely whether to act on the basis of what one knows, whether to rely upon one's knowledge is an important ingredient in managerial skill. It depends upon knowing what one knows and what one can do. Knowing that one knows is conceptualized as the ability to discriminate what one knows from what one does not. It is measured by detectability of correctness A(c), a nonparametric signal detection measure of the capacity to distinguish correct from incorrect responses. The research reported is an exploration of that concept and of the characteristics of A(c) as a measure of it. The importance of knowing that one knows has been recognized, but little work has been done on it. Previous research along three separate lines is critically reviewed: philosophical analysis in epistemology, empirical research on calibration of subjective probabilities, and studies related to feelings of knowing. The measure A(c) is then developed and experimental results indicate that it is independent of the measure of knowing, the proportion of correct responses. It is shown that when there is opportunity to choose which questions in a test are to be answered, the expected score can be increased by an amount depending on the value of A(c). This effect is, essentially, a statement of the condition for validity of A(c) as a measure of knowing that one knows. The prediction is verified empirically. Reliability of A(c), assessed by the split-half method, is found to be higher than the reliability of the proportion of correct responses. Measurement of A(c) requires respondent-generated answers that can be unambiguously scored. The cloze procedure, of filling in missing words in verbal passages, common in reading research and testing, meets these requirements and has been used in the present research. The application is the first, as far as is known, in which confidence judgments have been obtained with cloze, and A(c) is expected to prove useful as a performance measure in cloze testing. In cloze experiments with native speakers of three different languages, who were also fluent in English, it was found that cloze scores did not distinguish between the native language and English but that A(c) was consistently higher for the native language. This finding and the results of another experiment identifying words heard in noise tend strongly to disconfirm the hypothesis that conscious attention to skill learning leads to one's knowing more about what one knows. Perhaps the opposite is the case. Finally, there is significant, but not conclusive, evidence that detectability of correctness can be improved by learning. It is concluded that knowing that one knows is a useful concept separate from knowing and is suitably measured by A(c), but that the determinants of it and its relation to other abilities are yet to be determined.
Type:
text; Dissertation-Reproduction (electronic)
Keywords:
Decision making.; Management.
Degree Name:
Ph.D.
Degree Level:
doctoral
Degree Program:
Graduate College; Management
Degree Grantor:
University of Arizona

Full metadata record

DC FieldValue Language
dc.language.isoen_USen_US
dc.titleDETECTABILITY OF CORRECTNESS: A SIGNAL DETECTION MEASURE OF KNOWING THAT ONE KNOWSen_US
dc.creatorHosseini-Ardehali, Jinoosen_US
dc.contributor.authorHosseini-Ardehali, Jinoosen_US
dc.date.issued1981en_US
dc.publisherThe University of Arizona.en_US
dc.rightsCopyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.en_US
dc.description.abstractTo decide wisely whether to act on the basis of what one knows, whether to rely upon one's knowledge is an important ingredient in managerial skill. It depends upon knowing what one knows and what one can do. Knowing that one knows is conceptualized as the ability to discriminate what one knows from what one does not. It is measured by detectability of correctness A(c), a nonparametric signal detection measure of the capacity to distinguish correct from incorrect responses. The research reported is an exploration of that concept and of the characteristics of A(c) as a measure of it. The importance of knowing that one knows has been recognized, but little work has been done on it. Previous research along three separate lines is critically reviewed: philosophical analysis in epistemology, empirical research on calibration of subjective probabilities, and studies related to feelings of knowing. The measure A(c) is then developed and experimental results indicate that it is independent of the measure of knowing, the proportion of correct responses. It is shown that when there is opportunity to choose which questions in a test are to be answered, the expected score can be increased by an amount depending on the value of A(c). This effect is, essentially, a statement of the condition for validity of A(c) as a measure of knowing that one knows. The prediction is verified empirically. Reliability of A(c), assessed by the split-half method, is found to be higher than the reliability of the proportion of correct responses. Measurement of A(c) requires respondent-generated answers that can be unambiguously scored. The cloze procedure, of filling in missing words in verbal passages, common in reading research and testing, meets these requirements and has been used in the present research. The application is the first, as far as is known, in which confidence judgments have been obtained with cloze, and A(c) is expected to prove useful as a performance measure in cloze testing. In cloze experiments with native speakers of three different languages, who were also fluent in English, it was found that cloze scores did not distinguish between the native language and English but that A(c) was consistently higher for the native language. This finding and the results of another experiment identifying words heard in noise tend strongly to disconfirm the hypothesis that conscious attention to skill learning leads to one's knowing more about what one knows. Perhaps the opposite is the case. Finally, there is significant, but not conclusive, evidence that detectability of correctness can be improved by learning. It is concluded that knowing that one knows is a useful concept separate from knowing and is suitably measured by A(c), but that the determinants of it and its relation to other abilities are yet to be determined.en_US
dc.typetexten_US
dc.typeDissertation-Reproduction (electronic)en_US
dc.subjectDecision making.en_US
dc.subjectManagement.en_US
thesis.degree.namePh.D.en_US
thesis.degree.leveldoctoralen_US
thesis.degree.disciplineGraduate Collegeen_US
thesis.degree.disciplineManagementen_US
thesis.degree.grantorUniversity of Arizonaen_US
dc.identifier.proquest8120724en_US
dc.identifier.oclc7900418en_US
dc.identifier.bibrecord.b23465323en_US
All Items in UA Campus Repository are protected by copyright, with all rights reserved, unless otherwise indicated.