Empirical option weighting and polytomous scoring of multiple choice tests.

Persistent Link:
http://hdl.handle.net/10150/186867
Title:
Empirical option weighting and polytomous scoring of multiple choice tests.
Author:
Cannon, Gregg Eugene.
Issue Date:
1994
Publisher:
The University of Arizona.
Rights:
Copyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.
Abstract:
In order to glean more information from examinees' incorrect responses, various option weighting methods have been investigated. Option weighting involves giving partial credit for incorrect responses. The amount of credit for selection of a particular option may be determined in a number of ways. This study focuses on a method referred to as empirical option weighting. Empirical option weighting is based on the statistical relationship between the option and some criterion. Total scores are calculated by summing the points for each response. This method of scoring is referred to as polytomous scoring. Research in the area of empirical option weighting and polytomous scoring has demonstrated that use of option weighted scoring produces only slight gains in reliability, and gains in reliability are often not accompanied by gains in predictive validity. The rationale for this study was that reliability and predictive validity will be differentially affected by the choice of a criterion on which option weights are based. The hypotheses advanced in this study were: (1) Polytomous scoring using empirical option weights based on an external criterion (dependent variable) would maximize predictive validity over other methods of scoring. (2) Polytomous scoring using empirical option weights based on an internal criterion (total score) would maximize reliability. (3) Psychometrically poor items would be improved in terms of their contribution to reliability and predictive validity through the use of empirical option weighting. Although the results of this study were inconsistent with respect to increasing predictive validity by basing weights on the variable to be predicted, this was explained by the inconsistency in the criterion over cross-validations. In terms of maximizing internal consistency by basing weights on an internal criterion (total score) the results were more consistent. The results suggested that use of an internal criterion tended to maximize internal consistency compared to other methods of weighting. Finally, a comparison of psychometrically good and poor items clearly demonstrated that poor items are significantly improved, both in terms of reliability and predictive validity when scoring used weights based on the variable to be predicted.
Type:
text; Dissertation-Reproduction (electronic)
Degree Name:
Ph.D.
Degree Level:
doctoral
Degree Program:
Educational Psychology; Graduate College
Degree Grantor:
University of Arizona
Committee Chair:
Sabers, Darrell

Full metadata record

DC FieldValue Language
dc.language.isoenen_US
dc.titleEmpirical option weighting and polytomous scoring of multiple choice tests.en_US
dc.creatorCannon, Gregg Eugene.en_US
dc.contributor.authorCannon, Gregg Eugene.en_US
dc.date.issued1994en_US
dc.publisherThe University of Arizona.en_US
dc.rightsCopyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.en_US
dc.description.abstractIn order to glean more information from examinees' incorrect responses, various option weighting methods have been investigated. Option weighting involves giving partial credit for incorrect responses. The amount of credit for selection of a particular option may be determined in a number of ways. This study focuses on a method referred to as empirical option weighting. Empirical option weighting is based on the statistical relationship between the option and some criterion. Total scores are calculated by summing the points for each response. This method of scoring is referred to as polytomous scoring. Research in the area of empirical option weighting and polytomous scoring has demonstrated that use of option weighted scoring produces only slight gains in reliability, and gains in reliability are often not accompanied by gains in predictive validity. The rationale for this study was that reliability and predictive validity will be differentially affected by the choice of a criterion on which option weights are based. The hypotheses advanced in this study were: (1) Polytomous scoring using empirical option weights based on an external criterion (dependent variable) would maximize predictive validity over other methods of scoring. (2) Polytomous scoring using empirical option weights based on an internal criterion (total score) would maximize reliability. (3) Psychometrically poor items would be improved in terms of their contribution to reliability and predictive validity through the use of empirical option weighting. Although the results of this study were inconsistent with respect to increasing predictive validity by basing weights on the variable to be predicted, this was explained by the inconsistency in the criterion over cross-validations. In terms of maximizing internal consistency by basing weights on an internal criterion (total score) the results were more consistent. The results suggested that use of an internal criterion tended to maximize internal consistency compared to other methods of weighting. Finally, a comparison of psychometrically good and poor items clearly demonstrated that poor items are significantly improved, both in terms of reliability and predictive validity when scoring used weights based on the variable to be predicted.en_US
dc.typetexten_US
dc.typeDissertation-Reproduction (electronic)en_US
thesis.degree.namePh.D.en_US
thesis.degree.leveldoctoralen_US
thesis.degree.disciplineEducational Psychologyen_US
thesis.degree.disciplineGraduate Collegeen_US
thesis.degree.grantorUniversity of Arizonaen_US
dc.contributor.chairSabers, Darrellen_US
dc.contributor.committeememberMishra, Shitala P.en_US
dc.contributor.committeememberMencke, Reeden_US
dc.identifier.proquest9506998en_US
All Items in UA Campus Repository are protected by copyright, with all rights reserved, unless otherwise indicated.