A Comparison of Treatment Integrity Assessment Methods for Behavioral Intervention

Persistent Link:
http://hdl.handle.net/10150/193705
Title:
A Comparison of Treatment Integrity Assessment Methods for Behavioral Intervention
Author:
Koh, Seong A
Issue Date:
2010
Publisher:
The University of Arizona.
Rights:
Copyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.
Abstract:
The purpose of this study was to examine the similarity of outcomes from three different treatment integrity (TI) methods, and to identify the method which best corresponded to the assessment of a child’s behavior. Six raters were recruited through individual contact via snowball sampling. A modified intervention component list and 19 video clips were derived from Stahr’s (2005) study, “An Intervention for Children with Autism Spectrum Disorders (ASD) who Have Food Selectivity.” The raters, randomly and evenly assigned to three dyads. Each dyad received an average of six hours training and reached 85% interobserver agreement (IOA) with a 0.60 kappa score. After training, each dyad watched 5 video clips per day and measured both the child’s behavior and TI. The percentages of IOA, kappas, and indices of dependability for assessment of the child’s behavior and TI were analyzed. The data revealed that all raters reached over 80% IOA and the whole interval (WI) and yes/no (Y/N) dyads reached .60 kappa, but the two raters in the Likert-type (LIK) dyad could not reach .60 kappa. The indices of dependability indicated that the six raters consistently observed and rated both the child’s behavior and TI, but there was a discrepancy in scores (i.e., percentages of TI) between the two raters in the two indirect measure dyads (i.e., Y/N and LIK). An analysis of the percentages of total variance showed that the two indirect TI methods may affect the discrepancy between the two raters’ rating scores. A comparison of the three different TI methods and correlation between the child’s behavior and TI were examined using the PASW Statistics 18 software program. There was no significant difference between the WI and the Y/N dyads, while the assessments from the LIK dyad indicated a significant difference from the other two dyads. Both the WI and the Y/N dyads showed correlations between the degree of the child’s behavior and the degree of TI, but there was no significant difference between the two correlation coefficients. Questions about reliability with the indirect TI measures suggest one should be careful in considering these results.
Type:
text; Electronic Dissertation
Keywords:
Behavioral intervention; Comparison; Treatment Integrity method
Degree Name:
Ph.D.
Degree Level:
doctoral
Degree Program:
Special Education; Graduate College
Degree Grantor:
University of Arizona
Committee Chair:
Umbreit, John

Full metadata record

DC FieldValue Language
dc.language.isoENen_US
dc.titleA Comparison of Treatment Integrity Assessment Methods for Behavioral Interventionen_US
dc.creatorKoh, Seong Aen_US
dc.contributor.authorKoh, Seong Aen_US
dc.date.issued2010en_US
dc.publisherThe University of Arizona.en_US
dc.rightsCopyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.en_US
dc.description.abstractThe purpose of this study was to examine the similarity of outcomes from three different treatment integrity (TI) methods, and to identify the method which best corresponded to the assessment of a child’s behavior. Six raters were recruited through individual contact via snowball sampling. A modified intervention component list and 19 video clips were derived from Stahr’s (2005) study, “An Intervention for Children with Autism Spectrum Disorders (ASD) who Have Food Selectivity.” The raters, randomly and evenly assigned to three dyads. Each dyad received an average of six hours training and reached 85% interobserver agreement (IOA) with a 0.60 kappa score. After training, each dyad watched 5 video clips per day and measured both the child’s behavior and TI. The percentages of IOA, kappas, and indices of dependability for assessment of the child’s behavior and TI were analyzed. The data revealed that all raters reached over 80% IOA and the whole interval (WI) and yes/no (Y/N) dyads reached .60 kappa, but the two raters in the Likert-type (LIK) dyad could not reach .60 kappa. The indices of dependability indicated that the six raters consistently observed and rated both the child’s behavior and TI, but there was a discrepancy in scores (i.e., percentages of TI) between the two raters in the two indirect measure dyads (i.e., Y/N and LIK). An analysis of the percentages of total variance showed that the two indirect TI methods may affect the discrepancy between the two raters’ rating scores. A comparison of the three different TI methods and correlation between the child’s behavior and TI were examined using the PASW Statistics 18 software program. There was no significant difference between the WI and the Y/N dyads, while the assessments from the LIK dyad indicated a significant difference from the other two dyads. Both the WI and the Y/N dyads showed correlations between the degree of the child’s behavior and the degree of TI, but there was no significant difference between the two correlation coefficients. Questions about reliability with the indirect TI measures suggest one should be careful in considering these results.en_US
dc.typetexten_US
dc.typeElectronic Dissertationen_US
dc.subjectBehavioral interventionen_US
dc.subjectComparisonen_US
dc.subjectTreatment Integrity methoden_US
thesis.degree.namePh.D.en_US
thesis.degree.leveldoctoralen_US
thesis.degree.disciplineSpecial Educationen_US
thesis.degree.disciplineGraduate Collegeen_US
thesis.degree.grantorUniversity of Arizonaen_US
dc.contributor.chairUmbreit, Johnen_US
dc.contributor.committeememberLiaupsin, Carlen_US
dc.contributor.committeememberFerro, Joleneaen_US
dc.contributor.committeememberMacFarland, Stephanie Z. C.en_US
dc.identifier.proquest10872en_US
dc.identifier.oclc659753788en_US
All Items in UA Campus Repository are protected by copyright, with all rights reserved, unless otherwise indicated.