Establishing the Validity of the Task-Based English Speaking Test (TBEST) for International Teaching Assistants

Persistent Link:
http://hdl.handle.net/10150/195181
Title:
Establishing the Validity of the Task-Based English Speaking Test (TBEST) for International Teaching Assistants
Author:
Witt, Autumn
Issue Date:
2010
Publisher:
The University of Arizona.
Rights:
Copyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.
Abstract:
This dissertation follows an oral language assessment tool from initial design and implementation to validity analysis. The specialized variables of this study are the population: international teaching assistants and the purpose: spoken assessment as a hiring prerequisite. However, the process can easily be applied to other populations and assessment goals.While evaluating the TBEST (Task-Based English Speaking Test) and TAST (TOEFL Academic Speaking Test), I search for a preponderance of evidence for assessment validity that indicate the most appropriate tool for evaluating potential ITAs. The specific evidences of assessment validity that are examined are:1. Evidence of Domain (Content) Validity: Which test, the TBEST or the TAST most closely measures the actual skills needed to be an ITA?2. Evidence of Predictive Criterion Validity: Which test, the TBEST or the TAST, is more valid in predicting ITA teaching success based on end of semester student evaluation (TCEs)?Following the analyses of these points of evidence, the results of a follow-up survey of ITA impressions about the ITA training and evaluating process are reviewed. Reviewing the results of this survey places the language assessment and hiring process recommendations within its larger context, directing attention toward suggestions for improvement of ITA training and evaluating procedures.Over the course of 18 months, 335 ITAs were assessed using the TBEST. 193 ITAs took the TAST prior to taking the TBEST, and those scores are used for correlation analysis. 119 ITAs participated in a follow up survey about their ITA experience.Analysis of domain validity shows that the TBEST is better suited for assessing ITAs than the TAST due to specialized assessment content not present on the more generic TAST. The TBEST is marginally better at predicting teaching success, though the results were statistically insignificant and recommendations are made for a follow-up study. Post-hoc analysis of the discriminative utility of both tests show that the TBEST results show more useful shades of distinction between candidates while the TAST results place the majority of students in a `fair' category which requires secondary interviews to assess teaching ability.
Type:
text; Electronic Dissertation
Keywords:
Domain Validity; ESL; Hiring Assessment; International Teaching Assistants; Oral Langauge Assessment; Spoken English Test
Degree Name:
Ph.D.
Degree Level:
doctoral
Degree Program:
Second Language Acquisition & Teaching; Graduate College
Degree Grantor:
University of Arizona
Advisor:
Liu, Jun
Committee Chair:
Liu, Jun

Full metadata record

DC FieldValue Language
dc.language.isoENen_US
dc.titleEstablishing the Validity of the Task-Based English Speaking Test (TBEST) for International Teaching Assistantsen_US
dc.creatorWitt, Autumnen_US
dc.contributor.authorWitt, Autumnen_US
dc.date.issued2010en_US
dc.publisherThe University of Arizona.en_US
dc.rightsCopyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.en_US
dc.description.abstractThis dissertation follows an oral language assessment tool from initial design and implementation to validity analysis. The specialized variables of this study are the population: international teaching assistants and the purpose: spoken assessment as a hiring prerequisite. However, the process can easily be applied to other populations and assessment goals.While evaluating the TBEST (Task-Based English Speaking Test) and TAST (TOEFL Academic Speaking Test), I search for a preponderance of evidence for assessment validity that indicate the most appropriate tool for evaluating potential ITAs. The specific evidences of assessment validity that are examined are:1. Evidence of Domain (Content) Validity: Which test, the TBEST or the TAST most closely measures the actual skills needed to be an ITA?2. Evidence of Predictive Criterion Validity: Which test, the TBEST or the TAST, is more valid in predicting ITA teaching success based on end of semester student evaluation (TCEs)?Following the analyses of these points of evidence, the results of a follow-up survey of ITA impressions about the ITA training and evaluating process are reviewed. Reviewing the results of this survey places the language assessment and hiring process recommendations within its larger context, directing attention toward suggestions for improvement of ITA training and evaluating procedures.Over the course of 18 months, 335 ITAs were assessed using the TBEST. 193 ITAs took the TAST prior to taking the TBEST, and those scores are used for correlation analysis. 119 ITAs participated in a follow up survey about their ITA experience.Analysis of domain validity shows that the TBEST is better suited for assessing ITAs than the TAST due to specialized assessment content not present on the more generic TAST. The TBEST is marginally better at predicting teaching success, though the results were statistically insignificant and recommendations are made for a follow-up study. Post-hoc analysis of the discriminative utility of both tests show that the TBEST results show more useful shades of distinction between candidates while the TAST results place the majority of students in a `fair' category which requires secondary interviews to assess teaching ability.en_US
dc.typetexten_US
dc.typeElectronic Dissertationen_US
dc.subjectDomain Validityen_US
dc.subjectESLen_US
dc.subjectHiring Assessmenten_US
dc.subjectInternational Teaching Assistantsen_US
dc.subjectOral Langauge Assessmenten_US
dc.subjectSpoken English Testen_US
thesis.degree.namePh.D.en_US
thesis.degree.leveldoctoralen_US
thesis.degree.disciplineSecond Language Acquisition & Teachingen_US
thesis.degree.disciplineGraduate Collegeen_US
thesis.degree.grantorUniversity of Arizonaen_US
dc.contributor.advisorLiu, Junen_US
dc.contributor.chairLiu, Junen_US
dc.contributor.committeememberReinhardt, Jonathanen_US
dc.contributor.committeememberSabers, Darrellen_US
dc.identifier.proquest10953en_US
dc.identifier.oclc659754870en_US
All Items in UA Campus Repository are protected by copyright, with all rights reserved, unless otherwise indicated.