Automated Trait Scores for TOEFL® Writing Tasks. Research Report. ETS RR-15-14Reportar como inadecuado

Automated Trait Scores for TOEFL® Writing Tasks. Research Report. ETS RR-15-14 - Descarga este documento en PDF. Documentación en PDF para descargar gratis. Disponible también para leer online.

ETS Research Report Series, Jun 2015

The "e-rater"® automated essay scoring system is used operationally in the scoring of "TOEFL iBT"® independent and integrated tasks. In this study we explored the psychometric added value of reporting four trait scores for each of these two tasks, beyond the total e-rater score.The four trait scores are word choice, grammatical conventions, fluency and organization, and content. Trait scores were computed on the basis of several criteria for determining feature weights: regression parameters of the trait features on human scores, reliability of trait features, and coefficients of features from a principal component analysis. In addition, augmented trait scores, based on information from other trait scores, were also analyzed.The psychometric added value of trait scores beyond total e-rater scores was evaluated by comparing the ability to predict a particular trait score on one task from the same trait score on the other task versus the e-rater score on the other task. Results supported the use of trait scores, and are discussed in terms of their contribution to the construct validity of e-rater as an alternative essay scoring method.

Descriptors: Writing Tests, Scores, Language Tests, English (Second Language), Second Language Learning, Essays, Writing Evaluation, Computer Assisted Testing, Evaluators, Evaluation Criteria, Regression (Statistics), Psychometrics, Prediction, Comparative Analysis, Construct Validity, Scoring, Correlation, Factor Analysis, Weighted Scores

Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb[at]; Web site:

Autor: Attali, Yigal; Sinharay, Sandip


Documentos relacionados