Developing and Validating a Computerized Adaptive Testing System for Measuring the English Proficiency of Taiwanese EFL University Students

Abstract

Prompted by Taiwanese university students' increasing demand for English proficiency assessment, the absence of a test designed specifically for this demographic subgroup, and the lack of a localized and freely-accessible proficiency measure, this project set out to develop and validate a computerized adaptive English proficiency testing (E-CAT) system for Taiwanese EFL university students. Drawing on the guidelines posited by L2 testing specialists, we devised and followed a six-stage procedure to develop this E-CAT system: determining the test purpose, defining the construct, developing the test items, designing the administration processes, performing the field testing, and constructing the E-CAT system. Upon its completion, we performed two validation studies on the simulated data and as such offered the backing for the generalization inference and the explanation inference, which combined to lend support for the validity argument for the E-CAT score-based interpretations and uses. This project highlighted the importance of test localization, foregrounded the utility of computerized adaptive testing and item response theory in language test development and validation, and generated a localized, free-of-charge English proficiency test for Taiwanese university students to satisfy the graduation benchmark requirement and/or to demonstrate their English proficiency levels when job hunting.