The Estimation of Item Difficulty from Restricted CAT Calibration Samples.Report as inadecuate

The Estimation of Item Difficulty from Restricted CAT Calibration Samples. - Download this document for free, or read online. Document in PDF available to download.

Whether the presence of bidimensionality has any effect on the adaptive recalibration of test items was studied through live-data simulation of computer adaptive testing (CAT) forms. The source data were examinee responses to the 298 scored multiple choice items of a licensure examination in a health care profession. Three 75-item part-forms, differing in the degree of bidimensionality, were constructed from the examination forms. The dimensionality of each item was determined based on principal factor loadings using W. F. Stout's (1987) procedure. Samples of 2,100 examinees from an U.S.-educated group and a predominantly foreign-educated group were used, with samples sizes of 100, 200, 400, and 1,000 used for the analyses. As expected, an increase in sample size resulted in greater agreement between calibration and bank b-values. It appeared that dimensionality affected adaptive recalibration. Results from this study varied depending on the dimensionality of the part-forms. Results also affirm the importance of a well-defined reference group for recalibration, since samples from varying ability ranges sometimes produced considerably different b-values. Three tables present details of the analyses. (Contains six references.) (SLD)

Descriptors: Adaptive Testing, Computer Assisted Testing, Difficulty Level, Estimation (Mathematics), Factor Analysis, Licensing Examinations (Professions), Multiple Choice Tests, Sample Size, Test Format, Test Items

Author: Sykes, Robert C.; Ito, Kyoko


Related documents