Assessing the Efficiency of Item Selection in Computerized Adaptive Testing.Report as inadecuate

Assessing the Efficiency of Item Selection in Computerized Adaptive Testing. - Download this document for free, or read online. Document in PDF available to download.

This study investigated the efficiency of item selection in a computerized adaptive test (CAT), where efficiency was defined in terms of the accumulated test information at an examinee's true ability level. A simulation methodology compared the efficiency of 2 item selection procedures with 5 ability estimation procedures for CATs of 5, 10, 15, and 25 items in length. The two item selection procedures included maximum Fisher information (FI) and maximum Fisher interval information (FII) item selection. The five ability estimation procedures included maximum likelihood, modal a posteriori (MAP), golden section search (GSS), and two procedures proposed in this study. These were ML/Alt and MAP/Alt, adjusted ML or MAP estimates according to a specific decision rule based on hypothesis testing. For the conventional item selection procedure (FI) and ability estimation procedures (ML and MAP), the best performance was observed for FI with MAP at middle ability levels, with efficiency attaining or exceeding 90% even for the shortest test length. In contrast, large gaps in efficiency were observed for FI with MAP at extreme ability levels, and for FI with ML across al ability levels. Utilizing FII item selection with ML and MAP narrowed the gaps in efficiency at the lowest ability levels for 5- and 10-item tests. The greatest increase in test efficiency was observed when the alternative ability estimation procedures (ML/Alt, MAP/Alt, and GSS) were used. The gains in efficiency were most pronounced for shorter tests, but were noticeable even for longer tests. Overall, it appears that the ability estimation procedure impacts the efficiency of item selection to a large extent than the item selection procedure. (Author/SLD)

Descriptors: Ability, Adaptive Testing, Computer Assisted Testing, Maximum Likelihood Statistics, Selection, Simulation, Test Construction, Test Items

Author: Weissman, Alexander


Related documents