The Accuracy of Examinee Judgments of Relative Item Difficulty : Implications for Computerized Adaptive Testing / Steven L. Wise and Others.

The degree to which item review on a computerized adaptive test (CAT) could be used by examinees to inflate their scores artificially was studied. G. G. Kingsbury (1996) described a strategy in which examinees could use the changes in item difficulty during a CAT to determine which items' answers are incorrect and should be changed during item review. In CAT, a correctly answered item will typically be followed by a more difficult item, while an incorrectly answered item will typically be followed by an easier item. The results of two studies involving groups of 77 and 62 undergraduates suggest that examinees are not highly proficient at discriminating item difficulty, a skill needed for successful application of the Kingsbury strategy. In the third study, which used 243 introductory statistics students, the Kingsbury strategy, which examinees would use only for guessed items, was compared to a generalized strategy used for all sequential item pairs. The Kingsbury strategy yielded a small average score gain, while the generalized strategy yielded an average score loss. These results suggest that only the Kingsbury strategy would enable examinees to inflate their scores successfully. (Contains 2 tables, 3 figures, and 14 references.) (Author/SLD)

Microfiche. [Washington D.C.]: ERIC Clearinghouse microfiches : positive.