Methodological Considerations in the Development of Indicators of Achievement in the NAEP Data / Ronald E. Anderson and Others.

The organization of data at the National Assessment of Educational Progress (NAEP) is undergoing a significant transition from a system designed only for national assessment purposes to one designed both for assessment and a variety of academic research interests. The advent of NAEP public-use data files opens up many possibilities for those who have the skills, time, and resources to do secondary analysis. An analysis of the mathematics test items is presented which demonstrates alternate procedures for developing indicators of mathematics achievement. This analysis demonstrates that the NAEP item subsets will not always meet conventional psychometric criteria. This failure to meet standard achievement test criteria does not mean that secondary analysis of the data is unwise. It does imply, however, that interpretation of findings, especially those using subtests, must be made cautiously. Limitations of the methodology must be acknowledged. Conventional achievement testing is not item-centered like assessment testing. The measurement priority of assessment is stability across multiple testings, not relative comparisons among persons. Consequently, standards of item discrimination and construct validity have obviously less import. Of far greater importance for assessments are standards of face validity, content validity, internal consistency and the application of rigorous data analysis techniques. (Author/RL) Aspect of National Assessment (NAEP) dealt with in this document: Results (Secondary Analyses).

Microfiche. [Washington D.C.]: ERIC Clearinghouse microfiches : positive.