Reliability of Performance Assessments : Let's Make Sure We Account for the Errors / Michael S. Trevisan.

Some of the issues regarding the estimation of reliability for performance assessments are explored, and a methodology is suggested for determination of reliability. In performance assessment, the magnitude of the reliability coefficient may be less than that obtained for a standardized test, since part of the potential use of performance assessment is to obtain outcome data for targets difficult to assess with standardized tests. A further problem is the lack of training of many assessment specialists with regard to performance assessment. In addition, reliability may refer to more than internal consistency. There is a growing consensus that the intraclass correlation coefficient (ICC) provides the best means for establishing reliability for quantitative observation data. Three basic ICC models are discussed: (1) one-way analysis of variance; (2) two-way random effects; and (3) a two-way mixed model. The use of ICC methods is discussed, and computer softwares for ICC analyses are described. Four tables illustrate the discussion, and a 26-item list of references is included. (SLD)

Microfiche. [Washington D.C.]: ERIC Clearinghouse microfiches : positive.