Judgments of academic achievement by teachers and standardized, norm-referenced tests revisited : an issue of educational and political policy

No Thumbnail Available
Peters, Richard G.
Treloar, James H.
Issue Date
Thesis (Ph. D.)
Department of Educational Psychology
Other Identifiers

The purpose of this study was to investigate the degree of concurrence between teachers' judgments of the academic achievement of students and the results of standardized,norm-referenced achievement tests. Although this issue had been addressed before, results reported in the literature lacked a sensitivity to the informational needs of educational policy makers and were obfuscated by significant differences in research design and analytical techniques. This study attempted to address the potential moderating effect of teachers' pre-established notions of students' knowledge, academic subject area, grade level, and student gender on the agreement level between teachers' judgment of student achievement and test results, while focusing on the ever increasing use of test scores to make decisions regarding student readiness for promotion/graduation and overall school accountability.Approximately 670 teachers were asked to rate their students as "not ready to succeed at the next grade level without remedial assistance" (non-masters) or "ready to succeed without additional instruction or intervention" (masters). Ratings were obtained in both English/language arts and mathematics for 15,935 students in grades 1, 2, 3, 6, and 8. The sample utilized was representative of the demographics of the state of Indiana. While appropriate statistical tests of significance were performed when appropriate, this study focused on effect size as the final determinant of "educational significance."Analyses revealed no practical reason to believe that teachers' judgments were influenced by their initial ratings of students as masters or non-masters, student gender, grade level, or subject matter. On the average, teachers' mastery/non-mastery ratings were found to agree with "cutscores" established through discriminant analysis in about 78% of the cases. These results were seen as encouraging, in that test results could be used to support teacher judgment, which seemed unaffected by moderating variables, while not offering information completely redundant with pre-existing teacher knowledge of student achievement.