Using Binomial Trials Model for Examining the Judges’ Ratings of Cut Scores for a Criterion-Referenced Mathematics Test

Abstract

The study aimed at investigating the quality of judges’
rating of the cut score for a criterion-referenced test according to
Angoff Method using Binomial Trials model. To achieve this purpose,
marks of 110 tenth grade students on a 30- item criterion- referenced
test in mathematics were analyzed. 15 judges’ ratings were also
analyzed in three rounds of judgments. The results of the study
indicated that there were significant differences (α = 0.05) among cut
score estimates which can be attributed to judgment round. After
eliminating the judges’ ratings which did not fit the model
expectations, the differences among the estimated cut scores
decreased in all rounds while there was an increase in the hit rate for
students’ classification.