Consistency of eighth graders classification into multiple performance levels in mathematics by using the bookmark method

Abstract

The study aims at identifying multiple performance levels of
Jordanian eighth graders in mathematics by using the bookmark
method, and determining the degree of consistency of students
classification according to item response theory model used in the
calibration of test items. To achieve the purposes of the
study, responses of 2490 students to 128 performance indicators test
items in mathematics were used. Using the appropriate software to
analyze the data and the Bookmark method, three cut-off scores have
been identified to differentiate among the four levels of
performance (below basic, basic, full proficient, and advanced),
according to one parameter logistic IRT model in the first case and
three parameter logistic IRT model in the second case. The results
showed that, a small percentage of students reached the “ Full
proficient” level, only 21% in case one and 7% in case two. The
results also showed a clear difference in cut-off scores between the
two cases and a poor degree of consistency between the two
classifications of students into four performance levels, where the
value of Kappa coefficient ( 0.095) is very low and not statistically
significant. However, it is noted that the agreement between the two
models becomes better if students are classified into two performance
levels.