Using Generalizability Theory to Detect the ‎Contribution of Multiple Sources of Variance in the ‎Validity of a Test in Mathematics

Abstract

This study aimed to detect the contribution of multiple sources ‎of variance in the validity of a test in mathematics by using the ‎Generalizability Theory, through estimating the magnitude of ‎error variance that is explained by the facets (tasks, task ‎formulae and correction method) in the total variance. The study ‎sample consisted of (301) students from the fifth grade who ‎were subjected to a mathematics test that consisted of (12) tasks ‎in the domain of numbers and operations on them. The tasks ‎were equally distributed on (4) formulae (application, inference, ‎selection and opinion). The researchers used the designs ‎‎(person×formula), (person×task× formula), (person×method) ‎and (person×task×method) completely crossed and used ‎‎(EduG) software to analyze the data, The results of ‎generalizability studies indicated that the largest sources of error ‎variance were in the design (person×formula) which refers to ‎the interaction (person-formula) and in the design ‎‎(person×task×formula) which refers to the interaction (person-‎task-formula). The generalizability coefficients in the design ‎‎(person×formula) were better than in the design ‎‎(person×task×formula). The results of the study also found that ‎the largest sources of error variance were in the design ‎‎(person×method) which refers to the interaction (person-‎method) and in the design (person×task×method) ehich refers ‎to the interaction (person-task-method). The generalizability ‎coefficients in the design (person×task×method) were better ‎than in the design (person×method).‎

JJES,18(3), 2022, 569-583