Do test takers differ in ability?
Psychometrics deals with individual differences. Can we test whether test takers differ in ability to begin with? The open-source software package dexter, that we developed with two friends at Cito, has a function called ๐ข๐ง๐๐ข๐ฏ๐ข๐๐ฎ๐๐ฅ_๐๐ข๐๐๐๐ซ๐๐ง๐๐๐ฌ that allows you to do precisely this. If no differences in ability are found, observed differences in test scores are random. An IRT model will fit perfectly, yet we measure nothing.
Under the hood, the function works by estimating a single ability for all persons. Dexter’s documentation does not mention that this adds an important possibility to ๐ฌ๐ญ๐๐ง๐๐๐ซ๐ ๐ฌ๐๐ญ๐ญ๐ข๐ง๐ that is not available elsewhere. To wit, we can test whether judges have actually been able to generate passing scores for one, and only one, (border-line) candidate. If so, observed discrepancies between judges are as expected. Otherwise, the judges either agree too much, or too little.
Wish to know more? Contact us at info@metior.consulting
0 reacties