University of Twente Student Theses

Login

Applicability of comparative judgement as alternative to rubric-based assessment for high-stake exam items that differ in maximum score

Valk, J. (2021) Applicability of comparative judgement as alternative to rubric-based assessment for high-stake exam items that differ in maximum score.

[img] PDF
875kB
Abstract:To adequately assess candidates’ competences, candidates must provide proof of performance through answering items on a test. These items are often assessed through rubric-based assessments (RBA). However, studies have indicated that RBAs are not sufficiently reliable (< 0.7) for high-stake assessments. An alternative method, comparative judgement (CJ) had been introduced to tackle this reliability issue. In this study, three exam items with a maximum score of 2, 3, and 4, were assessed through CJ. The results of this study indicated a high reliability for each of the assessments. The mean time investment for raters increased as the maximum score of the items increased. Furthermore, the rank correlations between RBA and CJ were moderately positive, and the correlation increased with the increase of the maximum score. Although these results indicate that CJ might be applicable for high-stake exam items that vary in maximum score, there are some considerations. First, this study focused on single items. However, exams exist of multiple items, and it seemed inefficient to apply CJ for every single item. Secondly, the assessment method of CJ does not offer transparency towards the candidates. Lastly, it seems CJ is biased towards the differences in the candidate pool sample.
Item Type:Essay (Master)
Clients:
eX:plain
Faculty:BMS: Behavioural, Management and Social Sciences
Subject:81 education, teaching
Programme:Educational Science and Technology MSc (60023)
Link to this item:https://purl.utwente.nl/essays/89390
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page