University of Twente Student Theses


Exploring automatic scoring of an observation-based multimedia-based performance assessment

Ydo, E.J. (2020) Exploring automatic scoring of an observation-based multimedia-based performance assessment.

Full text not available from this repository.

Full Text Status:Access to this publication is restricted
Abstract:Performance-based assessment (PBA) is the most used assessment method in vocational education. However, the reliability and validity of PBA are questioned, mainly because of assessor, occasion, and task induced errors. Simulation-based assessment (SBA), especially with the use of technology, is becoming an increasingly popular assessment method for vocational education because it can strengthen and broaden this domain of assessment. This is especially the case for multimedia-based performance assessment (MBPA) which is a fully virtual type of SBA. Because of this, MBPA is more standardizable among other advantages. This study explored how to validly standardize a PBA by creating an observation-based MBPA using VR-technology for technical vocational education. The study consisted of two parts. First, an observation-based MBPA and an automated algorithm were designed and developed using the MBPA framework of de Klerk, Veldkamp, and Eggen (2018), and a focus group of subject matter experts. During the second part of the study, data in the form of video captures were gathered from participants who performed in the observation-based MBPA. The scoring of this data by two subject matter experts was compared with the scores of the automated algorithm. A high correlation was found between the experts and the algorithm. Furthermore, variation between the experts showed assessor induced errors, which as the results also showed, could even cause unjustifiable scoring of performance on high-risk tasks. This study laid a basis to further explore the automatization of observation-based MBPA for more complex assessments.
Item Type:Essay (Master)
SeriousVR, Enschede, NL
Faculty:BMS: Behavioural, Management and Social Sciences
Subject:70 social sciences in general, 81 education, teaching
Programme:Educational Science and Technology MSc (60023)
Link to this item:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page