Combining Dynamic and Static Analysis for the Automation of Grading Programming Exams

Manual grading of programming exams can take a significant amount of time and funding, and may additionally be subject to human variances in grading. Autograders can solve the aforementioned problems. However, there are currently no autograders available that utilise both a combination of dynamic and static analysis and that link test criteria to the Intended Learning Outcomes of a module. Therefore, we propose a theoretical autograding model that utilises the aforementioned analysis techniques and links test criteria to Intended Learning Outcomes. Additionally, we offer guidance for creating grading criteria based on Intended Learning Outcomes, and we demonstrate a proof-of-concept implementation of the aforementioned model, called Thoth. We verify Thoth by comparing the grading of a selection of exercises from an introductory programming exam. With this verification, we demonstrate the potential for autograders to aid in (partially) grading introductory programming exams.