University of Twente Student Theses
The smart annotation tool : optimizing semi-automated behavioural annotation using an AutoML framework supported by classification correctness prediction
Jutte, Annemarie (2022) The smart annotation tool : optimizing semi-automated behavioural annotation using an AutoML framework supported by classification correctness prediction.
PDF
5MB |
Abstract: | The manual annotation of behaviour is a time-consuming process,(semi-)automated methods could be employed to speed up the process. In this research, a tool is presented that can be used to quickly create annotations by optimizing human-machine interaction. With the help of the user and methods from AutoML, models for behavioural classification are trained. The increase of efficiency reached through semi-automation may only come at a limited loss in the quality of the annotation. In this research, classification correctness prediction methods are used to control the annotation quality. Only the samples a model for classification is expected to be certain about are automatically annotated. The tool resulting from these principles is the Smart Annotation Tool. The aim is to increase usability, compared to fully manual approaches, with a limited loss of quality. Results are presented through automatic experiments on several datasets. Small-scale user studies are conducted to gather information regarding user satisfaction. The Smart Annotation Tool comes close to creating annotations of controlled quality for some datasets. More research is required to preserve quality for all datasets. In future research, approaches that focus on obtaining more representative data should be used to increase the quality and the efficiency of the tool. |
Item Type: | Essay (Master) |
Clients: | Noldus Information Technology |
Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
Subject: | 54 computer science |
Programme: | Interaction Technology MSc (60030) |
Link to this item: | https://purl.utwente.nl/essays/92565 |
Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Repository Staff Only: item control page