University of Twente Student Theses
The smart annotation tool : optimizing semi-automated behavioural annotation using an AutoML framework supported by classification correctness prediction
Jutte, Annemarie (2022) The smart annotation tool : optimizing semi-automated behavioural annotation using an AutoML framework supported by classification correctness prediction.
|
PDF
5MB |
| Abstract: | The manual annotation of behaviour is a time-consuming process,(semi-)automated methods could be employed to speed up the process. The data recorded and the behaviour classes that need to be annotated vary from task to task. This means that custom models need to be created for specific tasks. In this research, a tool is presented that can be used to quickly create annotations by optimizing human-machine interaction. With the help of the user and methods from AutoML (the field of AI that aims to automatically build machine learning systems), models for behavioural classification are trained. The increase of efficiency reached through semi-automation may only come at a limited loss in the quality of the annotation. In this research, classification correctness prediction methods are used to control the annotation quality. Only the samples a model for classification is expected to be certain about are automatically annotated. The tool resulting from these principles is the Smart Annotation Tool. The aim is to increase usability, compared to fully manually annotation approaches, with a limited loss of quality. Results are presented through automatic experiments on several datasets. Small-scale user studies are conducted to gather information regarding user satisfaction. The Smart Annotation Tool comes close to creating annotations of controlled quality for some datasets. More research is required to preserve quality for all datasets. In future research, approaches that focus on obtaining more representative data should be used to increase both the quality and the efficiency of the tool. |
| Item Type: | Essay (Master) |
| Clients: | Noldus Information Technology |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Computer Science MSc (60300) |
| Link to this item: | https://purl.utwente.nl/essays/92625 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page