University of Twente Student Theses
Examining the effect of hyperparameters on the training of a residual network for emotion recognition
Winters, A. (2020) Examining the effect of hyperparameters on the training of a residual network for emotion recognition.
|
PDF
655kB |
| Abstract: | Setting hyperparameters is a complex and hard task when training a neural network. Neural networks performing badly can be the effect of a sub-optimal hyperparameters setup before training. This study aims to examine the effects of hyperparameters on the training of a residual network, while using a small dataset. This network is a SE-ResNet-50 that is being used to recognize emotions from facial expressions. Different methods, both constant values as well as schedules, are compared based on the validation loss and validation accuracy. For interesting outcomes, an examination is done to see whether these methods transfer well to a similar dataset in the same context of emotion recognition in facial ex- pressions. The study shows that low batch sizes contribute to a good performance, but tend to result in an unstable network. A balance must be found between a high accuracy and a sta- ble training. Learning rate schedules that use small step sizes outperform those which make larger adjustments. Momentum is a valuable addition, but too high of a mo- mentum shows signs of overfitting in the loss graph. A cyclical momentum causes the training to become unsta- ble, this can be reduced by using higher batch sizes. |
| Item Type: | Essay (Bachelor) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Computer Science BSc (56964) |
| Link to this item: | https://purl.utwente.nl/essays/80565 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page