University of Twente Student Theses

Login

Examining the effect of hyperparameters on the training of a residual network for emotion recognition

Winters, A. (2020) Examining the effect of hyperparameters on the training of a residual network for emotion recognition.

[img] PDF
655kB
Abstract:Setting hyperparameters is a complex and hard task when training a neural network. Neural networks performing badly can be the effect of a sub-optimal hyperparameters setup before training. This study aims to examine the effects of hyperparameters on the training of a residual network, while using a small dataset. This network is a SE-ResNet-50 that is being used to recognize emotions from facial expressions. Different methods, both constant values as well as schedules, are compared based on the validation loss and validation accuracy. For interesting outcomes, an examination is done to see whether these methods transfer well to a similar dataset in the same context of emotion recognition in facial ex- pressions. The study shows that low batch sizes contribute to a good performance, but tend to result in an unstable network. A balance must be found between a high accuracy and a sta- ble training. Learning rate schedules that use small step sizes outperform those which make larger adjustments. Momentum is a valuable addition, but too high of a mo- mentum shows signs of overfitting in the loss graph. A cyclical momentum causes the training to become unsta- ble, this can be reduced by using higher batch sizes.
Item Type:Essay (Bachelor)
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:54 computer science
Programme:Computer Science BSc (56964)
Link to this item:http://purl.utwente.nl/essays/80565
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page