University of Twente Student Theses

Login

Diversifying Multilayer Perceptron Ensembles in a Truly Sparse Context

Wal, P.R.D. van der (2023) Diversifying Multilayer Perceptron Ensembles in a Truly Sparse Context.

[img] PDF
1MB
Abstract:Artificial Neural Networks are state-of-the-art machine learning models, outperforming their competitors in many fields. One of the major drawbacks of Artificial Neural Networks are the long training times as a result of computationally expensive calculations. Sparse models aim to remove redundant parameters whilst maintaining good levels of performance. Ensembles of several weak learners have shown to be able to outperform individual networks. Crucial for the performance of the ensemble is the diversity of the individual subnetworks of which the ensemble is constructed. The work that has been done on the inter- section of sparse- and ensemble learning, does not provide actual benefits in terms of computational overhead as the sparsity is simulated using binary masks. In this paper, we propose two algorithms that promote diversity among individual ensemble members. We implement these two algorithms for a first-of-its-kind Truly Sparse Ensemble 1. We demonstrate the performance of the model at several high levels of sparsity on numerous datasets in terms of classification accuracy, Floating Point Operations (FLOPs), and running time. Moreover, we provide insight into the impact of our two diversification algorithms on the training trajectory and topological similarity of the subnetworks. Suggestions for future research are discussed as well.
Item Type:Essay (Master)
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:54 computer science
Programme:Computer Science MSc (60300)
Link to this item:https://purl.utwente.nl/essays/94636
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page