University of Twente Student Theses
Sparse artificial neural networks : Adaptive performance-based connectivity inspired by human-brain processes
Lapshyna, V. (2020) Sparse artificial neural networks : Adaptive performance-based connectivity inspired by human-brain processes.
|
PDF
675kB |
| Abstract: | Artificial Neural Networks are powerful machine learning systems. However, a high number of weights close to zero make networks unnecessary large and heavy. Sparse models remove redundant weights, aiming to decrease the number of parameters with minimal loss in accuracy. Sparse Evolutionary Training procedure adaptively evolves weights of the Artificial Neural Network topology. This technique proves to remove a vast number of weights and achieve higher accuracy than its non-evolutionary or densely connected counterparts, although the connection addition and removal follows a relatively simple algorithm. Inspired by the synaptic pruning in the human brain, we propose an advanced approach for weight evolution in the Sparse Evolutionary Training algorithm. We suggest gradually removing connections during the training phase as the accuracy increases. We show that the number of parameters can be significantly reduced with almost no loss in accuracy and negligible additional computational complexity. We demonstrate the performance of the algorithm on the Multilayer Perceptron trained on benchmark image and tabular datasets. This research contributes to the understanding of Sparse Artificial Neural Networks and makes a step towards more efficient models. |
| Item Type: | Essay (Bachelor) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Business & IT BSc (56066) |
| Link to this item: | https://purl.utwente.nl/essays/80559 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page