University of Twente Student Theses
Investigating the use of hub neuron identification for pruning sparse neural networks
Heath, Andrew (2021) Investigating the use of hub neuron identification for pruning sparse neural networks.
|
PDF
820kB |
| Abstract: | Sparsity constraints decrease computation time and memory requirements for Artificial Neural Networks (ANN). Further research has shown that pruning during each epoch based on accuracy shows similar improvements. Research on pruning has brought ANNs closer in line with their biological counterparts. However, the formation and reinforcement of hub neurons, as seen in a brain, has not been explored entirely in ANNs. Reinforcement of said hub neurons could reduce the time and memory requirements of network training pruning algorithms. This research investigates the Laplacian centrality of neurons in ANNs and Sparse Neural Networks (trained using SET) during training, showing changes in the distribution of Laplacian centrality of the neurons. We propose an ANN training method, CenSET, that uses Laplacian centrality to instruct pruning of connections during the training of a sparse ANN. We show that this approach does not dramatically decrease the accuracy compared to training an ANN using a conventional MLP or SET approach. |
| Item Type: | Essay (Bachelor) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Computer Science BSc (56964) |
| Awards: | Best paper nomination |
| Link to this item: | https://purl.utwente.nl/essays/87360 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page