University of Twente Student Theses
Robustness of sparse MLPs for supervised feature selection
Kichler, Neil (2021) Robustness of sparse MLPs for supervised feature selection.
|
PDF
803kB |
| Abstract: | Deep Neural Networks have seen great success yet require increasingly higher dimensional data to be applied successfully. To reduce the ever-increasing computational, energy and memory requirements, the concept of sparsity has emerged as a leading approach. Sparse-to-sparse training methods allow training and inference on more resource-limited devices. It has been hinted in previous work (SET and RigL) that such methods could be applied to feature selection since they may implicitly encode the input neuron strengths during training. However, a proper investigation of this potential idea has not taken place in the domain of supervised feature selection. This paper develops a method for supervised feature selection using Sparse Evolutionary Training applied to a Multi-Layer Perceptron (SET-MLP). The focus is on investigating the robustness of this feature selection mechanism to changes in the topology of SET-MLPs. We develop and perform an experimentally driven analysis on some prominent datasets to evaluate the generalizability, initialization-dependence and similarity of the underlying networks of the feature selection process. We find for the selected datasets that SET-MLP produces similar feature selections for different underlying network topologies and can recover from bad initialization. Our work provides a basis for understanding whether supervised feature selection using sparse training methods are robust to topological changes. The problem addressed can have further implications in understanding sparse training given that it visualizes some aspects of the random exploratory nature of these methods. Furthermore, it discusses the potential viability of sparse-to-sparse training methods for supervised feature selection. |
| Item Type: | Essay (Bachelor) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Computer Science BSc (56964) |
| Awards: | Best Paper Award (35th Twente Student Conference on IT) |
| Link to this item: | https://purl.utwente.nl/essays/86886 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page