University of Twente Student Theses
As of Friday, 8 August 2025, the current Student Theses repository is no longer available for thesis uploads. A new Student Theses repository will be available starting Friday, 15 August 2025.
Dynamic Parameter Rank Pruning of a Singular Value Decomposed Multilayer Perceptron
Stribos, W.H. (2025) Dynamic Parameter Rank Pruning of a Singular Value Decomposed Multilayer Perceptron.
|
PDF
1MB |
| Abstract: | Neural networks are an extremely impressive technology, allowing computers to model the real world at a level of effectiveness not previously thought possible. However, more complex ones require massive amounts of computing power and storage, creating a limit to their potential. Thus, much research is done into finding ways to decrease the computational efficiency and memory usage of more complex models without losing functionality. One of these methods is the post-training compression technique 'singular value decomposition', which approximates a weight matrix with smaller matrices. To avoid the necessity of extensive retraining, the paper proposes a dynamic implementation of the same mathematical concept, which starts training with full-size decomposed matrices and prunes itself during training. Through batched pruning and a relaxed orthogonality constraint, the final system effectively decreases model size during training while increasing overall accuracy. |
| Item Type: | Essay (Bachelor) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Computer Science BSc (56964) |
| Link to this item: | https://purl.utwente.nl/essays/107510 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page