University of Twente Student Theses
Eigenvalues of the Neural Tangent Kernel for different Network Architectures
Berg, Niels (2024) Eigenvalues of the Neural Tangent Kernel for different Network Architectures.
|
PDF
1MB |
| Abstract: | For fully connected neural networks the width, depth, and activation functions used are usually picked using intuition of what has worked well before. It would be better if we can know what structure of a neural network works well before starting training. For this purpose we can calculate the Neural Tangent Kernel (NTK). We show that the speed of training can be bounded using the smallest eigenvalue of the NTK. We look at how the smallest eigenvalue changes between networks with different widths, depths, and activation functions, and also how this bound may not directly correlate with the speed of training. |
| Item Type: | Essay (Bachelor) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 31 mathematics |
| Programme: | Applied Mathematics BSc (56965) |
| Link to this item: | https://purl.utwente.nl/essays/100610 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page