University of Twente Student Theses

Login
As of Friday, 8 August 2025, the current Student Theses repository is no longer available for thesis uploads. A new Student Theses repository will be available starting Friday, 15 August 2025.

Eigenvalues of the Neural Tangent Kernel for different Network Architectures

Berg, Niels (2024) Eigenvalues of the Neural Tangent Kernel for different Network Architectures.

[img] PDF
1MB
Abstract:For fully connected neural networks the width, depth, and activation functions used are usually picked using intuition of what has worked well before. It would be better if we can know what structure of a neural network works well before starting training. For this purpose we can calculate the Neural Tangent Kernel (NTK). We show that the speed of training can be bounded using the smallest eigenvalue of the NTK. We look at how the smallest eigenvalue changes between networks with different widths, depths, and activation functions, and also how this bound may not directly correlate with the speed of training.
Item Type:Essay (Bachelor)
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:31 mathematics
Programme:Applied Mathematics BSc (56965)
Link to this item:https://purl.utwente.nl/essays/100610
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page