University of Twente Student Theses
Predictive coding with generalized losses and its mathematical relationship with backpropagation
Sun, Wei-Ting (2024) Predictive coding with generalized losses and its mathematical relationship with backpropagation.
|
PDF
776kB |
| Abstract: | Backpropagation is the supervised learning algorithm that has underpinned the continued success of neural network models in a wide range of applications. However, it has not found success as a model for learning in the biological neural network of the brain, as the algorithm is generally considered to be biologically implausible. Predictive coding algorithms, on the other hand, are a promising class of learning algorithms that have a biological basis and have been shown to approximate backpropagation. One problem is that so far, they have mostly been formulated only for squared loss functions. In this paper, we present a framework for predictive coding algorithms with general loss functions, and we prove that for certain classes of loss functions, predictive coding approximates — and in a certain limit is equal to — backpropagation. Our results pave the way for predictive coding to be applied on more complex neural network architectures and machine learning tasks, and further closes the gap between biologically realistic models of learning and artificial neural networks. |
| Item Type: | Essay (Master) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 31 mathematics |
| Programme: | Applied Mathematics MSc (60348) |
| Link to this item: | https://purl.utwente.nl/essays/102576 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page