University of Twente Student Theses
Attention Mechanisms in Natural Language Processing
Dieten, J. J. van (2024) Attention Mechanisms in Natural Language Processing.
|
PDF
1MB |
| Abstract: | This paper examines the impact of attention mechanisms starting from their initial use cases up to their present-day roles in the field of Natural Language Processing (NLP). Traditional machine learning models struggle to capture contextual dependencies, which have been largely resolved by incorporating attention mechanisms into NLP models. Through an exploration of attention mechanisms, this research offers a comprehensive overview, looking into their evolutionary trajectory, performance enhancements, inherent limitations, and visualization techniques. Key findings highlight the remarkable performance improvements brought by attention mechanisms, particularly evident in tasks like Machine Translation and Sentiment Analysis. Challenges, including computational complexity and interpretability, are discussed, providing insights into the more nuanced landscape of attention in NLP. |
| Item Type: | Essay (Bachelor) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Computer Science BSc (56964) |
| Link to this item: | https://purl.utwente.nl/essays/98223 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page