University of Twente Student Theses

Login

Sentence Simplification using Syntactic Features and T5 model

Sara Michael Iyasu, Sara (2025) Sentence Simplification using Syntactic Features and T5 model.

[img] PDF
1MB
Abstract:Syntactic simplification seeks to alter a sentence’s grammatical structure while ensuring its meaning is maintained, thereby making it more understandable for a wider audience. Despite being trained on extensive datasets, large language models mainly capture broad language patterns and often find it challenging to maintain nuanced meanings in complex sentences. This limitation can result in oversimplification or ambiguity. Therefore, it is crucial to strike a proper balance between simplification and correctness. This research introduces an improvement to the attention mechanism by computing attention scores based on both hidden states and the grammatical relationships between words, rather than relying solely on hidden states as traditional attention mechanisms do. The paper also explains a second approach that incorporates a grammatical embedding layer to enhance the model’s understanding of grammatical structure. The paper investigates various methods for incorporating explicit grammatical information into the model. It adopts two strategies: one focuses on integrating grammatical information, while the other emphasizes finetuning the model for specific downstream tasks, which in this case is syntactic simplification. Fine-tuning the T5 model without explicit grammatical information yielded best results, with a rouge-1 score of 0.96, rouge-2 score of 0.9266, and R-L score of 0.9554 evaluated by ROUGE score.
Item Type:Essay (Bachelor)
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:54 computer science
Programme:Computer Science BSc (56964)
Link to this item:https://purl.utwente.nl/essays/105090
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page