University of Twente Student Theses
From Pit Stops to Paragraphs : Automatic Generation of Formula One Live Blogs based on Structured Race Data
Bonenkamp, Jetske (2024) From Pit Stops to Paragraphs : Automatic Generation of Formula One Live Blogs based on Structured Race Data.
|
PDF
4MB |
| Abstract: | Live blogs have been undergoing a major increase in popularity in the field of Formula One in recent years. This thesis explores the generation of such blog posts based on structured race data by deploying natural language generation techniques. The study emphasizes the importance of accuracy and perceived level of entertainment in its objective. Using the context of Formula One and text generation techniques, as well as the principles of news writing and (sports) commentary, a system architecture was proposed that uses three main components: an event identification model, a blog generation model, and a rephrasing model. The first mentioned component used a rule-based approach to extract noteworthy events from a race data set, which was composed by extracting numerical data from TFeed, a Russian platform providing race statistics and visuals, and race control messages from the live timing feature of Formula One. It was evaluated by comparing the events mentioned in the posts on Autosport.com to the identified events, and achieved a 70% overlap, with the most crucial actions (overtakes, pit stops, retirements, and car events) being correctly identified on nearly all occasions. The output of the event identifier was used as input to the blog generation component, which was based on a large language model pre-trained for data-to-text generation. The model was fine-tuned to the Formula One live blog generation task on a set of input-output pairs, established using scraped posts from Autosport.com and a linguistic feature extraction model to extract structured data from those posts. The fine-tuning was realized with low-rank adaptation, resulting in blog posts that successfully covered approximately 95% of the subjects, actions, and objects in the data. This was manually evaluated by a point scoring system that compared the events in the input data to those present in the generated posts. Around 5% of the posts contained hallucinations, which can potentially be lowered in future work by applying reinforcement learning. The third model, a language model trained on mixed-quality data, took the informative blog posts as input and rephrased the text with the aim of making the content more entertaining. Readers’ perception of the posts prior to and after rephrasing, as well as the perception of human-written posts, was evaluated via an online survey, in which post sequences were rated on entertainment, informativeness, and clarity. Applying a Mann Whitney U-test on the results pointed out that this rephrasing model improved the perceived level of entertainment of the generated posts significantly. No sufficient statistical evidence could be found to state that the posts differed on perceived level of informativeness nor clarity, which also holds for the differences in perceived entertainment between the rephrased and human-written posts. Therefore, further research with a larger number of sequences and/or respondents is recommended. |
| Item Type: | Essay (Master) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Interaction Technology MSc (60030) |
| Link to this item: | https://purl.utwente.nl/essays/97962 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page