University of Twente Student Theses
Exploitation of symmetry in Reinforcement Learning and Transformer-Based Reinforcement Learning
Alhannafi, Ammar (2024) Exploitation of symmetry in Reinforcement Learning and Transformer-Based Reinforcement Learning.
|
PDF
8MB |
| Abstract: | Robots operating in dynamic and unstructured environments require highly adaptable control mechanisms to handle uncertainty and variability. Reinforcement Learning (RL) carries the potential to realize such mechanisms and has already demonstrated notable successes in robotics. However, challenges such as data inefficiency, poor generalization, and limited scalability still hinder its broader adoption. This work aims to mitigate these challenges by proposing a unified framework that integrates symmetry exploitation with transformer-based decision models. Symmetry exploitation is formalized through the Markov Decision Process homomorphism framework, which abstracts redundancies arising from symmetries in the state-action space. This abstraction reduces the solution space and enables RL agents to converge faster, making them more sample-efficient and generalizable. Meanwhile, transformer architectures, renowned for their scalability in other domains, can be adapted to mitigate the scalability issues in RL. The proposed framework was validated through implementation and evaluation in simulated environments, including discrete and continuous control tasks such as CartPole and Inverted Pendulum. Results demonstrate that symmetry-aware models not only converge faster but also generalize higher across equivalent states compared to conventional RL and standard transformer-based methods. These findings confirm the hypothesis that exploiting symmetries in RL improves sample efficiency and model generalization across symmetric states and actions, aiding in putting a step forward in scalable robotic control. |
| Item Type: | Essay (Master) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Systems and Control MSc (60359) |
| Link to this item: | https://purl.utwente.nl/essays/104691 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page