University of Twente Student Theses
Robot Navigation in Dynamic Environment Based on Reinforcement Learning
Ouyang, Cijun (2020) Robot Navigation in Dynamic Environment Based on Reinforcement Learning.
|
PDF
9MB |
| Abstract: | Mobile robot navigation attracts research and industrial interests in various fields in recent years. Autonomous navigation in an environment has always been a challenge for mobile robots, especially in a dynamic environment. There are various approaches to solve this problem for mobile robots, but most of the approaches need a model of the entire map and precise prior knowledge of the environment, which is difficult to implement in the real world. Therefore, motivation is formed to apply reinforcement learning(RL) for this navigation task in an unknown environment. Since an optimal route to reach the target can be explored by RL through trial-and-error interactions with the unknown environment and gaining the maximum reward, which doesn't need any prior knowledge. This project aims to implement mobile robot navigation in an unknown dynamic environment with the reinforcement learning method. Deep Q-network(DQN) is used in this project because of the advantage of the training stability. To obtain an optimal policy for path planning with high efficiency and shorter trajectory, we explore several kinds of reward functions and design a proper one for the task in dynamic environments concerning the features of the current states receiving from the environment. Laser sensors are used to obtain the distance information of the target and obstacles around the robot. The two main metrics, Q-value loss and accumulated reward are considered to evaluate the performance of the reward functions. Then it is validated that the reward function concerning both distance and orientation information performs best among the proposed reward functions with low loss value, high accumulated reward, and high stability. Another problem to solve in this project is to extract high-dimensional observation from the environment and compress the observation to low-dimensional states. We use an RGB camera to get observation images and use an auto-encoder to implement state representation of the images. We proposed two methods to extract main features from the dynamic environment. The first one is combining the encoded states from the auto-encoder with laser measurements and additional position states to get precise positions of moving obstacles. The second is inputting a sequence of observations to auto-encoder to get the motion pattern of the moving objects. It is proved that with these methods, the positions of the moving obstacles can be tracked, which improves the success rate of the navigation significantly. |
| Item Type: | Essay (Master) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Embedded Systems MSc (60331) |
| Link to this item: | https://purl.utwente.nl/essays/85320 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page