Developing Effective Autonomous Driving for Sim Racing through Reinforcement Learning in Assetto Corsa

This study contributes insights into the intersection of reinforcement learning (RL), sim(ulation) racing, and autonomous driving, specifically within Assetto Corsa (AC) as a sim racing environment. The difference in RL algorithms is explained with a reasoning for the suitability of the Soft-Actor-Critic (SAC) algorithm for an autonomous car racing agent in AC. Based on this, a system design is presented for using AC as an experimentation environment for training the model-free, off-policy SAC algorithm. Specific policy details, hyperparameters, and reward factors are discussed in the context of addressing the lap-completion problem. Here, the objective is to find a policy that achieves completion of a given track with a given car. Results are presented for five different reward functions, on which we conclude the fifth (using the heading and off-center errors) to be the most effective. Future steps for research are laid out with the goal of actually completing a full lap, and ultimately optimizing the minimum-time problem as well. Here the goal is not only to finish, but finish with minimal time.