University of Twente Student Theses


Impact on how AI in automobile industry has affected the type approval process at RDW

Ravishankaran, Charan (2021) Impact on how AI in automobile industry has affected the type approval process at RDW.

[img] PDF
Abstract:The automobile industry has increased its use of artificial intelligence (AI) over the last decade. One of the primary concerns regarding the use of AI in vehicles is ensuring "safety." Because AI can be subjected to incorrect predictions or make incorrect decisions, this can result in harm to the driver or passenger. Manufacturers test their production units prior to launch in order to avoid such harm or hazardous behaviours. However, in order to establish a manufacturing facility in a region (i.e., country), they must obtain approval from a government body. The government agency certifies that the manufacturing unit is safe. Due to the fact that AI is a type of software, it falls under the software category and must be validated prior to receiving government approval. Artificial intelligence software is based on machine learning, deep learning, and reinforcement learning algorithms. As the use of AI in vehicles increases, validation of the AI software and its capabilities becomes more challenging due to its non-deterministic (black box) behaviour. The primary objective of this paper is to identify and address the current challenges associated with validating the AI software used in autonomous vehicles. Three factors affecting the validation of AI software in autonomous vehicles during the vehicle approval process were identified through an extensive literature review. The three factors are data-related issues, model-related issues, and security-related issues. This paper will focus on data-related issues, with experiments and recommendations. Security concerns are discussed but not prioritized because they are more concerned with cybersecurity principles than with AI. Model-oriented issues such as the explainability of AI, human-machine interaction, and faults in AI model networks have been discussed. For data-related issues, the data used to train and test the AI model was evaluated. The impact of data issues was demonstrated through experiments such as labeling quality estimation (for the training set), quality dataset estimation (for the training and testing sets), and spatial uncertainty estimation. To address these issues, a framework and evaluation metrics were proposed. For autonomous vehicles, data will be collected via sensors installed on the vehicle, such as a camera, LiDAR, or RADAR, and used to make decisions. A case study revealed that camera sensors are widely used by the majority of vehicle manufacturers. As a result, all experiments were conducted using the ImageNet dataset [39], because the camera produces video output of the environment, which is then fed into the AI model as images/frames for decision-making. Finally, these experiments were evaluated using real-time deep learning models such as ResNet50 [39] and SSD-MobileNet [35]. From a data perspective, the proposed framework and evaluation metrics provided an adequate assessment of the AI model's robustness. To demonstrate which metrics are best suited for an autonomous vehicle scenario, the proposed evaluation metrics were compared to real-time metrics such as intersection over union (IoU) and mean Average Precision (mAP). Based on the results of the experiment, a recommendation was made to improve the type approval or safety assessment process at RDW.
Item Type:Essay (Master)
RDW, Enschede, Netherlands
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:54 computer science
Programme:Computer Science MSc (60300)
Link to this item:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page