Development of Deep Learning Model for Building Opening Detection as the Application of Unmanned Aerial Vehicle (UAV) in Urban Search and Rescue Mission

Automatic Unmanned Aerial Vehicle (UAV) is a promising technology to minimise human involvement in dangerous activities like urban search and rescue missions (USAR). For this purpose, the ability of a UAV that can map and locate the openings in the damaged building is needed to be able to transit between outdoor and indoor environments. This study focuses on developing a deep learning model for real-time damaged building opening detection by comparing the performance of single and multi-task learning-based detectors. This study consists of four phases: dataset development, single-task network training, multi-task network design, and model performance testing and comparison. This work successfully provides a novel damaged building opening dataset containing images and mask annotations. The deep learning-based detector used in this study is based on YOLOv5 as the most stable current state-of-the-art of real-time object detection model. First, this study compares the different versions of YOLOv5 (i.e. small, medium, and large) to perform damaged building opening detections. Second, multi-task learning YOLOv5 is trained on the same dataset and compared with the single-task detector. This study found that multi-task learning (MTL) based YOLOv5 can improve detection performance by combining detection and segmentation losses. The YOLOv5s-MTL trained on the damaged building opening dataset obtained 0.648 mAP, an increase of 0.167 from the single-task-based network. Regarding the inference speed, the YOLOv5s-MTL can run inference in 73 frames per second on the tested platform. Overall, this study provides a novelty by contributing to developing an initial damaged building opening dataset and detection model. It also demonstrates that a multi-task learning method can improve detection accuracy.