University of Twente Student Theses


Integrating remote sensing and street view images to map slums using deep learning approach

Najmi, Abbas (2021) Integrating remote sensing and street view images to map slums using deep learning approach.

[img] PDF
Link to full-text:
(only accessible for UT students and staff)
Abstract:The United Nations includes slum upliftment as one of the agenda in the Sustainable Development Goals 11, Target 11.1- "safe and affordable housing" to fight against poverty. The information to keep track of target 11.1, such as physical location and size of slums, is lacking or inadequate in governmental documents. Therefore it is vital to map slums in order to comprehend the existing situation and build future slum development policy plans to achieve target 11.1. Remote Sensing (RS)-based approaches have gained much recognition in the slum mapping field in the last few decades due to the availability of Remote Sensing Imagery (RSI) of Very High Resolution (VHR). In RS-based approaches, the Deep Learning (DL) approaches such as Fully Convolutional Network (FCN) have been shown to achieve reasonably higher accuracies for slum mapping than other RS-based approaches. However, using RSI alone has its limitation, i.e., the absence of ground-level information, making slum identification difficult in the dense urban scene. Previous studies show that adding ground-level information with RSI can help identify slums more precisely than using RSI alone, but none of the studies used Street View Imagery (SVI) as the source of ground-level information to compliment RSI in the field of slum mapping. Therefore this research aims to integrate RSI with SVI using FCN for slum mapping. Implementing FCN has three significant challenges, from which the first challenge is general for all slum mapping approaches, and the remaining two are specifically for the FCN. First is the conceptualization of slums because there is no unique definition of slums, i.e., it varies from institution to institution. Second, extraction of ground-level information through SVI to identify slums. Third, setting up an FCN pipeline to integrate overhead information with ground-level information, i.e., integrating RSI with extracted features of SVI. The city of Jakarta was chosen for this study because of two main reasons. First, the presence of kampungs (urban villages) in Jakarta. Around 60% of Jakarta's population lives in kampungs, and the diverse socioeconomic conditions in kampungs make it challenging to identify slums inside kampungs, i.e., the line between slums and non-slums is vague. There are two types of kampungs such as legal and illegal. This research focused on the illegal kampungs called slums. Second, there are various local definitions of slums used in Jakarta, making the conceptualization of slums more difficult. Initially, the western region of Jakarta was chosen for study because of the high density of slum settlements according to the official slum reference map of 2017, but due to data constraints, approximately half of the western region with some part of the northern and central region was selected as a study area. In this research, four deep neural networks are applied with different datasets, i.e., FCN-DK6 used RSI alone, Places365-VGG16 was fine-tuned using SVI, and FCN-DK6-i and Modified FCN-DK6 used a combination of RSI and SVI in the study area. The FCN-DK6 network was trained with RSI alone to map slums in the study area. The Places365-VGG16 network was fine-tuned in the context of Jakarta's slums using SVI captured in the study area. Further, the fine-tuned Places365-VGG16 network was used to extract the features from widely dispersed SVI and spatially interpolated them to precisely match the spatial resolution of RSI, which are combined with RSI for slum mapping using FCN-DK6-i and Modified FCN-DK6 networks. The result shows that the Modified FCN-DK6 outperforms FCN-DK6 and FCN-DK6-i in slum mapping, demonstrating that combining RSI and SVI can achieve higher accuracy because SVI contains useful ground-level information which helps to identify slums in an urban setting than using RSI alone. Furthermore, we describe experimental investigations by combining the extracted SVI features with RSI at different levels in FCN-DK6-i and Modified FCN-DK6, which shows that the combination of RSI and SVI can improve the accuracy obtained from RSI alone, but it also depends on how they are integrated. The Modified FCN-DK6 presented here obtains better results than a direct integration through FCN-DK6-i.
Item Type:Essay (Master)
Faculty:ITC: Faculty of Geo-information Science and Earth Observation
Programme:Geoinformation Science and Earth Observation MSc (75014)
Link to this item:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page