University of Twente Student Theses

Login

Deepfake detection using capsule networks with long short-term memory networks

Mehra, A. (2020) Deepfake detection using capsule networks with long short-term memory networks.

[img] PDF
19MB
Abstract:With the recent advancement of technology, particularly with graphics processing and artificial intelligence algorithms, fake media generation has become easier. Using deep learning techniques like Deepfakes and FaceSwap, anyone can generate fake videos by manipulating the face/voice of the target in the video. These deepfakes can be used for malicious purposes like phishing scams and fake news. Detecting face tampering in realistic forged videos generated has become of utmost importance. This paper provides an overview of what inconsistencies are introduced in videos due to deepfake generation and proposes a spatio-temporal hybrid model of Capsule Networks integrated with LSTM Networks. This model exploits the inconsistencies and identifies real and fake videos and is our contribution towards deepfake detection. Using visualization of the capsule’s activation, we understand what features the capsules learn and provide an explanation for identifying deepfakes and real videos. Using 3 different frame selection techniques, we also show that frame selection has a significant impact on performance. With almost comparable performance with the state-of-the-art model, in contrast to the size, our model has 1/5th the number of parameters and 1/4th the size of the state-of-the-art model and hence, is a lighter model and has reduced computational cost.
Item Type:Essay (Master)
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:54 computer science
Programme:Computer Science MSc (60300)
Link to this item:https://purl.utwente.nl/essays/83028
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page