University of Twente Student Theses


VICE-GAN: Video Identity-Consistent Emotion Generative Adversarial Network

Jayagopal, Tarun Narain (2021) VICE-GAN: Video Identity-Consistent Emotion Generative Adversarial Network.

[img] PDF
Abstract:We propose the Video Identity-Consistent Emotion Generative Adversarial Network (VICE-GAN) model for video generation. The proposed model can generate realistic videos of six emotional expressions while allowing the identity of the individual to be preserved. This was achieved by introducing (i) a pre-trained autoencoder which produces a compressed representation of the individual present in an input video and therefore preserves the content of the video and (ii) a content consistency loss to further enforce identity consistency by extracting and comparing the content representations between the generated and real frames of a video. In addition, we experimented with three variables in order to determine their impact on model performance. Eight model variants were evaluated based on visual quality, emotion generation and identity consistency. Overall, models which were exposed to the test subjects beforehand for a limited number of emotions produced video sequences of higher visual quality and identity consistency when compared to models in which the test subjects were removed from the training data entirely. Using the content representation of the first frame for all subsequent frames in contrast to using a unique representation for each frame appears to benefit identity consistency only. There is also evidence to suggest that freezing autoencoder weights during GAN training results in improvements for visual quality and emotion generation.
Item Type:Essay (Master)
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:54 computer science
Programme:Computer Science MSc (60300)
Link to this item:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page