University of Twente Student Theses

This website will be unavailable due to maintenance December 1st between 8:00 and 12:00 CET.

A picture is worth 1000 words : Introducing the visual modality to the query dependent video clip selection process

Voortwis, K.J. W. te (2021) A picture is worth 1000 words : Introducing the visual modality to the query dependent video clip selection process.

[img] PDF
Abstract:Video search and recommendation systems that work with long videos that covermultiple subjects have a unique feature. These videos often contain only parts thatare relevant to the user. Providing access to the relevant parts in the video quicklyand easily is favourable for the user, this can be done using query dependent clipselection. The visual relevance and visual interest of these clips is important whenonly the visual modality is shown to users. The perceived system performance bythe users might not be satisfactory when the relevant information only resides inother modalities, the lacking visual information does not satisfy the user.We propose a system that includes the visual modality in the clip selection pro-cess to improve the visual relevance and visual interest. The information from thespeech and visual modalities are transformed to the same semantic feature spaceto be able to compare clips to the query in this same feature space. The proposedsystem is evaluated on visual relevance and visual interest against a comparablesystem that does not use the visual modality.Including the visual modality does significantly improve the visual relevance ofthe clip selection. The visual interest of the selections improved significantly, but notas convincingly as visual relevance. This research suggests that the type of queryhas an influence on the performance of the proposed system.
Item Type:Essay (Master)
Beeld & Geluid, Hilversum, Netherlands
Faculty:EEMCS: Electrical Engineering, Mathematics and Computer Science
Subject:54 computer science
Programme:Interaction Technology MSc (60030)
Link to this item:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page