University of Twente Student Theses
Leveraging Graded Similarity in Self-Supervised Learning
Rappange, Sam (2024) Leveraging Graded Similarity in Self-Supervised Learning.
|
PDF
17MB |
| Abstract: | Self-supervised methods in computer vision have become dominated by contrastive learning approaches, surpassing various pretext tasks such as rotation prediction and colorization. Contrastive methods learn augmentation invariant representations of images by pulling augmented views of the same image to the same representation while pushing views of other images away. Non-contrastive methods reach similar performance without requiring negative views. These methods pull all possible crops of an image to the same representation, regardless of their content, position, or shared information. This can hinder training and force the model to discard valuable information about the views, as view pairs have widely varying amounts of similarity. We propose a novel learning objective utilizing a graded similarity measure to address this limitation. The graded similarity measure uses the overlap of crops as a measure regarding the distance between representations in the latent space. This novel learning objective better encodes the nuanced similarity between views while emphasizing spatial relations. We implement this graded similarity in contrastive (SimCLR) and non-contrastive methods (SimSiam). Our results show that pre-trained encoders using our approach reach slightly better performance in transfer learning and up to 1.4× better performance in retrieval tasks. Notably, SimCLR improves significantly from this novel learning objective. |
| Item Type: | Essay (Master) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 30 exact sciences in general, 50 technical science in general, 54 computer science |
| Programme: | Computer Science MSc (60300) |
| Link to this item: | https://purl.utwente.nl/essays/102853 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page