University of Twente Student Theses
A Real-Time Framework for Generalizable Segmentation, Tracking, and Depth Estimation using Stereo Vision
Bömer, T.K. (2025) A Real-Time Framework for Generalizable Segmentation, Tracking, and Depth Estimation using Stereo Vision.
|
PDF
40MB |
| Abstract: | Robotic systems require accurate, real-time perception to interact effectively in dynamic and diverse environments. However, many existing vision pipelines lack generalization and cannot meet real-time constraints. This study presents a unified framework that combines segmentation, tracking, and stereo-based depth estimation to enable object-level robotic interaction. The framework integrates prompt-based segmentation (SAM2), a zero-shot open-set detector (GroundingDINO), and stereo depth from a ZED Mini camera. It supports both language and spatial prompts and uses a memory-based mechanism to track objects across frames. An optional co-planar refinement module applies RANSAC and WLS-based hybrid plane fitting to improve depth quality within segmented regions. The system is evaluated through robotic grasping experiments under varied conditions, including translucent objects, low-contrast backgrounds, and cluttered scenes, at 720p and 1080p. Results demonstrate accurate segmentation in well-contrasted scenes, consistent object tracking, and reliable depth estimation, with real-time performance maintained above 10 FPS. Depth refinement improves accuracy but adds computational overhead and may oversimplify curved surfaces. While generalization was observed across test cases, all experiments occurred in a controlled lab setting. Overall, the results show that the proposed framework delivers real-time, object-level visual feedback suitable for robotic manipulation, with promising potential for use in less structured environments. |
| Item Type: | Essay (Master) |
| Faculty: | ET: Engineering Technology |
| Subject: | 53 electrotechnology, 54 computer science |
| Programme: | Biomedical Engineering MSc (66226) |
| Link to this item: | https://purl.utwente.nl/essays/106325 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page