University of Twente Student Theses
Using a Multi-Scale Patching With Vision Transformers for 3D Image Segmentation
Imre, Baris (2023) Using a Multi-Scale Patching With Vision Transformers for 3D Image Segmentation.
|
PDF
9MB |
| Abstract: | Automatic semantic segmentation of abdominal aortic aneurysms is a crucial medical task, given that manual segmentation, often required multiple times for various scans over time, can be a highly tedious job for professionals. Over the past decade, convolutional neural networks (CNNs), especially U-Net like models, have been the predominant research area in this field. Recently, vision transformer models, with segmentation-related modifications, have exhibited significant promise. However, these models almost invariably adopt the same patching mechanism that divides the input into equal-sized, non-overlapping sections. This method is not necessarily the most effective, but it has become a common practice since it was a remnant from the transition of transformers from text to images and was the first approach to successfully accomplish this. Consequently, it has been employed simply because it works. In this study, we introduce a tree-like patching method that utilizes a multi-scale perspective of the input with a vision transformer. By tokenizing multiple levels of the image with constant size patches, we aim to provide the transformer with more information, taking advantage of the long-range attention inherent in transformer networks. Furthermore, we propose an architecture that fundamentally incorporates this patching approach, in tandem with a fusion of U-Net-like structures and vision transformers. We demonstrate that, given the same architecture, the multi-scale patching outperforms its traditional counterpart in semantic segmentation of an abdominal aortic aneurysm dataset consisting of 90 CT scans. Our findings clearly show that there is a promising research direction in experimenting with more complex patching mechanisms. |
| Item Type: | Essay (Master) |
| Faculty: | EEMCS: Electrical Engineering, Mathematics and Computer Science |
| Subject: | 54 computer science |
| Programme: | Computer Science MSc (60300) |
| Link to this item: | https://purl.utwente.nl/essays/95058 |
| Export this item as: | BibTeX EndNote HTML Citation Reference Manager |
Show download statistics for this publication
Show download statistics for this publicationRepository Staff Only: item control page