University of Twente Student Theses


Towards real-time placental surface reconstruction during fetal surgery : deep-learned placental vessel identification and segmentation

Niekolaas, M. (2021) Towards real-time placental surface reconstruction during fetal surgery : deep-learned placental vessel identification and segmentation.

[img] PDF
Abstract:Introduction – Twin-To-Twin Transfusion Syndrome (TTTS) is a condition that occurs in monochorionic twin pregnancies and is characterized by a disbalanced blood supply between the two fetuses. When left untreated, TTTS is associated with a mortality rate of 90-95%. To date, Fetoscopic Lasercoagulation Of Vascular Anastomoses (FLOVA) is the only treatment option for TTTS that addresses the underlying pathology. One of the main drawbacks of FLOVA is the limited Field of View (FOV). Therefore, providing the surgeon with an overview of the placental vasculature is thought to increase the success rates of FLOVA. Through the recent years, more and more research has been conducted on reconstruction of the placental surface using image stitching algorithms. However, to date, none of these approaches were proven successful when applied to (longer) in-vivo video sequences. The in-vivo fetoscopic videos contain numerous frames that are either irrelevant or disruptive for image stitching. Therefore, the first part of this thesis focused on automatic classification of frames that are suitable for image stitching using a deep learning approach. The second part focused on the training and potential use of a vessel segmentation network. We hypothesized that the resulting segmentation maps can be used for 1) improved inlier feature detection by using selective regional image enhancement and 2) intensity-based image stitching. Methods – Ten in-vivo fetoscopic videos from FLOVA procedures were included in this thesis. First, the effect of the frame content on the number and quality of the detected inlier feature matches is evaluated. Thereafter, a total of 62,422 labeled frames were extracted and labeled. A pre-trained CNN with a VGG-16 architecture was trained for binary classification of the in-vivo video frames. For the vessel segmentation network (VesSeg), a U-Net was trained using 729 in-vivo frames and ground truth vessel segmentations. Lastly, the potential use of the vessel segmentations for both feature-based and intensity-based image stitching was briefly explored. Results & Discussion – Frames in which the vessels are visible without any occlusions were most suitable for image stitching, followed by frames that show vessels that are partly occluded. These frames were therefore labeled as suitable for the vessel identification network. The trained vessel identification network (VesDet) generated predictions with an ROC-AUC of 0.95 when tested using an unseen video. A prediction rate of 714 fps was reported when using Google Colab’s GPU. Based on these results, the network is considered useful and applicable for future clinical implementation. Our best performing U-Net generated vessel segmentations with a Dice Score of 0.80 (± 0.13) and ROC-AUC of 0.98. In literature, two studies proposed similar networks and reported Dice Scores of 0.55 (± 0.22) and 0.78 (± 0.13) for their best networks. Therefore, our network significantly outperformed the network from the first study and slightly outperformed the network from the second study described in literature. Additional qualitative analysis supported these findings. Moreover, an average prediction rate of 7 fps was measured, which is considered sufficient for future clinical applications of the network. vi Lastly, experimentations with VesSeg for multiple different feature-based and intensity-based image stitching approaches showed an increase in the number of frames that were stitched together successfully. However, systematic research on the different image stitching approaches is highly recommended. Conclusion – Based on the studies and experimentations performed in this thesis, we conclude that the vessel identification and segmentation deep learning networks are of added value for image stitching of in-vivo fetoscopic video frames. Moreover, the networks are considered suitable for clinical applications based on their high performance when tested using unseen in-vivo data and fast prediction rates. However, the image stitching algorithm requires further development before it can be used in clinical settings.
Item Type:Essay (Master)
Faculty:TNW: Science and Technology
Subject:01 general works, 30 exact sciences in general, 42 biology, 44 medicine, 50 technical science in general, 54 computer science
Programme:Technical Medicine MSc (60033)
Link to this item:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page