Exploring the Prognostic Value of Deep Learning Image-to-Image Registration for Immunotherapy Patient Monitoring

CT imaging is performed for the monitoring of treatment response in cancer patients receiving immunotherapy. RECIST is currently used for prognostication but has several limitations. This study implements a novel deep learning approach, the prognostic AI Monitor (PAM) on a large pancancer dataset that predicts survival by quantifying morphological deformations. The approach used to quantify these deformations was to pretrain a deep learning network to perform image registration in an unsupervised fashion. From the pre-trained network, features can be extracted that represent the deformations and these can be linked to 1-year survival. To provide explainability, these latent space was disentangled using the Hessian Penalty.