Machine Learning Models for Multimodal Retinal Imaging

  • Xin Tian

Student thesis: Doctoral ThesisDoctor of Philosophy (PhD)

Abstract

Retinal imaging leverages various modalities, each with unique capabilities and limitations. This thesis introduces innovative machine-learning frameworks designed to optimise the utility and accuracy of these imaging techniques by focusing on three pivotal areas: image registration, image-to-image translation, and image fusion.
The first part of the research establishes a novel longitudinal Optimal Transport-Based Graph Matching (OT-GM) framework for 3D Optical Coherence Tomography (OCT) registration. This framework introduces Adaptive Weighted Vessel Graph Descriptors (AWVGD) and Cube Descriptors (CD), tailored to address the inherent issues in OCT imaging, such as noise, low contrast, and deformation. By precisely aligning OCT scans from different days, our model facilitates accurate monitoring of disease progression, which is essential for effective clinical diagnosis and monitoring in diseases such as uveitis.

The second part presents the OCT2Confocal dataset alongside a novel 3D CycleGAN model that translates OCT images into high-resolution, multichannel confocal microscopy images. By employing a model incorporating 3D network structures and CycleGAN losses with additional gradient loss enhancements, the 3D CycleGAN maintains critical pathological features, colour code preservation, and spatial consistency, which is superior to 2D approaches. This translation capability is particularly valuable for enhancing early disease detection and studying drug responses without the need for invasive procedures.

The third part is the Topology-Aware Graph Attention Network (TaGAT) for multimodal retinal image fusion. This model integrates images from different retinal imaging modalities into one representation that preserves essential diagnostic features from each source. Employing a Topology-Aware Encoder (TAE) with a Graph Attention Network (GAT), TaGAT effectively preserves and utilises the retinal vascular topology, ensuring that no critical information is lost during the fusion process, especially for retinal images. This approach is applied to integrating Colour Fundus (CF) and Fundus Fluorescein Angiography (FFA) images, proving particularly beneficial in retaining complex vascular and textural details crucial for diagnostic accuracy.

Collectively, these frameworks enhance the capabilities of retinal imaging techniques, providing clinicians and researchers with advanced tools for a more profound understanding of retinal pathologies, thus supporting informed clinical decisions and advancing retinal disease research.
Date of Award1 Oct 2024
Original languageEnglish
Awarding Institution
  • University of Bristol
SupervisorAlin Achim (Supervisor), Lindsay B Nicholson (Supervisor) & Pui Anantrasirichai (Supervisor)

Cite this

'