Abstract
Parkinson’s disease (PD) is a common neurodegenerative disorder that affects millions of people worldwide. Current clinical assessments of PD symptoms require trained raters and are subjective. Computer vision and machine learning can be used to automate PD assessments, reducing the reliance on trained raters and introducing a more objective measure. However, complexity of human movements, subtle motion differences, and scarcity of annotated data present challenges that this thesis attempts to address by developing novel deep learning frameworks to predict PD severity in videos.First, to assess PD using RGB data, we propose an end-to-end model, built on a temporal segment framework to capture both spatial and long-term temporal structures. We enhance the performance of our model by incorporating a temporal attention mechanism. Motion boundaries are also explored as an extra input modality to assist in obfuscating the effects of camera motion. We evaluate this method on the PD2T dataset, which includes two PD motor function tasks performed by actual patients. Our results suggest that a deep learning-based approach to assess PD from only RGB data is not only feasible, but also effective.
Next, in response to the scarcity of annotated videos, we focus on self-supervised learning (SSL). Unlike traditional SSL methods which struggle with small pretraining data, our approach leverages an auxiliary pretraining phase with knowledge similarity distillation, enabling improved generalisation with significantly less data. We further introduce a novel SSL pretext task, Video Segment Pace Prediction or VSPP, to provide more reliable self-supervised representation. Our SSL framework shows state of the art performance on UCF101 and HMDB50 datasets under a low-data regime. Furthermore, this approach outperforms fully-supervised pretraining when evaluated on a new PD dataset (PD4T), which includes four different PD motor tasks.
Finally, this thesis presents a novel, parameter-efficient, continual pretraining workflow (PECoP) that significantly improves upon conventional fine-tuning techniques. Its primary objective is to enhance the transfer of knowledge gained from existing large-scale video datasets to AQA target tasks by updating only a small number of parameters in additional bottleneck layers (called 3D-Adapters) through self-supervised learning. Evaluating our method on PD4T, and three public AQA benchmarks (JIGSAWS, MTL-AQA, FineDiving), we show that PECoP can boost the robustness of recent state of the art AQA methods, by a considerable margin.
| Date of Award | 23 Jan 2024 |
|---|---|
| Original language | English |
| Awarding Institution |
|
| Supervisor | Majid Mirmehdi (Supervisor) & Alan L Whone (Supervisor) |