Abstract
Modern Neural Networks are being applied to all kinds of industrial areas from lite personal devices, for example, mobile phones or mmWave healthcare devices, to powerhungry supercomputers. The complexity of some network models means that deploying them on low-cost FPGA devices is increasingly challenging due to resource constraints and limited power. Hardware reuse and heterogeneous execution can be used to address this problem and this opens the opportunity for techniques such as early-exiting where the prediction confidence level is evaluated early on.In the first phase, this thesis investigates the application of early-exit strategies to neural networks, mapped to low-cost Xilinx PYNQ-Z2 Field Programmable Gate Array (FPGA) System on Chip (SoC). An early-exit strategy is applied to a network model suitable for ImageNet classification that combines weights with floating-point and binary arithmetic precision. The experiments show an improvement in inferred speed using a single early-exit branch, compared with using a single primary neural network, with a negligible accuracy drop.
In the second phase, a neural network enhancement strategy is devised to improve both accuracy and performance in the FPGA device. The initial floating-point neural network layers are a bottleneck when executed on the Processing System (PS) of the FPGA device. Simply quantizing and re-deploying them on the Programmable Logic (PL) without any structural modification results in a significant accuracy drop which is unacceptable. To address this issue, statistical experiments are devised and conducted to find which strategy is able to satisfy both accuracy and execution efficiency.
In the final phase, this work presents a heterogeneous neural network system that is executed on the low-cost FPGA device with improved inference accuracy and real-time image recognition. The efficiency of inference of the improved system is beatable compared with an Intel i5-9300H CPU with 2.44× gain of speed, and is comparable with power-hungry RTX2060 GPU with 0.87× gain of speed.
| Date of Award | 24 Jan 2023 |
|---|---|
| Original language | English |
| Awarding Institution |
|
| Supervisor | Jose L Nunez-Yanez (Supervisor) & Simon N McIntosh-Smith (Supervisor) |