On-Sensor Visual Inference with A Pixel Processor Array

  • Yanan Liu

Student thesis: Doctoral ThesisDoctor of Philosophy (PhD)


Considering the limitations of latency and power consumption in conventional machine vision systems, this thesis investigates a new visual information process scheme with an emerging visual sensor: the Pixel Processor Array (PPA) by directly processing signals where they are collected, hence avoiding the aforementioned issues with conventional machine vision systems. In particular, this thesis establishes mobile robotic control systems with on-sensor computed results for multiple navigation research. Then, our work investigates novel parallel visual inference approaches, with a particular emphasis on parallel machine vision algorithms and cutting-edge machine learning-based algorithms. Specifically, we are motivated to perform neural networks to extract higher-level helpful information from the analogue signals. An edge computing platform can be established based on our neural network with the PPA, where only a small quantity of extracted information is obtained, allowing for more efficient data transmission with less bandwidth. Hence, this thesis presents a lightweight and high-speed binary convolutional neural network on the sensor to categorise a range of objects. With the proposed methods to implement networks, all floating-point time-consuming multiplication operations can be replaced by efficient addition/subtraction and bit shifting operations. The focal-plane visual inference is difficult due to hardware resource constraints, such as limited registers and analogue noises. Hence, this work further proposes the purely binarised convolutional neural networks with both binary weights and activations. This thesis trains and implements neural networks with batch normalisation and adaptive threshold to binarise activations. The binary activations on the sensor benefit the neural network performance by alleviating the noises introduced by using analogue signals.
Date of Award12 May 2022
Original languageEnglish
Awarding Institution
  • University of Bristol
SupervisorWalterio W Mayol-Cuevas (Supervisor)

Cite this