Towards Overlay-based Rapid In-Circuit Tuning of Deep Learning Designs

This paper presents a novel overlay-based approach for rapid in-circuit debugging and tuning of Deep Learning (DL) designs targeting Field-Programmable Gate Array (FPGA). We first propose overlay-based instruments designed to provide profiling information of hardware to builders of deep learning applications for tuning and debugging FPGA designs. Our instrumentation is optimized to take advantage of characteristics of DL application domain-specific information. We also implement a light-weight overlay-based Deep Neural Networks (DNN) processing engine that supports rapid word length tuning for each DNN layer's datapath without requiring time-consuming compilation of FPGA designs multiple times. Combining the overlay-based instrument and DNN processing engine together, we can visualize the patterns of overflow exceptions and support rapid mixed precision tuning of DNN layers on FPGAs.