Evaluation of Early-exit Strategies in Low-cost FPGA-based Binarized Neural Networks

Minxuan Kong; Kris Nikov; Jose Luis Nunez-Yanez

doi:10.1109/DSD57027.2022.00035

Evaluation of Early-exit Strategies in Low-cost FPGA-based Binarized Neural Networks

Minxuan Kong, Kris Nikov, Jose Luis Nunez-Yanez

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

Abstract

In this paper, we investigate the application of early-exit strategies to quantized neural networks with binarized weights, mapped to low-cost FPGA SoC devices. The increasing complexity of network models means that hardware reuse and heterogeneous execution are needed and this opens the opportunity to evaluate the prediction confidence level early on. We apply the early-exit strategy to a network model suitable for ImageNet classification that combines weights with floating-point and binary arithmetic precision. The experiments show an improvement in inferred speed of around 20% using an early-exit network, compared with using a single primary neural network, with a negligible accuracy drop of 1.56%.

Original language	English
Title of host publication	Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022
Editors	Himar Fabelo, Samuel Ortega, Amund Skavhaug
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	197-204
Number of pages	8
ISBN (Electronic)	9781665474047
DOIs	https://doi.org/10.1109/DSD57027.2022.00035
Publication status	Published - 2022
Event	25th Euromicro Conference on Digital System Design, DSD 2022 - Maspalomas, Spain Duration: 31 Aug 2022 → 2 Sept 2022

Publication series

Name	Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022
ISSN (Print)	2639-3859
ISSN (Electronic)	2771-2508

Conference

Conference	25th Euromicro Conference on Digital System Design, DSD 2022
Country/Territory	Spain
City	Maspalomas
Period	31/08/22 → 2/09/22

Bibliographical note

Funding Information:
ACKNOWLEDGMENTS This research was partially funded by the Royal Society Industry fellowship, INF\R2\192044 Machine Intelligence at the Network Edge (MINET), EPSRC HOPWARE EP\RV040863\1 and Leverhulme trust international fellowship High-performance video analytics with parallel heterogeneous neural networks IF-2021-003 .

Publisher Copyright:
© 2022 IEEE.

Keywords

Early-exit
FPGAs
Hardware Acceleration
Neural Network

Access to Document

10.1109/DSD57027.2022.00035

Handle.net

Persistent link

Cite this

Kong, M., Nikov, K., & Nunez-Yanez, J. L. (2022). Evaluation of Early-exit Strategies in Low-cost FPGA-based Binarized Neural Networks. In H. Fabelo, S. Ortega, & A. Skavhaug (Eds.), Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022 (pp. 197-204). (Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/DSD57027.2022.00035

Kong, Minxuan ; Nikov, Kris ; Nunez-Yanez, Jose Luis. / Evaluation of Early-exit Strategies in Low-cost FPGA-based Binarized Neural Networks. Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022. editor / Himar Fabelo ; Samuel Ortega ; Amund Skavhaug. Institute of Electrical and Electronics Engineers (IEEE), 2022. pp. 197-204 (Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022).

@inproceedings{5e3342aa026b4a4fb2e288866081580e,

title = "Evaluation of Early-exit Strategies in Low-cost FPGA-based Binarized Neural Networks",

abstract = "In this paper, we investigate the application of early-exit strategies to quantized neural networks with binarized weights, mapped to low-cost FPGA SoC devices. The increasing complexity of network models means that hardware reuse and heterogeneous execution are needed and this opens the opportunity to evaluate the prediction confidence level early on. We apply the early-exit strategy to a network model suitable for ImageNet classification that combines weights with floating-point and binary arithmetic precision. The experiments show an improvement in inferred speed of around 20% using an early-exit network, compared with using a single primary neural network, with a negligible accuracy drop of 1.56%.",

keywords = "Early-exit, FPGAs, Hardware Acceleration, Neural Network",

author = "Minxuan Kong and Kris Nikov and Nunez-Yanez, {Jose Luis}",

note = "Funding Information: ACKNOWLEDGMENTS This research was partially funded by the Royal Society Industry fellowship, INF\R2\192044 Machine Intelligence at the Network Edge (MINET), EPSRC HOPWARE EP\RV040863\1 and Leverhulme trust international fellowship High-performance video analytics with parallel heterogeneous neural networks IF-2021-003 . Publisher Copyright: {\textcopyright} 2022 IEEE.; 25th Euromicro Conference on Digital System Design, DSD 2022 ; Conference date: 31-08-2022 Through 02-09-2022",

year = "2022",

doi = "10.1109/DSD57027.2022.00035",

language = "English",

series = "Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

pages = "197--204",

editor = "Himar Fabelo and Samuel Ortega and Amund Skavhaug",

booktitle = "Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022",

address = "United States",

}

Kong, M, Nikov, K & Nunez-Yanez, JL 2022, Evaluation of Early-exit Strategies in Low-cost FPGA-based Binarized Neural Networks. in H Fabelo, S Ortega & A Skavhaug (eds), Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022. Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022, Institute of Electrical and Electronics Engineers (IEEE), pp. 197-204, 25th Euromicro Conference on Digital System Design, DSD 2022, Maspalomas, Spain, 31/08/22. https://doi.org/10.1109/DSD57027.2022.00035

Evaluation of Early-exit Strategies in Low-cost FPGA-based Binarized Neural Networks. / Kong, Minxuan; Nikov, Kris; Nunez-Yanez, Jose Luis.
Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022. ed. / Himar Fabelo; Samuel Ortega; Amund Skavhaug. Institute of Electrical and Electronics Engineers (IEEE), 2022. p. 197-204 (Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - Evaluation of Early-exit Strategies in Low-cost FPGA-based Binarized Neural Networks

AU - Kong, Minxuan

AU - Nikov, Kris

AU - Nunez-Yanez, Jose Luis

N1 - Funding Information: ACKNOWLEDGMENTS This research was partially funded by the Royal Society Industry fellowship, INF\R2\192044 Machine Intelligence at the Network Edge (MINET), EPSRC HOPWARE EP\RV040863\1 and Leverhulme trust international fellowship High-performance video analytics with parallel heterogeneous neural networks IF-2021-003 . Publisher Copyright: © 2022 IEEE.

PY - 2022

Y1 - 2022

N2 - In this paper, we investigate the application of early-exit strategies to quantized neural networks with binarized weights, mapped to low-cost FPGA SoC devices. The increasing complexity of network models means that hardware reuse and heterogeneous execution are needed and this opens the opportunity to evaluate the prediction confidence level early on. We apply the early-exit strategy to a network model suitable for ImageNet classification that combines weights with floating-point and binary arithmetic precision. The experiments show an improvement in inferred speed of around 20% using an early-exit network, compared with using a single primary neural network, with a negligible accuracy drop of 1.56%.

AB - In this paper, we investigate the application of early-exit strategies to quantized neural networks with binarized weights, mapped to low-cost FPGA SoC devices. The increasing complexity of network models means that hardware reuse and heterogeneous execution are needed and this opens the opportunity to evaluate the prediction confidence level early on. We apply the early-exit strategy to a network model suitable for ImageNet classification that combines weights with floating-point and binary arithmetic precision. The experiments show an improvement in inferred speed of around 20% using an early-exit network, compared with using a single primary neural network, with a negligible accuracy drop of 1.56%.

KW - Early-exit

KW - FPGAs

KW - Hardware Acceleration

KW - Neural Network

UR - http://www.scopus.com/inward/record.url?scp=85146689634&partnerID=8YFLogxK

U2 - 10.1109/DSD57027.2022.00035

DO - 10.1109/DSD57027.2022.00035

M3 - Conference Contribution (Conference Proceeding)

AN - SCOPUS:85146689634

T3 - Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022

SP - 197

EP - 204

BT - Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022

A2 - Fabelo, Himar

A2 - Ortega, Samuel

A2 - Skavhaug, Amund

PB - Institute of Electrical and Electronics Engineers (IEEE)

T2 - 25th Euromicro Conference on Digital System Design, DSD 2022

Y2 - 31 August 2022 through 2 September 2022

ER -

Kong M, Nikov K, Nunez-Yanez JL. Evaluation of Early-exit Strategies in Low-cost FPGA-based Binarized Neural Networks. In Fabelo H, Ortega S, Skavhaug A, editors, Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022. Institute of Electrical and Electronics Engineers (IEEE). 2022. p. 197-204. (Proceedings - 2022 25th Euromicro Conference on Digital System Design, DSD 2022). doi: 10.1109/DSD57027.2022.00035

Evaluation of Early-exit Strategies in Low-cost FPGA-based Binarized Neural Networks

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Handle.net

Other files and links

Fingerprint

Cite this