Optimizing FPGA-Based CNN Accelerator Using Differentiable Neural Architecture Search

Hongxiang Fan; Martin Ferianc; Shuanglong Liu; Zhiqiang Que; Xinyu Niu; Wayne Luk

doi:10.1109/ICCD50377.2020.00085

Optimizing FPGA-Based CNN Accelerator Using Differentiable Neural Architecture Search

Hongxiang Fan, Martin Ferianc, Shuanglong Liu, Zhiqiang Que, Xinyu Niu, Wayne Luk

School of Electrical, Electronic and Mechanical Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

10 Citations (Scopus)

Abstract

Neural architecture search (NAS) aims to find the optimal neural network automatically for different scenarios. Among various NAS methods, the differentiable NAS (DNAS) approach has demonstrated its effectiveness in terms of searching cost and final accuracy. However, most of previous efforts focus on applying DNAS to GPU or CPU platforms, and its potential is less exploited on the FPGA. In this paper, we first propose a novel FPGA-based CNN accelerator. An accurate performance model of the proposed hardware design is also introduced. To improve accuracy as well as hardware performance, we then apply DNAS and encapsulate the proposed performance model into the objective function. Based on our FPGA design and NAS method, the experiments demonstrate that the network generated by NAS achieves nearly 95% accuracy on CIFAR-10, while decreasing latency by nearly 12 times compared with existing work.

Original language	English
Title of host publication	Proceedings - 2020 IEEE 38th International Conference on Computer Design, ICCD 2020
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	465-468
Number of pages	4
ISBN (Electronic)	9781728197104
ISBN (Print)	9781728197111
DOIs	https://doi.org/10.1109/ICCD50377.2020.00085
Publication status	Published - 21 Dec 2020
Event	38th IEEE International Conference on Computer Design, ICCD 2020 - Hartford, United States Duration: 18 Oct 2020 → 21 Oct 2020

Publication series

Name	Proceedings - IEEE International Conference on Computer Design: VLSI in Computers and Processors
ISSN (Print)	1063-6404
ISSN (Electronic)	2576-6996

Conference

Conference	38th IEEE International Conference on Computer Design, ICCD 2020
Country/Territory	United States
City	Hartford
Period	18/10/20 → 21/10/20

Bibliographical note

Publisher Copyright:
© 2020 IEEE.

Keywords

FPGA
Neural Architecture Search (NAS)

Access to Document

10.1109/ICCD50377.2020.00085

Handle.net

Persistent link

Cite this

Fan, H., Ferianc, M., Liu, S., Que, Z., Niu, X., & Luk, W. (2020). Optimizing FPGA-Based CNN Accelerator Using Differentiable Neural Architecture Search. In Proceedings - 2020 IEEE 38th International Conference on Computer Design, ICCD 2020 (pp. 465-468). Article 9283571 (Proceedings - IEEE International Conference on Computer Design: VLSI in Computers and Processors). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCD50377.2020.00085

Fan, Hongxiang ; Ferianc, Martin ; Liu, Shuanglong et al. / Optimizing FPGA-Based CNN Accelerator Using Differentiable Neural Architecture Search. Proceedings - 2020 IEEE 38th International Conference on Computer Design, ICCD 2020. Institute of Electrical and Electronics Engineers Inc., 2020. pp. 465-468 (Proceedings - IEEE International Conference on Computer Design: VLSI in Computers and Processors).

@inproceedings{1908c8e0916546d48a1c07e5679fa98f,

title = "Optimizing FPGA-Based CNN Accelerator Using Differentiable Neural Architecture Search",

abstract = "Neural architecture search (NAS) aims to find the optimal neural network automatically for different scenarios. Among various NAS methods, the differentiable NAS (DNAS) approach has demonstrated its effectiveness in terms of searching cost and final accuracy. However, most of previous efforts focus on applying DNAS to GPU or CPU platforms, and its potential is less exploited on the FPGA. In this paper, we first propose a novel FPGA-based CNN accelerator. An accurate performance model of the proposed hardware design is also introduced. To improve accuracy as well as hardware performance, we then apply DNAS and encapsulate the proposed performance model into the objective function. Based on our FPGA design and NAS method, the experiments demonstrate that the network generated by NAS achieves nearly 95\% accuracy on CIFAR-10, while decreasing latency by nearly 12 times compared with existing work.",

keywords = "FPGA, Neural Architecture Search (NAS)",

author = "Hongxiang Fan and Martin Ferianc and Shuanglong Liu and Zhiqiang Que and Xinyu Niu and Wayne Luk",

note = "Publisher Copyright: {\textcopyright} 2020 IEEE.; 38th IEEE International Conference on Computer Design, ICCD 2020 ; Conference date: 18-10-2020 Through 21-10-2020",

year = "2020",

month = dec,

day = "21",

doi = "10.1109/ICCD50377.2020.00085",

language = "English",

isbn = "9781728197111",

series = "Proceedings - IEEE International Conference on Computer Design: VLSI in Computers and Processors",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "465--468",

booktitle = "Proceedings - 2020 IEEE 38th International Conference on Computer Design, ICCD 2020",

}

Fan, H, Ferianc, M, Liu, S, Que, Z, Niu, X & Luk, W 2020, Optimizing FPGA-Based CNN Accelerator Using Differentiable Neural Architecture Search. in Proceedings - 2020 IEEE 38th International Conference on Computer Design, ICCD 2020., 9283571, Proceedings - IEEE International Conference on Computer Design: VLSI in Computers and Processors, Institute of Electrical and Electronics Engineers Inc., pp. 465-468, 38th IEEE International Conference on Computer Design, ICCD 2020, Hartford, United States, 18/10/20. https://doi.org/10.1109/ICCD50377.2020.00085

Optimizing FPGA-Based CNN Accelerator Using Differentiable Neural Architecture Search. / Fan, Hongxiang; Ferianc, Martin; Liu, Shuanglong et al.
Proceedings - 2020 IEEE 38th International Conference on Computer Design, ICCD 2020. Institute of Electrical and Electronics Engineers Inc., 2020. p. 465-468 9283571 (Proceedings - IEEE International Conference on Computer Design: VLSI in Computers and Processors).

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - Optimizing FPGA-Based CNN Accelerator Using Differentiable Neural Architecture Search

AU - Fan, Hongxiang

AU - Ferianc, Martin

AU - Liu, Shuanglong

AU - Que, Zhiqiang

AU - Niu, Xinyu

AU - Luk, Wayne

PY - 2020/12/21

Y1 - 2020/12/21

N2 - Neural architecture search (NAS) aims to find the optimal neural network automatically for different scenarios. Among various NAS methods, the differentiable NAS (DNAS) approach has demonstrated its effectiveness in terms of searching cost and final accuracy. However, most of previous efforts focus on applying DNAS to GPU or CPU platforms, and its potential is less exploited on the FPGA. In this paper, we first propose a novel FPGA-based CNN accelerator. An accurate performance model of the proposed hardware design is also introduced. To improve accuracy as well as hardware performance, we then apply DNAS and encapsulate the proposed performance model into the objective function. Based on our FPGA design and NAS method, the experiments demonstrate that the network generated by NAS achieves nearly 95% accuracy on CIFAR-10, while decreasing latency by nearly 12 times compared with existing work.

AB - Neural architecture search (NAS) aims to find the optimal neural network automatically for different scenarios. Among various NAS methods, the differentiable NAS (DNAS) approach has demonstrated its effectiveness in terms of searching cost and final accuracy. However, most of previous efforts focus on applying DNAS to GPU or CPU platforms, and its potential is less exploited on the FPGA. In this paper, we first propose a novel FPGA-based CNN accelerator. An accurate performance model of the proposed hardware design is also introduced. To improve accuracy as well as hardware performance, we then apply DNAS and encapsulate the proposed performance model into the objective function. Based on our FPGA design and NAS method, the experiments demonstrate that the network generated by NAS achieves nearly 95% accuracy on CIFAR-10, while decreasing latency by nearly 12 times compared with existing work.

KW - FPGA

KW - Neural Architecture Search (NAS)

U2 - 10.1109/ICCD50377.2020.00085

DO - 10.1109/ICCD50377.2020.00085

M3 - Conference Contribution (Conference Proceeding)

AN - SCOPUS:85098855857

SN - 9781728197111

T3 - Proceedings - IEEE International Conference on Computer Design: VLSI in Computers and Processors

SP - 465

EP - 468

BT - Proceedings - 2020 IEEE 38th International Conference on Computer Design, ICCD 2020

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 38th IEEE International Conference on Computer Design, ICCD 2020

Y2 - 18 October 2020 through 21 October 2020

ER -

Fan H, Ferianc M, Liu S, Que Z, Niu X, Luk W. Optimizing FPGA-Based CNN Accelerator Using Differentiable Neural Architecture Search. In Proceedings - 2020 IEEE 38th International Conference on Computer Design, ICCD 2020. Institute of Electrical and Electronics Engineers Inc. 2020. p. 465-468. 9283571. (Proceedings - IEEE International Conference on Computer Design: VLSI in Computers and Processors). doi: 10.1109/ICCD50377.2020.00085

Optimizing FPGA-Based CNN Accelerator Using Differentiable Neural Architecture Search

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Handle.net

Fingerprint

Cite this