Benchmarking the NVIDIA V100 GPU and Tensor Cores

Matt Martineau; Patrick Atkinson; Simon McIntosh-Smith

doi:10.1007/978-3-030-10549-5_35

Benchmarking the NVIDIA V100 GPU and Tensor Cores

Matt Martineau^*, Patrick Atkinson, Simon McIntosh-Smith

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

20 Citations (Scopus)

Abstract

The V100 GPU is the newest server-grade GPU produced by NVIDIA and introduces a number of new hardware and API features. This paper details the results of benchmarking the V100 GPU and demonstrates that it is a significant generational improvement, increasing memory bandwidth, cache bandwidth, and reducing latency. A major new addition is the Tensor core units, which have been marketed as deep learning acceleration features that enable the computation of a 4 × 4 × 4 half precision matrix-multiply-accumulate operation in a single clock cycle. This paper confirms that the Tensor cores offer considerable performance gains for half precision general matrix multiplication; however, programming them requires fine control of the memory hierarchy that is typically unnecessary for other applications.

Original language	English
Title of host publication	Euro-Par 2018
Subtitle of host publication	Parallel Processing Workshops - Euro-Par 2018 International Workshops, Revised Selected Papers
Editors	Gabriele Mencagli, Dora B. Heras
Publisher	Springer Verlag
Pages	444-455
Number of pages	12
ISBN (Print)	9783030105488
DOIs	https://doi.org/10.1007/978-3-030-10549-5_35
Publication status	Published - 1 Jan 2019
Event	24th International Conference on Parallel and Distributed Computing, Euro-Par 2018 - Turin, Italy Duration: 27 Aug 2018 → 28 Aug 2018

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11339 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Conference on Parallel and Distributed Computing, Euro-Par 2018
Country/Territory	Italy
City	Turin
Period	27/08/18 → 28/08/18

Access to Document

10.1007/978-3-030-10549-5_35

Handle.net

Persistent link

Cite this

Martineau, M., Atkinson, P., & McIntosh-Smith, S. (2019). Benchmarking the NVIDIA V100 GPU and Tensor Cores. In G. Mencagli, & D. B. Heras (Eds.), Euro-Par 2018: Parallel Processing Workshops - Euro-Par 2018 International Workshops, Revised Selected Papers (pp. 444-455). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11339 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-10549-5_35

Martineau, Matt ; Atkinson, Patrick ; McIntosh-Smith, Simon. / Benchmarking the NVIDIA V100 GPU and Tensor Cores. Euro-Par 2018: Parallel Processing Workshops - Euro-Par 2018 International Workshops, Revised Selected Papers. editor / Gabriele Mencagli ; Dora B. Heras. Springer Verlag, 2019. pp. 444-455 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{ef829dbc7ee044f1aad51cb15941e7a3,

title = "Benchmarking the NVIDIA V100 GPU and Tensor Cores",

abstract = "The V100 GPU is the newest server-grade GPU produced by NVIDIA and introduces a number of new hardware and API features. This paper details the results of benchmarking the V100 GPU and demonstrates that it is a significant generational improvement, increasing memory bandwidth, cache bandwidth, and reducing latency. A major new addition is the Tensor core units, which have been marketed as deep learning acceleration features that enable the computation of a 4 × 4 × 4 half precision matrix-multiply-accumulate operation in a single clock cycle. This paper confirms that the Tensor cores offer considerable performance gains for half precision general matrix multiplication; however, programming them requires fine control of the memory hierarchy that is typically unnecessary for other applications.",

author = "Matt Martineau and Patrick Atkinson and Simon McIntosh-Smith",

year = "2019",

month = jan,

day = "1",

doi = "10.1007/978-3-030-10549-5_35",

language = "English",

isbn = "9783030105488",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "444--455",

editor = "Gabriele Mencagli and Heras, {Dora B.}",

booktitle = "Euro-Par 2018",

address = "Germany",

note = "24th International Conference on Parallel and Distributed Computing, Euro-Par 2018 ; Conference date: 27-08-2018 Through 28-08-2018",

}

Martineau, M, Atkinson, P & McIntosh-Smith, S 2019, Benchmarking the NVIDIA V100 GPU and Tensor Cores. in G Mencagli & DB Heras (eds), Euro-Par 2018: Parallel Processing Workshops - Euro-Par 2018 International Workshops, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11339 LNCS, Springer Verlag, pp. 444-455, 24th International Conference on Parallel and Distributed Computing, Euro-Par 2018, Turin, Italy, 27/08/18. https://doi.org/10.1007/978-3-030-10549-5_35

Benchmarking the NVIDIA V100 GPU and Tensor Cores. / Martineau, Matt; Atkinson, Patrick; McIntosh-Smith, Simon.
Euro-Par 2018: Parallel Processing Workshops - Euro-Par 2018 International Workshops, Revised Selected Papers. ed. / Gabriele Mencagli; Dora B. Heras. Springer Verlag, 2019. p. 444-455 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11339 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - Benchmarking the NVIDIA V100 GPU and Tensor Cores

AU - Martineau, Matt

AU - Atkinson, Patrick

AU - McIntosh-Smith, Simon

PY - 2019/1/1

Y1 - 2019/1/1

N2 - The V100 GPU is the newest server-grade GPU produced by NVIDIA and introduces a number of new hardware and API features. This paper details the results of benchmarking the V100 GPU and demonstrates that it is a significant generational improvement, increasing memory bandwidth, cache bandwidth, and reducing latency. A major new addition is the Tensor core units, which have been marketed as deep learning acceleration features that enable the computation of a 4 × 4 × 4 half precision matrix-multiply-accumulate operation in a single clock cycle. This paper confirms that the Tensor cores offer considerable performance gains for half precision general matrix multiplication; however, programming them requires fine control of the memory hierarchy that is typically unnecessary for other applications.

AB - The V100 GPU is the newest server-grade GPU produced by NVIDIA and introduces a number of new hardware and API features. This paper details the results of benchmarking the V100 GPU and demonstrates that it is a significant generational improvement, increasing memory bandwidth, cache bandwidth, and reducing latency. A major new addition is the Tensor core units, which have been marketed as deep learning acceleration features that enable the computation of a 4 × 4 × 4 half precision matrix-multiply-accumulate operation in a single clock cycle. This paper confirms that the Tensor cores offer considerable performance gains for half precision general matrix multiplication; however, programming them requires fine control of the memory hierarchy that is typically unnecessary for other applications.

UR - http://www.scopus.com/inward/record.url?scp=85061713392&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-10549-5_35

DO - 10.1007/978-3-030-10549-5_35

M3 - Conference Contribution (Conference Proceeding)

AN - SCOPUS:85061713392

SN - 9783030105488

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 444

EP - 455

BT - Euro-Par 2018

A2 - Mencagli, Gabriele

A2 - Heras, Dora B.

PB - Springer Verlag

T2 - 24th International Conference on Parallel and Distributed Computing, Euro-Par 2018

Y2 - 27 August 2018 through 28 August 2018

ER -

Martineau M, Atkinson P, McIntosh-Smith S. Benchmarking the NVIDIA V100 GPU and Tensor Cores. In Mencagli G, Heras DB, editors, Euro-Par 2018: Parallel Processing Workshops - Euro-Par 2018 International Workshops, Revised Selected Papers. Springer Verlag. 2019. p. 444-455. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-10549-5_35

Benchmarking the NVIDIA V100 GPU and Tensor Cores

Abstract

Publication series

Conference

Access to Document

Handle.net

Other files and links

Fingerprint

Cite this