A survey of application memory usage on a national supercomputer: An analysis of memory requirements on ARCHER

Andy Turner; Simon McIntosh-Smith

doi:10.1007/978-3-319-72971-8_13

A survey of application memory usage on a national supercomputer: An analysis of memory requirements on ARCHER

Andy Turner^*, Simon McIntosh-Smith

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

6 Citations (Scopus)

370 Downloads (Pure)

Abstract

In this short paper we set out to provide a set of modern data on the actual memory per core and memory per node requirements of the most heavily used applications on a contemporary, national-scale supercomputer. This report is based on data from all jobs run on the UK national supercomputing service, ARCHER, a 118,000 core Cray XC30, in the 1 year period from 1^st July 2016 to 30^th June 2017 inclusive. Our analysis shows that 80% of all usage on ARCHER has a maximum memory use of 1 GiB/core or less (24 GiB/node or less) and that there is a trend to larger memory use as job size increases. Analysis of memory use by software application type reveals differences in memory use between periodic electronic structure, atomistic N-body, grid-based climate modelling, and grid-based CFD applications. We present an analysis of these differences, and suggest further analysis and work in this area. Finally, we discuss the implications of these results for the design of future HPC systems, in particular the applicability of high bandwidth memory type technologies.

Original language	English
Title of host publication	High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation
Subtitle of host publication	8th International Workshop, Proceedings
Editors	S Jarvis, S Wright, S Hammond
Publisher	Springer Verlag
Pages	250-260
Number of pages	11
ISBN (Electronic)	9783319729718
ISBN (Print)	9783319729701
DOIs	https://doi.org/10.1007/978-3-319-72971-8_13
Publication status	Published - 1 Jan 2018
Event	8th International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computing Systems, PMBS 2017 - [state] CO, United States Duration: 13 Nov 2017 → 13 Nov 2017

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10724 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	8th International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computing Systems, PMBS 2017
Country/Territory	United States
City	[state] CO
Period	13/11/17 → 13/11/17

Keywords

HPC
Memory
Profiling

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1007/978-3-319-72971-8_13

Full-text PDF (accepted author manuscript)
This is the author accepted manuscript (AAM). The final published version (version of record) is available online via Springer Nature at https://link.springer.com/chapter/10.1007%2F978-3-319-72971-8_13. Please refer to any applicable terms of use of the publisher.
Accepted author manuscript, 908 KB

Handle.net

Persistent link

Cite this

Turner, A., & McIntosh-Smith, S. (2018). A survey of application memory usage on a national supercomputer: An analysis of memory requirements on ARCHER. In S. Jarvis, S. Wright, & S. Hammond (Eds.), High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation: 8th International Workshop, Proceedings (pp. 250-260). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10724 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-72971-8_13

Turner, Andy ; McIntosh-Smith, Simon. / A survey of application memory usage on a national supercomputer : An analysis of memory requirements on ARCHER. High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation: 8th International Workshop, Proceedings. editor / S Jarvis ; S Wright ; S Hammond. Springer Verlag, 2018. pp. 250-260 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{9fd7b3d2a33742e0a52701e46821c55b,

title = "A survey of application memory usage on a national supercomputer: An analysis of memory requirements on ARCHER",

abstract = "In this short paper we set out to provide a set of modern data on the actual memory per core and memory per node requirements of the most heavily used applications on a contemporary, national-scale supercomputer. This report is based on data from all jobs run on the UK national supercomputing service, ARCHER, a 118,000 core Cray XC30, in the 1 year period from 1st July 2016 to 30th June 2017 inclusive. Our analysis shows that 80% of all usage on ARCHER has a maximum memory use of 1 GiB/core or less (24 GiB/node or less) and that there is a trend to larger memory use as job size increases. Analysis of memory use by software application type reveals differences in memory use between periodic electronic structure, atomistic N-body, grid-based climate modelling, and grid-based CFD applications. We present an analysis of these differences, and suggest further analysis and work in this area. Finally, we discuss the implications of these results for the design of future HPC systems, in particular the applicability of high bandwidth memory type technologies.",

keywords = "HPC, Memory, Profiling",

author = "Andy Turner and Simon McIntosh-Smith",

year = "2018",

month = jan,

day = "1",

doi = "10.1007/978-3-319-72971-8_13",

language = "English",

isbn = "9783319729701",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "250--260",

editor = "S Jarvis and S Wright and S Hammond",

booktitle = "High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation",

address = "Germany",

note = "8th International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computing Systems, PMBS 2017 ; Conference date: 13-11-2017 Through 13-11-2017",

}

Turner, A & McIntosh-Smith, S 2018, A survey of application memory usage on a national supercomputer: An analysis of memory requirements on ARCHER. in S Jarvis, S Wright & S Hammond (eds), High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation: 8th International Workshop, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10724 LNCS, Springer Verlag, pp. 250-260, 8th International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computing Systems, PMBS 2017, [state] CO, United States, 13/11/17. https://doi.org/10.1007/978-3-319-72971-8_13

A survey of application memory usage on a national supercomputer: An analysis of memory requirements on ARCHER. / Turner, Andy; McIntosh-Smith, Simon.
High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation: 8th International Workshop, Proceedings. ed. / S Jarvis; S Wright; S Hammond. Springer Verlag, 2018. p. 250-260 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10724 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - A survey of application memory usage on a national supercomputer

T2 - 8th International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computing Systems, PMBS 2017

AU - Turner, Andy

AU - McIntosh-Smith, Simon

PY - 2018/1/1

Y1 - 2018/1/1

N2 - In this short paper we set out to provide a set of modern data on the actual memory per core and memory per node requirements of the most heavily used applications on a contemporary, national-scale supercomputer. This report is based on data from all jobs run on the UK national supercomputing service, ARCHER, a 118,000 core Cray XC30, in the 1 year period from 1st July 2016 to 30th June 2017 inclusive. Our analysis shows that 80% of all usage on ARCHER has a maximum memory use of 1 GiB/core or less (24 GiB/node or less) and that there is a trend to larger memory use as job size increases. Analysis of memory use by software application type reveals differences in memory use between periodic electronic structure, atomistic N-body, grid-based climate modelling, and grid-based CFD applications. We present an analysis of these differences, and suggest further analysis and work in this area. Finally, we discuss the implications of these results for the design of future HPC systems, in particular the applicability of high bandwidth memory type technologies.

AB - In this short paper we set out to provide a set of modern data on the actual memory per core and memory per node requirements of the most heavily used applications on a contemporary, national-scale supercomputer. This report is based on data from all jobs run on the UK national supercomputing service, ARCHER, a 118,000 core Cray XC30, in the 1 year period from 1st July 2016 to 30th June 2017 inclusive. Our analysis shows that 80% of all usage on ARCHER has a maximum memory use of 1 GiB/core or less (24 GiB/node or less) and that there is a trend to larger memory use as job size increases. Analysis of memory use by software application type reveals differences in memory use between periodic electronic structure, atomistic N-body, grid-based climate modelling, and grid-based CFD applications. We present an analysis of these differences, and suggest further analysis and work in this area. Finally, we discuss the implications of these results for the design of future HPC systems, in particular the applicability of high bandwidth memory type technologies.

KW - HPC

KW - Memory

KW - Profiling

UR - http://www.scopus.com/inward/record.url?scp=85039707323&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-72971-8_13

DO - 10.1007/978-3-319-72971-8_13

M3 - Conference Contribution (Conference Proceeding)

AN - SCOPUS:85039707323

SN - 9783319729701

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 250

EP - 260

BT - High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation

A2 - Jarvis, S

A2 - Wright, S

A2 - Hammond, S

PB - Springer Verlag

Y2 - 13 November 2017 through 13 November 2017

ER -

Turner A, McIntosh-Smith S. A survey of application memory usage on a national supercomputer: An analysis of memory requirements on ARCHER. In Jarvis S, Wright S, Hammond S, editors, High Performance Computing Systems. Performance Modeling, Benchmarking, and Simulation: 8th International Workshop, Proceedings. Springer Verlag. 2018. p. 250-260. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2017 Dec 23. doi: 10.1007/978-3-319-72971-8_13

A survey of application memory usage on a national supercomputer: An analysis of memory requirements on ARCHER

Abstract

Publication series

Conference

Keywords

UN SDGs

Access to Document

Handle.net

Other files and links

Fingerprint

Professor Simon N McIntosh-Smith

Cite this

A survey of application memory usage on a national supercomputer: An analysis of memory requirements on ARCHER

Abstract

Publication series

Conference

Keywords

UN SDGs

Access to Document

Handle.net

Other files and links

Fingerprint

Profiles

Professor Simon N McIntosh-Smith

Cite this