A method for automatically generating analogue benchmarksuites using low-level hardware metrics

Research output: Chapter in Book/Report/Conference proceedingConference Contribution (Conference Proceeding)

Abstract

Real workloads on large-scale, multi-user high performance computing systems can be complex in multiple dimensions: the range of applications run on the hardware system may be diverse and could number in the hundreds or even thousands, and the application mix may vary over time. Constructing and then maintaining representative benchmark suites for such situations can be extremely challenging and can be further complicated when the systems are running codes of a confidential nature.

In this paper we present a new method which can automatically characterise any workload on a large-scale, multi-user and multi-application system. Our approach uses microarchitecture-level performance metrics, such as the number of branch mispredictions or cache misses. These low-level metrics can be gathered using standard tools on live systems running production codes with very little performance overhead and with no change to any of the codes being analysed. Our method uses microarchitecture-level metrics to construct a statistical model of the real workload. In a second step, a set of ‘analogue’ benchmarks are also profiled using the same set of microarchitecture performance metrics. In the final step, an automated process constructs a benchmark workload from the set of simple analogue benchmarks. This ‘analogue workload’ closely approximates the real workload in terms of its statistical behaviour on the hardware and can be used for subsequent relative performance benchmarking.
Original languageEnglish
Title of host publicationACM SIGMETRICS Performance Evaluation Review
PublisherAssociation for Computing Machinery (ACM)
Pages17-18
Number of pages2
ISBN (Print)9781450311021
DOIs
Publication statusPublished - Nov 2011
EventPerformance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS11) - Seattle, United States
Duration: 12 Nov 201118 Nov 2011

Conference

ConferencePerformance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS11)
CountryUnited States
CitySeattle
Period12/11/1118/11/11

Fingerprint Dive into the research topics of 'A method for automatically generating analogue benchmarksuites using low-level hardware metrics'. Together they form a unique fingerprint.

Cite this