A Metric for HPC Programming Model Productivity

There has been a healthy growth of heterogeneous programming models that cover different paradigms in the HPC space. Selecting an appropriate programming model for new projects is challenging: how does one select a model that is both productive and performant? The same applies for existing projects aiming to leverage heterogeneous offload capabilities. While characterisation of programming model performance has been abundant and comprehensive, productivity metrics are often reduced to basic measures like Source Line of Code (SLOC). This study introduces a novel model divergence measure to objectively evaluate productivity. We cover common aspects of productivity, including syntax, semantics, and optimisation overhead. We present a productivity analysis framework supporting GCC and Clang, covering models for C/C++ and Fortran. We evaluate our metric using this framework on mini-apps from SPEChpc and other established mini-apps, and propose a combined productivity and performance probability visualisation for a comprehensive picture. Index Terms—Performance portability, productivity, StdPar, CUDA, HIP, Kokkos, OpenMP, OpenMP target, SYCL, TBB, Fortran, benchmarking, semantics