Instead, they have a scientific or engineering mandate. No computing center has a mission statement asking for a place on a benchmark ranking list (like Top500). I think it is actually quite undesirable for benchmark performance to be trivially computed from one parameter in machine provisioning. Depending on the fidelity of the performance model, these dimensions can be a fairly accurate predictor of performance, but ILP, compiler quality, on-node synchronization latency, cache sizes, and similar factors also matter (more for HPGMG-FE than HPGMG-FV). We see many active constraints in practice e.g., improving any of peak flop/s, memory bandwidth, network latency, or network bandwidth produces a tangible improvement in HPGMG performance. HPGMG is representative of most structure-exploiting algorithms in that it does not have this abundance of flops, thus theoretical performance is actively constrained by both memory bandwidth and flop/s.

This is a problem because architectural tricks like doubling the vector registers without commensurate improvements in bandwidth, cache sizes, load/store/gather/scatter produce huge (nearly 2x) benefit for HPL and little or no benefit to a large fraction of real applications. HPL has an abundance of flops at all scales (N^ flops on N data), so one can expect a decent fraction of peak flop/s on any architecture with enough memory and adequate cache performance. HPGMG also reports dynamic range expressed as Performance versus Time-to-solution as the problem size is varied, which allows applications to see performance barriers that might be relevant to them (e.g., see how Titan cannot do a solve in less than 200 ms while Edison can do 50 ms, and how that relates to climate simulation performance targets see slide 7 of ). HPGMG tries to strike a balance between memory bandwidth demands and compute using a matrix-free representation. If they are represented explicitly (sparse or dense), then matrix-vector product performance (thus CG as a whole) is dominated by memory bandwidth for problem sizes that do not fit in cache. Operators need not be represented as a table of numbers or a sparse matrix format, but could use a tensor product, fast transform, or other information to compute the action using less storage. Even with locality in the matrix-vector product, preconditioners often need to communicate globally in a scalable way similar to HPGMG. If there is no locality, then matrix-vector products require all-to-all communication which tend to dwarf the cost of the reductions in CG. With respect to deep learning and other applications using CG or related algorithms, the bottlenecks depend on the scale, and ability to expose locality, and operator/preconditioner representation. It can be pipelined in various ways (we have several implementations in PETSc), but performance requires a quality implementation of asynchronous reduction (e.g., MPI_Iallreduce) which the vendors have been slow about developing (I've been working with some on fixing this and Cray has made recent progress).

CG isn't truly asynchronous due to its reductions.

The sims 4 hack supercomputer