A Holistic White-Box Approach to Performance Modeling for Supercomputing

Overview & Synopsis

As high-performance computing systems push toward exascale and future zettascale capabilities, their unprecedented architectural complexity, featuring massive core counts, deep memory hierarchies, and heterogeneous components, poses significant challenges to performance modeling. Traditional performance models, often relying on decoupled computation and communication models or additive cost models that assume an idealized bulk synchronous parallel pattern, fail to capture the emergent process overlap dynamics observed in large-scale, distributed-memory programs. Disturbances can trigger large-scale, nonlinear effects that profoundly impact performance. This dissertation introduces a holistic, white-box modeling framework grounded in first principles to capture the coupled effects of noise, bandwidth bottlenecks, code execution, the message passing library, and the network in parallel applications. The work systematically formulates several fundamental challenges across three interconnected domains: (i) performance modeling, (ii) performance engineering, and (iii) performance simulation and physical modeling. It reveals how localized delays can propagate through inter-process dependencies, amplify system-wide variability, and invalidate additive- or max-based time-to-solution predictions. Through analytical derivations, controlled experiments, and large-scale validation, the work identifies critical performance phenomena, such as idle wave propagation, memory bandwidth-sharing interference, and spontaneous communication-computation overlap, as key drivers of performance variability. These effects arise not from algorithmic flaws but from fundamental hardware–software interactions under shared bottlenecks. These models are validated using both synthetic and real applications, exploring whether noise, traditionally seen as detrimental, can potentially be leveraged for beneficial purposes. The dissertation develops new analytical models: one that explains desynchronization in scalable programs via idle wave dynamics; another that models bandwidth-sharing among overlapping computational kernels and computation-communication overlap. To support controlled experimentation and reproducible simulation, two novel tools are introduced: DisCostiC, a white-box MPI performance simulator that emulates MPI applications via system-aware skeletons without requiring execution on real hardware; and OsciLite, a Kuramoto-inspired oscillator framework that captures the synchronization dynamics of parallel programs using coupled differential equations. By combining theoretical insights, simulation, and empirical validation, this dissertation provides a new lens through which the behavior of parallel programs can be understood, predicted, and optimized. Its insights and tools lay the foundation for building performance-aware software, guiding hardware co-design, and redefining performance modeling in the zettascale era.