How Numerical Errors Scale with Hardware

Overview & Synopsis

In this short technical note, we outline how to bound numerical errors with respect to computational resources. We consider algorithms with known complexity estimates that can be executed in parallel and combine these estimates with parallelization laws. We suppose that the proposed approach generalizes beyond the presented examples and argue that it provides a straightforward and fair framework for comparing algorithms, hardware systems, and software implementations. This is achieved by imposing identical resource or energy budgets on each configuration, and comparing the resulting numerical error evaluations.