Performance Optimization
Measure first, change one thing, and only trust numbers you can reproduce
- use a profiler to locate real hotspots instead of guessing,
- reason about the memory hierarchy and write cache-friendly data layouts,
- eliminate needless copies with references, moves,
reserve, and views, - understand what inlining and link-time optimization really do,
- write honest micro-benchmarks that survive the optimizer.
1. Profile before you optimize
The single most expensive mistake in performance work is optimizing code that was never slow. Programs spend most of their time in a small fraction of the code, and that fraction is routinely not where an experienced engineer expects. A profiler measures where time is truly spent so you fix the part that matters.
Two families of profiler are common:
- Sampling profilers (perf, VTune, Instruments) interrupt the program many times per second and record the call stack. They add almost no overhead and give a statistical picture of the hotspots. Great for a first look at a real workload.
- Instrumenting profilers (callgrind, gprof) insert counting code around every function. They give exact call counts but slow the program by a large factor and can distort tiny functions.
Optimization is also bounded by arithmetic. Amdahl's law says that if a fraction p of the runtime is spent in the part you speed up, and you make that part infinitely fast, the whole program can at best get faster by a factor of 1 / (1 - p). If 10% of the time is in a function, perfecting it can save at most 10% overall - so profile to find the big p, and ignore the rest.