Performance and Profiling
Measure first, optimise the hotspot, then measure again
timeit and cProfile, you reason about how cost grows with input size (big-O), and only then do you change the part that matters — usually by picking a better algorithm or data structure, caching repeated work, or pushing tight loops into vectorised operations. The rest of the code you leave clear and simple.
- time a small snippet correctly with
timeitand avoid its common traps, - profile a whole program with
cProfileand read apstatsreport, - read big-O growth and connect it to real wall-clock time,
- replace an
O(n)membership test with anO(1)one usingset/dict, - cache repeated work with
functools.lru_cache, - recognise when to vectorise a hot loop instead of hand-writing it.
1. Measure, do not guess
The first rule of performance work is that programmer intuition about "what is slow" is usually wrong. The interpreter, the operating system, memory caches and library internals all interact in ways that are hard to predict by reading code. So the professional workflow is a loop: measure -> find the hotspot -> change one thing -> measure again.
A related rule: optimise only what matters. A function that runs once at start-up does not deserve the attention that a function called ten million times inside a loop does. Donald Knuth's famous line — "premature optimization is the root of all evil" — is really about spending effort where it does not pay. Clear code is the default; you trade clarity for speed only on the proven hotspot.
timeit) answers "which of these two small snippets is faster?". A profiler (cProfile) answers "in this whole program, where does the time actually go?". Reach for the right one.