How do I compare performance before and after a code change

To compare performance before and after a code change, measure both versions under the same conditions and compare the results.

Practical approach

1. Define what “performance” means

   • Response time / latency
   • Throughput
   • CPU, memory, I/O
   • Startup time
   • Query time, frame rate, etc.

2. Keep conditions identical

   • Same hardware
   • Same dataset / input size
   • Same environment
   • Same configuration
   • Same warm-up steps if applicable

3. Measure the baseline

   • Run the old code multiple times
   • Record average, median, and variation
   • Avoid relying on a single run

4. Measure the changed version

   • Use the same benchmark/test
   • Run multiple times
   • Compare to the baseline

5. Look at relative change

   • Percent improvement/regression: [ \frac{\text{new} - \text{old}}{\text{old}} \times 100% ]
   • Negative means faster/less resource usage, depending on metric

6. Use proper tools

   • Microbenchmarks: benchmark, pytest-benchmark, JMH, BenchmarkDotNet, etc.
   • System profiling: perf, top, htop, dtrace, pprof, flame graphs
   • App-level logging/metrics: Prometheus, Grafana, APM tools

Important tips

• Run enough iterations to reduce noise.
• Watch for caching, JIT warm-up, and GC effects.
• Measure statistically if the differences are small.
• Don’t optimize based on one “lucky” run.

Simple example

If old code takes 100 ms and new code takes 85 ms:

[ \frac{85 - 100}{100} \times 100 = -15% ]

So the new code is 15% faster.

If you want, I can also show you how to do this in a specific language or toolchain, like Python, Java, JavaScript, or C++.