Evals
Benchmark methodology
Inspect measurement conditions and evidence.
How it works
Read a benchmark's measurement notes to see its dataset, machine, and what the clock includes. Use those details to choose a fair comparison.
Related research
- A protocol for comparing engine performance
See how the study matched inputs and output checks and kept each timing boundary explicit.
- CSV load timing in Zig and n0is C++
See how the study timed first and warmed loads separately, with file reading outside the benchmark clock.
- Measuring evaluation throughput and latency
See why the study measures a whole batch separately from the time one chain takes.
Example
- Start with
Two CSV timing results
- Try this
Check both measurement notes
- Result
Compare the same work