Chart · millijoules a run
sum by region, in energy.
neo 1.55 mJ in the energy run. Closest: Polars at 32.8×. Farthest: Spark at 1,518×.
Every engine's multiple of neo
- neo1×
- Polars32.8× · 50.9 vs 1.55
- DuckDB46.7× · 72.3 vs 1.55
- pandas58.5× · 90.6 vs 1.55
- Spark1,518× · 2,352 vs 1.55
Bars to scale, from zero to the slowest engine. The gray pair is the engine's measurement and neo's in the same run, in mJ.
What this measures
Total the amount column for each of five regions: a grouped sum, the kernel every dashboard runs. Energy is the same run in millijoules: what the machine billed each engine's whole process tree, with its draw at rest subtracted for the seconds of the window. neo's pool is told to rest after the kernel, as the editor tells it.
Why neo is quick here
The region column was dictionary-encoded at load, so its ids are the group numbers and no string is hashed. The sum is taken on the pool in any order only when the column is whole numbers small enough that no addition can round; otherwise it is added in row order on one thread, because a floating-point sum depends on its order, and the contract is the JavaScript loop's result to the bit.
The code that ran, this kernel only
Polars32.8× neo · neo vs Polars
timed("agg", lambda: df.group_by("region").agg(pl.col("amount").sum()))DuckDB46.7× neo · neo vs DuckDB
timed("agg", lambda: con.execute("SELECT region, sum(amount) FROM t GROUP BY region").fetchall())pandas58.5× neo · neo vs pandas
timed("agg", lambda: df.groupby("region")["amount"].sum())Spark1,518× neo · neo vs Spark
timed("agg", lambda: df.groupBy("region").agg(F.sum("amount")).collect())neo
The kernel's body stays in the engine. Its declaration and the guarantees it is held to are below, with the harness line that calls it.
194 l.t = csv::parse(raw);
337 const double f5 = timeIt([&] { Bitmap b = compareDoubles(amt.data(), n, Cmp::Gt, 500, &pool); sink = b.words[0]; }, 7);
372 const double s4 = timeIt([&] { sortIndexByDouble(amt.data(), n, perm, &pool); sink = perm[0]; }, 3);
454 const double a5 = G <= 16 ? timeIt([&] { groupSumAnyOrder(amtP, idP, n, G, &pool, pairTotal); sink = static_cast<size_t>(pairTotal[0]); }, 9) : 0;// groupsum.h — a typed column's sum, count and extremes per group: what the aggregate node's measures need
// once the column is doubles (toDoubles) and every row has a group number.
//
// The results are the ones a loop over the rows in order gives — `if (isFinite(v)) { sum += v; n++;
// if (v < min) min = v; if (v > max) max = v; }` a group, which is what the JavaScript does — to the bit:
// • counts and extremes are taken a part of the rows at a time on the pool and the parts joined in row
// order (a strict comparison keeps the earlier of two equal values, as the loop does);
// • a sum is added in row order on one thread, because a floating-point sum depends on its order —
// unless the column is whole numbers small enough that no addition can round (wholeNumbers below),
// where every order gives the same bits and the rows are summed on the pool, two a step.
// bench/bench_agg_node.cpp times each path against the node and checks it; PHD.md §30.
// The sum alone, any order, two rows a step; for tests and the benchmark (groupStats calls it when it may).
// False, and nothing written, when there are more groups than it is built for.
bool groupSumAnyOrder(const double* x, const uint32_t* id, size_t n, size_t groups, Pool* pool, double* sums);// energy.h — what a process has used, as the operating system's own ledger has it: the energy it was billed,
// its processor time, the wake-ups it caused. The one reader of that ledger, for two questions:
// what a kernel costs — the Evals page's Energy view (bench_engine_options.cpp reads it around each kernel;
// tools/energy/energy.cpp checks the counter and reads it for a command it runs;
// bench/lang/python/yardstick.py reads the same field; PHD.md section 33)
// what a process costs at rest — the idle studies (scripts/idle_cost.sh and scripts/idle_server.py take
// `neon_energy --pid <pid>` twice and subtract; PHD.md section 31.1)
//
// macOS on Apple silicon: proc_pid_rusage(RUSAGE_INFO_V6). ri_energy_nj — nanojoules billed to the process's
// threads for the processor cores they ran on, since it started. Not the memory, the disk or the screen, and
// not the machine at rest: what running this code added. (Not ri_billed_energy: against half a second of a
// busy loop that field read 0.08 mJ where ri_energy_nj read 1,887.) Beside it, processor time in nanoseconds
// — `ps` has hundredths of a second, more than a resting window uses in a minute — and the wake-ups. No
// privilege is needed, and any process of the same user can be read, so a tree of processes can be summed.
// Linux: /proc/<pid>/stat and /proc/<pid>/status — clock ticks and context switches. No energy: RAPL is one
// figure for the whole machine and root's to read.
// Elsewhere: nothing is read (ok stays false).
//
// A reading is cumulative; a measurement is the difference of two. The counter moves in steps (a thread's
// share is settled when it leaves a core or enters the kernel), so a window should be a quarter of a second
// or more: `neon_energy --check` prints how far apart the steps are on the machine it runs on.Measured on Apple M5 Max, 18 cores, 26.4.1, 2026-10-11. Every number here is also on the comparison page of its engine, with the whole file it came from.