Chart · millijoules a run

filter amount > 500, count, in energy.

neo 1.08 mJ in the energy run. Closest: Polars at 1.92×. Farthest: Spark at 1,313×.

—

neo crosses once

neo

Every engine's multiple of neo

  1. neo
    1×
  2. Polars
    1.92× · 2.07 vs 1.08
  3. pandas
    2.42× · 2.61 vs 1.08
  4. DuckDB
    13.7× · 14.8 vs 1.08
  5. Spark
    1,313× · 1,413 vs 1.08

Bars to scale, from zero to the slowest engine. The gray pair is the engine's measurement and neo's in the same run, in mJ.

What this measures

Count the rows whose amount is over 500: a million comparisons on a typed column, the answer one integer. Each library's count is written the way its users write it. Energy is the same run in millijoules: what the machine billed each engine's whole process tree, with its draw at rest subtracted for the seconds of the window. neo's pool is told to rest after the kernel, as the editor tells it.

Why neo is quick here

neo's filter produces a bitmap: one bit per row, 125 KB for a million rows, written at memory speed, with indices made only when something needs them. Below about a million values it runs on one thread, above that on the pool. The contract is the JavaScript node's: NaN never matches.

The code that ran, this kernel only

Polars1.92× neo · neo vs Polars

Polars, this kernel · neon-cpp/bench/lang/python/yardstick.py:218
timed("filter", lambda: df.lazy().filter(pred).select(pl.len()).collect().item())

pandas2.42× neo · neo vs pandas

pandas, this kernel · neon-cpp/bench/lang/python/yardstick.py:197
timed("filter", lambda: int((amount > 500).sum()))

DuckDB13.7× neo · neo vs DuckDB

DuckDB, this kernel · neon-cpp/bench/lang/python/yardstick.py:237
timed("filter", lambda: con.execute("SELECT count(*) FROM t WHERE amount > 500").fetchone())

Spark1,313× neo · neo vs Spark

Spark, this kernel · neon-cpp/bench/lang/python/yardstick.py:255
timed("filter", lambda: df.filter(pred).count())

neo

The kernel's body stays in the engine. Its declaration and the guarantees it is held to are below, with the harness line that calls it.

the harness, calling the engine · neon-cpp/bench/bench_engine_options.cpp
 194  l.t = csv::parse(raw);
 337  const double f5 = timeIt([&] { Bitmap b = compareDoubles(amt.data(), n, Cmp::Gt, 500, &pool); sink = b.words[0]; }, 7);
 372  const double s4 = timeIt([&] { sortIndexByDouble(amt.data(), n, perm, &pool); sink = perm[0]; }, 3);
 454  const double a5 = G <= 16 ? timeIt([&] { groupSumAnyOrder(amtP, idP, n, G, &pool, pairTotal); sink = static_cast<size_t>(pairTotal[0]); }, 9) : 0;
the filter's contract · neon-cpp/core/selection.h:1
// selection.h — the result of a filter: one bit per row. 1M rows = 125 KB,
// written at memory speed; indices are produced only when something needs them.

struct Bitmap {
    std::vector<uint64_t> words;
    size_t bits = 0;
    bool test(size_t i) const { return (words[i >> 6] >> (i & 63)) & 1; }
    size_t count() const;
    std::vector<uint32_t> indices() const;  // ascending row indices of the set bits
};

// JS measurefilter semantics: NaN never matches. Sequential below ~1M values; the pool otherwise.
Bitmap compareDoubles(const double* v, size_t n, Cmp op, double threshold, Pool* pool);
how energy is read · neon-cpp/bench/energy.h:1
// energy.h — what a process has used, as the operating system's own ledger has it: the energy it was billed,
// its processor time, the wake-ups it caused. The one reader of that ledger, for two questions:
//   what a kernel costs — the Evals page's Energy view (bench_engine_options.cpp reads it around each kernel;
//     tools/energy/energy.cpp checks the counter and reads it for a command it runs;
//     bench/lang/python/yardstick.py reads the same field; PHD.md section 33)
//   what a process costs at rest — the idle studies (scripts/idle_cost.sh and scripts/idle_server.py take
//     `neon_energy --pid <pid>` twice and subtract; PHD.md section 31.1)
//
// macOS on Apple silicon: proc_pid_rusage(RUSAGE_INFO_V6). ri_energy_nj — nanojoules billed to the process's
// threads for the processor cores they ran on, since it started. Not the memory, the disk or the screen, and
// not the machine at rest: what running this code added. (Not ri_billed_energy: against half a second of a
// busy loop that field read 0.08 mJ where ri_energy_nj read 1,887.) Beside it, processor time in nanoseconds
// — `ps` has hundredths of a second, more than a resting window uses in a minute — and the wake-ups. No
// privilege is needed, and any process of the same user can be read, so a tree of processes can be summed.
// Linux: /proc/<pid>/stat and /proc/<pid>/status — clock ticks and context switches. No energy: RAPL is one
// figure for the whole machine and root's to read.
// Elsewhere: nothing is read (ok stays false).
//
// A reading is cumulative; a measurement is the difference of two. The counter moves in steps (a thread's
// share is settled when it leaves a core or enters the kernel), so a window should be a quarter of a second
// or more: `neon_energy --check` prints how far apart the steps are on the machine it runs on.

Measured on Apple M5 Max, 18 cores, 26.4.1, 2026-10-11. Every number here is also on the comparison page of its engine, with the whole file it came from.

All evalsEvery comparisonStart a conversation