Chart · milliseconds

filter into a new table.

neo 0.380 ms in the library run. Closest: Polars at 2.34×. Farthest: DuckDB at 106×.

—

neo crosses once

neo

Every engine's multiple of neo

  1. neo
    1×
  2. Polars
    2.34× · 0.889 vs 0.380
  3. pandas
    31.1× · 11.8 vs 0.380
  4. Spark
    57.8× · 22.0 vs 0.380
  5. DuckDB
    106× · 40.3 vs 0.380

Bars to scale, from zero to the slowest engine. The gray pair is the engine's measurement and neo's in the same run, in ms.

What this measures

Keep the rows whose amount is over 500 and build a new table of them, about half the file, every column copied.

Why neo is quick here

The bitmap from the filter drives a parallel gather of every column, and the dictionary column copies its ids rather than its strings.

The code that ran, this kernel only

Polars2.34× neo · neo vs Polars

Polars, this kernel · neon-cpp/bench/lang/python/yardstick.py:218
timed("filter_mat", lambda: df.filter(pred))

pandas31.1× neo · neo vs pandas

pandas, this kernel · neon-cpp/bench/lang/python/yardstick.py:197
timed("filter_mat", lambda: df[amount > 500])

Spark57.8× neo · neo vs Spark

Spark, this kernel · neon-cpp/bench/lang/python/yardstick.py:255
timed("filter_mat", lambda: sink(df.filter(pred)))

DuckDB106× neo · neo vs DuckDB

DuckDB, this kernel · neon-cpp/bench/lang/python/yardstick.py:237
timed("filter_mat", lambda: con.execute("CREATE OR REPLACE TABLE f AS SELECT * FROM t WHERE amount > 500"))

neo

The kernel's body stays in the engine. Its declaration and the guarantees it is held to are below, with the harness line that calls it.

the harness, calling the engine · neon-cpp/bench/bench_engine_options.cpp
 194  l.t = csv::parse(raw);
 337  const double f5 = timeIt([&] { Bitmap b = compareDoubles(amt.data(), n, Cmp::Gt, 500, &pool); sink = b.words[0]; }, 7);
 372  const double s4 = timeIt([&] { sortIndexByDouble(amt.data(), n, perm, &pool); sink = perm[0]; }, 3);
 454  const double a5 = G <= 16 ? timeIt([&] { groupSumAnyOrder(amtP, idP, n, G, &pool, pairTotal); sink = static_cast<size_t>(pairTotal[0]); }, 9) : 0;
the filter's contract · neon-cpp/core/selection.h:1
// selection.h — the result of a filter: one bit per row. 1M rows = 125 KB,
// written at memory speed; indices are produced only when something needs them.

struct Bitmap {
    std::vector<uint64_t> words;
    size_t bits = 0;
    bool test(size_t i) const { return (words[i >> 6] >> (i & 63)) & 1; }
    size_t count() const;
    std::vector<uint32_t> indices() const;  // ascending row indices of the set bits
};

// JS measurefilter semantics: NaN never matches. Sequential below ~1M values; the pool otherwise.
Bitmap compareDoubles(const double* v, size_t n, Cmp op, double threshold, Pool* pool);

Measured on Apple M5 Max, 18 cores, 26.4.1, 2026-10-11. Every number here is also on the comparison page of its engine, with the whole file it came from.

All evalsEvery comparisonStart a conversation