Chart · milliseconds · public data

load ClickBench · 1M, warm.

neo 96.0 ms in the public-data run. Closest: Polars at 1.97×. Farthest: pandas at 61.2×.

—

neo crosses once

neo

Every engine's multiple of neo

  1. neo
    1×
  2. Polars
    1.97× · 189 vs 96.0
  3. Arrow
    3.89× · 373 vs 96.0
  4. DuckDB
    15.0× · 1,438 vs 96.0
  5. pandas
    61.2× · 5,876 vs 96.0

Bars to scale, from zero to the slowest engine. The gray pair is the engine's measurement and neo's in the same run, in ms.

What this measures

Read ClickBench · 1M from its path to a table in memory: 1,000,000 rows by 105 columns, 842 MB of web hits. Every engine in a fresh process each round, the file in the OS cache, each engine's start-up done before the clock; warm is the median of the loads after the first. neo's table is text, with no types, no nulls and no UTF-8 check, so the rows as text are the like-for-like ones.

Why neo is quick here

The same parser as the synthetic load, on a file this project did not make, checked column by column and row by row against DuckDB's reading of it before any time is taken.

The code that ran, this kernel only

Polars1.97× neo · neo vs Polars

Polars reading the public files · neon-cpp/bench/lang/python/standards.py:79
def load_polars(path, mode, loads, opt=None):
    import polars as pl
    kw = {}
    if opt and "infer" in opt: kw = {"infer_schema_length": opt["infer"]}
    if opt and "overrides" in opt: kw = {"schema_overrides": {c: (pl.Float64 if t == "float" else pl.String) for c, t in opt["overrides"].items()}}
    pl.read_csv(io.BytesIO(b"a,b\n1,x\n2,y\n"))  # untimed: the reader's own start-up (its thread pool), on three lines that are not the file
    out, rows, cols = [], 0, 0
    for _ in range(loads):
        t0 = time.perf_counter()
        df = pl.read_csv(path, infer_schema=False) if mode == "text" else pl.read_csv(path, **kw)
        out.append(ms_since(t0))
        rows, cols = df.height, df.width
        del df
    return pl.__version__, out, rows, cols, pl.thread_pool_size()

Arrow3.89× neo · neo vs Arrow

Arrow reading the public files · neon-cpp/bench/lang/python/standards.py:95
def load_arrow(path, mode, loads, opt=None):
    import pyarrow as pa
    import pyarrow.csv as pc
    convert = pc.ConvertOptions(column_types={n: pa.string() for n in header_names(path)}) if mode == "text" else pc.ConvertOptions()
    parse = pc.ParseOptions(newlines_in_values=True) if opt else pc.ParseOptions()
    pc.read_csv(io.BytesIO(b"a,b\n1,x\n2,y\n"))  # untimed: the reader's own start-up (its thread pools), on three lines that are not the file
    out, rows, cols = [], 0, 0
    for _ in range(loads):
        t0 = time.perf_counter()
        t = pc.read_csv(path, convert_options=convert, parse_options=parse)
        out.append(ms_since(t0))
        rows, cols = t.num_rows, t.num_columns
        del t
    return pa.__version__, out, rows, cols, pa.cpu_count()

DuckDB15.0× neo · neo vs DuckDB

DuckDB reading the public files · neon-cpp/bench/lang/python/standards.py:40
def load_duckdb(path, mode, loads, opt=None):
    import duckdb
    con = duckdb.connect()
    con.execute("SELECT 42").fetchall()  # untimed: what the binding sets up on its first query (it imports pandas) is not CSV loading
    opts = ", all_varchar=true" if mode == "text" else ""
    out, rows, cols = [], 0, 0
    for _ in range(loads):
        con.execute("DROP TABLE IF EXISTS t")
        t0 = time.perf_counter()
        con.execute(f"CREATE TABLE t AS SELECT * FROM read_csv(?, header=true{opts})", [path])
        out.append(ms_since(t0))
        rows = con.execute("SELECT count(*) FROM t").fetchone()[0]
        cols = len(con.execute("SELECT * FROM t LIMIT 0").description)
    threads = con.execute("SELECT current_setting('threads')").fetchone()[0]
    return duckdb.__version__, out, rows, cols, int(threads)

pandas61.2× neo · neo vs pandas

pandas reading the public files · neon-cpp/bench/lang/python/standards.py:111
def load_pandas(path, mode, loads, opt=None):
    import pandas as pd
    pd.read_csv(io.BytesIO(b"a,b\n1,x\n2,y\n"), engine="pyarrow")  # untimed: the reader's own start-up
    out, rows, cols = [], 0, 0
    for _ in range(loads):
        t0 = time.perf_counter()
        # engine="pyarrow": pandas' multithreaded reader, one keyword from the default (its C engine is single-threaded:
        # 1.3-1.7 s on worldcitiespop where this takes tens of ms — bench/results/standards.jsonl, the records of 09:02-09:22)
        eng = (opt or {}).get("engine", "pyarrow")
        df = pd.read_csv(path, dtype=str, keep_default_na=False, engine=eng) if mode == "text" else pd.read_csv(path, engine=eng)
        out.append(ms_since(t0))
        rows, cols = df.shape
        del df
    import pyarrow as pa
    return pd.__version__, out, rows, cols, (pa.cpu_count() if (opt or {}).get("engine", "pyarrow") == "pyarrow" else 1)

neo

The kernel's body stays in the engine. Its declaration and the guarantees it is held to are below, with the harness line that calls it.

the harness, calling the engine · neon-cpp/bench/bench_engine_options.cpp
 194  l.t = csv::parse(raw);
 337  const double f5 = timeIt([&] { Bitmap b = compareDoubles(amt.data(), n, Cmp::Gt, 500, &pool); sink = b.words[0]; }, 7);
 372  const double s4 = timeIt([&] { sortIndexByDouble(amt.data(), n, perm, &pool); sink = perm[0]; }, 3);
 454  const double a5 = G <= 16 ? timeIt([&] { groupSumAnyOrder(amtP, idP, n, G, &pool, pairTotal); sink = static_cast<size_t>(pairTotal[0]); }, 9) : 0;
the parser's contract · neon-cpp/core/csv.h:55
Table parse(std::string_view raw);  // parallel above 1 MB, serial below; identical results

Measured on Apple M5 Max, 18 cores, 26.4.1, 2026-10-11. Every number here is also on the comparison page of its engine, with the whole file it came from.

All evalsEvery comparisonStart a conversation