A deliberately weird benchmark

DoomQL

Doom E1M1, rendered by GROUP BY. Every frame is one analytical SQL query — scan millions of voxel-surface observations, clip them to the camera frustum, reduce to the nearest visible surface per ray, shade the result in a terminal. Same query, same WAD geometry, replayed across every engine.

This is not a claim that databases should be game engines. It is a latency-shaped, row-returning workload — the exact inverse of ClickBench — built to expose fixed per-query overhead that aggregate benchmarks structurally cannot see. It works: see “what the weird benchmark caught” below.
22.5 fpsauto-router, 5M-row world
5M → 200Mworld sizes, same tour
byte-identicalframe hash on every engine

DoomQL

How a frame happens

120×40 terminal frame, one query each
1 · The world is a table

Real Episode 1 geometry from the shareware WAD, expanded into repeated observations of a 256×256×16 voxel volume — 1,048,576 rows per complete observation. 5M to 200M rows total.

2 · Each frame is a query

Fixed-point camera vectors rotate the world into camera space; a frustum clip keeps what is visible; a GROUP BY reduces observations to the nearest surface per ray. Camera position changes selectivity every frame.

3 · The terminal shades it

Returned surface strips become shaded wall slices. Every engine must produce the same frame hash — the HUD prints it live, and parity is checked on every run.

WITH camera_space AS (          -- rotate world into camera space
    SELECT surface_id, depth_scaled, lateral_scaled,
           z_bottom, z_top, surface_kind, material,
           effective_light, sector_id
    FROM doomql_episode1
    WHERE map_name = 'E1M1'
      AND world_x BETWEEN cam.x - draw_dist AND cam.x + draw_dist
      AND world_y BETWEEN cam.y - draw_dist AND cam.y + draw_dist
), visible_space AS (            -- frustum + draw-distance clip
    SELECT * FROM camera_space
    WHERE depth_scaled BETWEEN near AND far
      AND lateral_scaled * focal
          BETWEEN -half_w * depth_scaled
              AND  half_w * depth_scaled
)
SELECT lateral_scaled, depth_scaled, z_bottom, z_top,
       surface_kind, material, sector_id,
       avg(effective_light) AS avg_light,
       count(*) AS samples, surface_id
FROM visible_space               -- reduce observations to visible surfaces
GROUP BY lateral_scaled, depth_scaled, z_bottom, z_top,
         surface_kind, material, sector_id, surface_id
ORDER BY depth_scaled, lateral_scaled, surface_kind, surface_id;

Episode 1 replay tour

Frame time vs world size

median ms per frame, log–log; lower is better
30ms100ms300ms1s3s5M15M50M100M200MPostgreSQL heap · 5M rows · 127ms/framePostgreSQL heap · 15M rows · 338ms/framePostgreSQL heap · 50M rows · 807ms/framePostgreSQL heap · 100M rows · 1.53s/framePostgreSQL heap · 200M rows · 2.99s/frameHydra · 5M rows · 110ms/frameHydra · 15M rows · 193ms/frameHydra · 50M rows · 582ms/frameHydra · 100M rows · 1.17s/frameHydra · 200M rows · 2.33s/frameClickHouse · 5M rows · 32.8ms/frameClickHouse · 15M rows · 44.3ms/frameClickHouse · 50M rows · 99.2ms/frameClickHouse · 100M rows · 176ms/frameClickHouse · 200M rows · 323ms/frameDuckDB (in-process) · 5M rows · 35.7ms/frameDuckDB (in-process) · 15M rows · 59.2ms/frameDuckDB (in-process) · 50M rows · 71.2ms/frameDuckDB (in-process) · 100M rows · 97.6ms/frameDuckDB (in-process) · 200M rows · 185ms/frameRVBBIT Duck/Vortex (forced) · 5M rows · 39.7ms/frameRVBBIT Duck/Vortex (forced) · 15M rows · 53.1ms/frameRVBBIT Duck/Vortex (forced) · 50M rows · 96.7ms/frameRVBBIT Duck/Vortex (forced) · 100M rows · 170ms/frameRVBBIT Duck/Vortex (forced) · 200M rows · 350ms/frameRVBBIT auto-router · 5M rows · 44.4ms/frameRVBBIT auto-router · 15M rows · 70.8ms/frameRVBBIT auto-router · 50M rows · 153ms/frameRVBBIT auto-router · 100M rows · 195ms/frameRVBBIT auto-router · 200M rows · 345ms/frame
RVBBIT auto-routerRVBBIT Duck/Vortex (forced)DuckDB (in-process)ClickHouseHydraPostgreSQL heap
5M rowsDuck / Vortex44.4ms · 22.5 fps
15M rowsDuck / Vortex70.8ms · 14.1 fps
50M rowsDuck / Vector153ms · 6.5 fps
100M rowsDuck / Vector195ms · 5.1 fps
200M rowsDuck / Vector345ms · 2.9 fps

The router shifting gears: nobody told it the world grew. It moves from the Vortex layout to the vector path as scan cost overtakes per-query overhead — each frame is routed like any other query.

System5M rows15M rows50M rows100M rows200M rows
RVBBIT Auto44.4ms70.8ms153ms195ms345ms
RVBBIT Native107ms233ms690ms996ms1.87s
RVBBIT Duck Vector98.2ms141ms145ms189ms339ms
RVBBIT Duck Vortex39.7ms53.1ms96.7ms170ms350ms
RVBBIT DataFusion Vector89.6ms136ms319ms575ms1.10s
RVBBIT DataFusion Vortex121ms212ms520ms961ms1.85s
DuckDB35.7ms59.2ms71.2ms97.6ms185ms
PostgreSQL127ms338ms807ms1.53s2.99s
Citus Columnar133ms363ms1.19s2.42s4.94s
Hydra Columnar110ms193ms582ms1.17s2.33s
AlloyDB Omni217ms583ms2.06s4.16s8.64s
ClickHouse32.8ms44.3ms99.2ms176ms323ms

Standalone DuckDB reads the exact source Parquet in-process — the most favorable possible setup, kept as an honest baseline. AlloyDB numbers at 100M+ exceed its 4GB columnar pool and degrade toward its row store. Full per-run JSON lives in bench/doomql/results/.

Why it exists

What the weird benchmark caught

four real engine bugs in one week

Every Duck query ran twice

Symptom: Frame times were exactly 2× what engine telemetry reported.

Cause: A root-owned Arrow IPC temp dir made a chmod fail, silently killing the Arrow result path after execution — every sidecar query re-ran through JSON.

Fixed in the sidecar; every duck-routed query on the box got faster.

Two ghost fleet workers

Symptom: A fixed per-frame overhead that no engine could explain.

Cause: Stale fleet_endpoints rows fired a doomed remote dispatch — plus an ORDER BY random() catalog query — on every duck query, during weeks of benchmarks.

Fleet dispatch now requires a configured token, and failed endpoints retire themselves.

11µs per output row

Symptom: 15.5ms of engine work arrived 103ms later at 8K rows per frame.

Cause: The Arrow → jsonb → tuple result path taxed every returned row. Aggregate benchmarks return a handful of rows and never noticed; a renderer returns thousands per frame.

rvbbit._engine_rows now decodes Arrow straight to typed Datums. Frames went 111 → 40ms — and ClickBench geomean dropped 46 → 41ms, TPC-DS 278 → 223ms. A Doom benchmark made TPC-DS 20% faster.

GPU crash cascade

Symptom: Forced-GPU frames didn't get slow — they took the whole database down.

Cause: In a container where postgres runs as PID 1, a crashing GPU engine process re-parents to the postmaster, which treats it as shared-memory corruption and restarts the cluster.

Run GQE containers with docker --init, plus stale shared-memory cleanup in the auto-start path.

The thesis: aggregate benchmarks measure throughput and return a handful of rows, so fixed per-query costs vanish into the noise. A renderer chasing a frame budget measures everything else — dispatch overhead, result materialization, transport, route quality — thirty times a second, with a hash check on every frame.

Methodology Notes