Performance on Dwarf

Cloud Benchmarks

Mon, 01 Jan 0001 00:00:00 +0000

Cloud Benchmarks

For operators sizing a deployment. This page answers how much one database serves, what adding shards and engine replicas does, and what the engine itself costs. Every number was measured, and the configuration that produced it is stated beside it.

The Short Version

What it delivers

One 32 vCPU PostgreSQL database runs 18,900–24,300 steps/s with the archive on, 1.6–2.1 billion steps a day, at the engine’s shipped defaults and with the database’s checkpoints sized as recommended. The range is the instance: the slower one spent 22% more database CPU per step and took 25% longer per log flush. Check yours (Method Notes).
Payload is paid in write-ahead log. With 8 KB of JSON per step the same database runs ~10,300 steps/s, with 64 KB ~2,800 (~176 MB/s of payload). Every flush of the log costs about 0.8 ms plus its bytes, so the bytes a step logs set the rate.
Size the database’s checkpoints. With 8 KB payloads, a 20 GB max_wal_size ran 42% more steps/s than the 1.5 GB default.
32 vCPU is the most economical database size, $0.036–0.047 per million steps at full load. Smaller databases run out of CPU; larger ones are held back by the write-ahead log’s flush, so past 32 vCPU you add shards.
The engine is cheap. A shard’s work takes about 5 engine cores. Two 8 vCPU engine hosts per 32 vCPU database serve it at about a third of their CPU each, and either can carry it alone.
Shards add throughput linearly. Four 32 vCPU shards ran 91,800 steps/s, 3.8 times one shard, each shard at close to what one shard runs alone.
No flow failed in any run. In the 243 runs behind this page, across 40 sessions, no flow errored and no step was left stuck. The engine recovered a crashed worker’s step only in the runs that killed engines on purpose.
Failover takes about 40 seconds and loses nothing. With every dispatcher killed at once, dispatch paused for ~40 s, standby replicas took over, and every flow completed.
Replicas add availability, not cost. One, two or four replicas on a shard deliver the same throughput within 6%. Each shard is served by two of them, and any others stand by to take over.
Long tasks cost no throughput, and mixed durations share one engine. With 8-second tasks the engine delivered the full offered rate, and a mix of 5 ms, 200 ms and 8 s tasks ran at the latency each class predicts. 10,000 concurrent 60-second tasks ran on under one engine core.
Large state works up to the database column’s limit. 250 MB per step completed cleanly, at ~45 MB/s for one flow and ~135 MB/s for four.
A day of varied load ran without falling behind. 4.5 hours of mixed shapes, payloads, task durations and priorities, with bursts above capacity, a rolling restart and a database restart: flows completed at the rate they arrived, bursts cost only the lowest priorities a wait, and the live tables levelled off at 10 GB (A Day of Varied Load).
A growing archive costs nothing measurable. A 32 vCPU database ran saturated until its archive held 100 million step rows, and ran at its empty rate the whole way, while the live tables stayed at ~3 GB. Archived reads took 7–20 ms at every size (Database Volume).

What it means for sizing

A Day of Varied Load

Mon, 01 Jan 0001 00:00:00 +0000

A Day of Varied Load

For operators deciding whether one database carries their traffic through a day, not just at a steady rate. Four and a half hours of mixed flows on one PostgreSQL database, with bursts above its capacity, a rolling restart of the engines and a restart of the database itself. The steady-rate measurements are in Cloud Benchmarks, whose database and settings this run shares.

Sizing & Tuning

Mon, 01 Jan 0001 00:00:00 +0000

Sizing & Tuning

For operators who want to know how the engine sizes itself, or who need to override it. You don’t need this page to run Dwarf: the decisions are in Deployment, and the engine applies these numbers for you. The measurements behind them are in the Cloud Benchmarks.

Connection Pool

Each shard’s connection budget is a constant adjusted for distance and capped by the server: 250 connections up to 0.25 ms of round-trip time, rising evenly to 500 at 1 ms, and never more than half the server’s own max_connections (a fixed 8 on SQLite). The budget is shared by the shard’s two dispatching replicas, about 125 connections each at the default. The rules for overriding it are in Deployment.

Dialect Benchmarks

Mon, 01 Jan 0001 00:00:00 +0000

Benchmarks

For developers and operators comparing SQL dialects or shard counts on their own hardware. These are single-host laptop-scale numbers — good for relative comparison, not for capacity planning. For absolute numbers and how to size a deployment, see Cloud Benchmarks.

The benchmarks in fixtures in the Dwarf repository measure end-to-end flow throughput and latency through the public API (Run = create + await) against whatever SQL dialect SEQUEL_TESTING_DSN points at. They measure what code review cannot tell you: how many flows and steps per second each dialect sustains at each shard count, and the dispatch-latency percentiles.