Benchmarks

For developers and operators comparing SQL dialects or shard counts on their own hardware. These are single-host laptop-scale numbers — good for relative comparison, not for capacity planning. For absolute numbers and how to size a deployment, see Cloud Benchmarks.

The benchmarks in fixtures in the Dwarf repository measure end-to-end flow throughput and latency through the public API (Run = create + await) against whatever SQL dialect SEQUEL_TESTING_DSN points at. They measure what code review cannot tell you: how many flows and steps per second each dialect sustains at each shard count, and the dispatch-latency percentiles.

These numbers are reference points for relative comparison, not a capacity guarantee. They were taken on a single developer laptop with every database in a local Docker container on the same host — a co-located setup that is a dev/test shape, not production. Read the ratios and trends (which dialect is faster, how latency changes with load, what sharding does), not the absolute numbers, which reflect one laptop.

Running

Pick one dialect per invocation (the dialect is the DSN, not a Go flag). -run=X selects no unit tests so only benchmarks run; -benchtime=1000x fixes the flow count so each shard count does one measured pass.

# SQLite in-memory (no env)
go test ./fixtures/ -run=X -bench=BenchmarkFlowThroughput -benchtime=1000x

# PostgreSQL
SEQUEL_TESTING_DSN='postgres://USER:[email protected]:5432/dwarfbench_%d?sslmode=disable' \
  go test ./fixtures/ -run=X -bench=BenchmarkFlowThroughput -benchtime=1000x

# MySQL / MariaDB
SEQUEL_TESTING_DSN='USER:PASS@tcp(127.0.0.1:3306)/dwarfbench_%d' \
  go test ./fixtures/ -run=X -bench=BenchmarkFlowThroughput -benchtime=1000x

# SQL Server  (enable READ_COMMITTED_SNAPSHOT on the model database first to avoid deadlocks;
#              add &encrypt=disable against a container using a self-signed certificate)
SEQUEL_TESTING_DSN='sqlserver://sa:[email protected]:1433?database=dwarfbench_%d&encrypt=disable' \
  go test ./fixtures/ -run=X -bench=BenchmarkFlowThroughput -benchtime=1000x

SQL Server TLS. The stock SQL Server container ships a self-signed certificate whose serial number is negative, which Go’s crypto/x509 refuses to parse — every connection then dies with TLS Handshake failed: x509: negative serial number, long before any query runs. &encrypt=disable (or a real certificate) is required against such a server. This is a driver/TLS constraint, not a Dwarf one.

Swap -bench=BenchmarkFlowThroughput for -bench=BenchmarkStatePayload (use a smaller -benchtime, e.g. 100x, since large payloads are slow) to measure byte throughput across payload sizes — see the section below.

The %d is required — the harness creates an isolated, auto-dropped database per shard (and drops it on shutdown), so no manual provisioning is needed; the connecting user only needs CREATE DATABASE rights.

The custom metrics are what to look at: flows/s, steps/s, and p50_ms / p95_ms / p99_ms end-to-end latency. Go’s built-in ns/op is elapsed time per flow averaged across the concurrent submitters, which is less useful here.

Workload and Configuration

Because the tasks are no-ops, essentially all of each step’s time is its durable DB transaction (claim CAS

• transition commit). So these numbers isolate the engine’s own per-step cost — the database work and commit latency, with almost nothing else. A real workload does actual work in each task, so its task bodies, not this overhead, would dominate the time; these numbers are the engine’s floor.

Results

Hardware: Apple M1 Pro (arm64, macOS); PostgreSQL 18, MariaDB 10.11, and SQL Server 2019 each in a local Docker container on the same host.

Each cell is the median of three runs. Run-to-run variance on a shared laptop is larger than it looks: SQLite was reproducible to within 4%, but PostgreSQL spanned up to 2.1× and SQL Server 1.7× between rounds on the same configuration. A single run of any server dialect here can land far from its median, so treat differences under ~25% as noise and never quote a single pass.

SQLite (In-Memory)

PostgreSQL

MySQL / MariaDB

SQL Server

How to Read These

• Dialect ranking (1 shard): PostgreSQL is the fastest real server (≈1152 steps/s), then MariaDB (≈848), then SQL Server (≈441) — consistent with the deployment guidance that recommends PostgreSQL for production. SQLite in-memory is fastest of all, for one replica: it cannot be shared between hosts.

• Sharding on a single host neither helps nor hurts much for server databases. Across three rounds every server dialect stayed within ~15% of its own 1-shard median at 2 and 3 shards, with no consistent direction: PostgreSQL and SQL Server happened to peak at 2 shards, MariaDB at 1. Those gaps sit inside the run-to-run spread above, so the honest reading is no measurable effect, not a cost and not a gain. That is the expected result — when shards are separate databases on the same server they add connection and coordination overhead without adding hardware, and sharding is a way to scale across servers (one shard per database server — see Deployment). A single-host benchmark can only show its overhead, never its benefit, so treat the shard rows as a reliability check — proof that a multi-database engine still routes and completes every flow correctly — rather than as a scaling demo.

• SQLite is the exception, and for an instructive reason. It gets faster with more shards (460 → 672 flows/s) because each shard is a separate in-memory database with its own single-writer lock, so sharding parallelizes the one-writer-at-a-time bottleneck that is SQLite’s defining constraint. The gain lands by 2 shards and does not continue at 3. It is specific to independent in-memory databases and does not transfer to a shared server.

• Latency comes mainly from durable commits. A ~74 ms p50 for a 3-step flow on PostgreSQL works out to ~25 ms per step, and each step is a synchronously-committed transaction (claim + transition) running under 32-way concurrency on laptop Docker. A real workload adds task time on top of this, so treat these figures as the engine’s baseline latency, not the total you would see in production.

State Payload Throughput (bytes/sec)

BenchmarkStatePayload seeds each flow’s initial state with a JSON string of a given size and carries it unchanged through the three no-op steps, so every step reads it and hands it on. The headline metric is payloadMB/s = payload size × steps/sec: the rate at which caller state passes through steps. Fixed at 1 shard. (Same hardware/config as above; median of three runs at -benchtime=100x.)

Persisted bytes are not a fixed multiple of this number. The engine avoids re-storing a payload a step did not change, referring to the copy the predecessor already stored instead. How many bytes actually land on disk therefore depends on the flow’s shape: a chain that rewrites the payload at every step stores roughly one copy per step, while a chain that merely carries a payload it never touches (what this benchmark measures) stores far less than one. Size storage from your own workload’s shape, not from a constant.

payloadMB/s by payload size:

What a 1 MB payload costs — throughput drops sharply and latency rises:

Reading it:

• The byte rate rises steeply with payload size. With a tiny payload, each step’s fixed overhead dominates, so few caller bytes move per second; a larger payload spreads that fixed cost over more bytes, so the rate climbs — by roughly 20–50× from 4 KB to 1 MB on every dialect. SQLite is the only one clearly flattening by 1 MB (74 → 83 MB/s), which is its single writer reaching its limit; PostgreSQL was still climbing at the largest size measured, so its ceiling lies above this range. On a 32 vCPU cloud database, with every step writing a fresh 64 KB JSON document, PostgreSQL moved ~176 MB/s of payload (see Cloud Benchmarks).
• The dialect ranking flips compared with small state. SQLite led on tiny-state flow rate, but at 1 MB PostgreSQL is well ahead (~212 MB/s), then SQL Server (~101), MariaDB (~92), and SQLite (~83). For large payloads what matters is how well the database parallelizes big writes: PostgreSQL and MySQL spread them across many connections, whereas SQLite has a single writer that handles them one at a time, so its byte rate levels off early. So the best dialect depends on how large your state is, not just on the raw flow rate.
• SQL Server holds up at large payloads (~101 MB/s, a 1 MB flow at ~0.8 s median) because its JSON payload columns are stored as binary rather than UTF-16 text: half the bytes written and read for the same caller payload, and no large-object path.
• Large state is still expensive. A flow carrying 1 MB of state takes hundreds of milliseconds, not milliseconds, and small payloads move caller bytes 20–50× less efficiently than large ones. Workflows that move large documents should trim state early with flow.Del, and keep big blobs in object storage — passing only a key through flow state. (A flow’s final state can be up to 256 MB on all four dialects — there is no 64 KB cap.)

SQLite on a File

The tables above run SQLite in memory, where nothing reaches the disk. A durable single-replica deployment runs it on a file (Deployment), and the journal and sync settings in the file’s DSN decide what each commit costs. The test harness always opens SQLite in memory, so these runs use the load harness in bench/, one arm per DSN:

go run ./bench -dsn 'file:/tmp/dwarf_1.db?_pragma=journal_mode(WAL)&_pragma=synchronous(NORMAL)' \
  -workload linear -concurrency 8,32 -window 60s -warmup 15s

Each cell is the median of three runs, interleaved across the arms. Throughput in steps/s:

These runs use a different harness and workload from the tables above, so compare the rows with each other, not with those tables. Reading it:

• Write-ahead logging is the setting that matters most. The rollback journal blocks readers while a transaction writes: a third of WAL’s throughput, multi-second tails, and claims that timed out with SQLITE_BUSY (retried, so no step was lost).
• synchronous(NORMAL) added 18-24% over the default FULL, here. It trades durability for that: under NORMAL, a power loss can roll back the last commits.
• macOS understates what FULL costs. A sync on macOS does not flush the drive’s cache unless fullfsync is on, and with it on a commit waits for the flush: 60 steps/s. On Linux a sync does flush, so FULL costs more there than in this table. Measure on the hardware you deploy.
• A second file nearly doubles the ceiling (1.65x at 32 submitters), since each file has its own writer.
• A step took about 420 bytes of file with an empty state, in every file arm: the live and archive tables together, before any retention drops a day.

What This Does Not Measure

A single-host laptop run cannot show what production hardware does — those dimensions are measured by the Cloud Benchmarks: true horizontal scale on shard-per-server hardware, connection-pool budget and deployment sizing, the effect of real network latency, byte-throughput ceilings, and throughput at high volume (100M+ accumulated rows). Still open anywhere: a sustained multi-hour soak for drift (goroutine/memory/connection growth). This benchmark remains the per-dialect throughput/latency baseline the cloud numbers build on — the cloud benchmarks run PostgreSQL only, on the ranking measured here.

Next: Reference.