Building Workflows on Dwarf

Building Graphs

Mon, 01 Jan 0001 00:00:00 +0000

Building Graphs

For developers defining workflows. This is the graph-authoring API: the tasks, transitions, conditions and reducers that decide what runs when. Writing the task bodies themselves is Writing Tasks.

A workflow.Graph is the definition of a workflow. You build one in code and hand it to the engine via your host’s LoadGraph. This guide covers the full graph-authoring API.

Tasks

A graph is a set of tasks connected by transitions. Bind a task with SetURL, giving a task name (its identity in the graph) and a URL (the address your host’s ExecuteTask resolves to reach it):

Writing Tasks

Mon, 01 Jan 0001 00:00:00 +0000

Writing Tasks

For developers writing the code that runs inside a workflow. It covers the workflow.Flow carrier a task receives: reading and writing state, the control signals, and handling failure. The graph that decides which task runs is Building Graphs.

A task is a function your host’s ExecuteTask runs. It receives a *workflow.Flow — the carrier holding the step’s state and control signals — does its work, writes outputs back onto the flow, and returns. This guide covers the Flow API a task uses.

Building a Host

Mon, 01 Jan 0001 00:00:00 +0000

Building a Host

For developers putting Dwarf behind a real application. It turns Access Control Is the Host’s Job into code. Read Getting Started first for the two-method Host interface.

The engine has no notion of a caller: no authentication, no authorization, no tenants. Every access decision belongs to the layer you build around it. This page builds that layer for a multi-tenant service, as one type, App, that is three things at once:

Tasks in Other Languages

Mon, 01 Jan 0001 00:00:00 +0000

Tasks in Other Languages

For developers whose tasks are, or need to be, written in a language other than Go. Read Writing Tasks first: everything there about state, outputs and errors still applies, only the transport changes.

The Dwarf engine is written in Go, and so is the host that embeds it. Your tasks don’t have to be. The host decides how each task is reached, so it can just as well call a service written in Python, TypeScript, Java or anything else, over HTTP or any other transport. Graphs are JSON too (workflow.Graph marshals to and from JSON), so LoadGraph can fetch them from a service in any language as well.

Fan-Out & Subgraphs

Mon, 01 Jan 0001 00:00:00 +0000

Fan-Out & Subgraphs

For developers building workflows that branch. Both features here are graph-authoring plus task-side API, so read Building Graphs and Writing Tasks first.

Two features let a workflow do more than a straight line: fan-out/fan-in runs work in parallel and merges the results, and subgraphs call another workflow as a child.

Fan-Out

A fan-out runs several branches concurrently from one task. There are two kinds.

Driving Flows

Mon, 01 Jan 0001 00:00:00 +0000

Driving Flows

For developers integrating Dwarf into an application. This is the reference for the calls your code makes against a running engine — starting flows, reading them back, resuming and terminating them, and managing retention. It assumes you have an engine configured; standing one up in production is Operating Dwarf.

Every interaction with a flow is a method on *engine.Engine. They fall into a few groups: creating & running, inspecting, pausing & resuming, terminating & recovering, continuing, and retention. Flows are addressed by their flow key ({shard}-{hour}-{flowID}-{token}), returned at creation; steps by their step key ({shard}-{hour}-{flowID}-{stepID}-{token}), which names its flow too.

Detecting Completion

Mon, 01 Jan 0001 00:00:00 +0000

Detecting Flow Completion

For developers whose application needs to know when a flow finished — to answer an HTTP request, send a notification, or trigger downstream work. It explains the three ways to learn a flow’s outcome and when to use each.

There are three ways to learn a flow’s outcome, and they suit different situations:

Await — block a live caller until the flow stops, then read the outcome.
Poll — Await with a bounded wait: on a ctx timeout it returns the flow’s current outcome instead of an error, so a caller whose own deadline is shorter than the flow can answer now and ask again.
Orchestration — model the follow-up work as tasks inside a workflow that calls the real work as a subgraph, so the reaction to success or failure is itself durable.

Pick Await when a caller is standing by and can wait for the result. Pick Poll when the caller is bounded by its own deadline — an HTTP handler backing a status endpoint — but the flow is not. Pick orchestration when the reaction must happen reliably regardless of who is (or isn’t) waiting — a push notification, a downstream call, a compensation.

Scheduling

Mon, 01 Jan 0001 00:00:00 +0000

Scheduling

For developers and operators. Developers set priority, fairness and time budgets per flow; operators need the same model to read a backlog. The behaviour described here is what the Runbook assumes you know.

Dwarf decides which pending step runs next, and how long each task may run. Most of this is automatic; this guide explains the knobs you control. How a step survives a crashed worker is covered in How Dwarf Works, and how a task retries a transient failure in Writing Tasks.

Testing

Mon, 01 Jan 0001 00:00:00 +0000

Testing

For developers writing tests for their own workflows. It covers the in-process harness Dwarf ships so your test suite can run real flows with no database and no transport.

Dwarf ships an in-process test harness so you can exercise real workflows — scheduling, fan-out, subgraphs, interrupts, retries — with no database to set up and no transport to stand up.

The Two Pieces

engine.NewEngineUnderTest(name) constructs an engine wired for testing: an isolated SQLite in-memory database (per name, and per shard for multi-shard tests), auto-dropped, with migrations run at Startup — no DSN. It takes a plain name rather than a testing.TB, so a harness with no *testing.T to hand over reaches it the same way a Go test does; pass t.Name() from a test. You call Startup(ctx) yourself, so any Set* configuration goes in between, and you own teardown: defer eng.Shutdown(ctx). The name is the isolation key — give several engines the same name to share one database, or distinct names to isolate them. It logs to stderr at Error for CI clues (DWARF_TEST_LOG_LEVEL overrides — info/debug for the flow-status play-by-play, silent/off to quiet it); a benchmark or fuzz target silences it outright with SetLogger(slog.New(slog.DiscardHandler)), since per-iteration logging would swamp the measurement.
hosts.Mux is an in-memory implementation of the Host interface. Register graphs and task functions on it, then register it with SetHost(mux); it implements LoadGraph and ExecuteTask from those registrations.

func TestCheckout(t *testing.T) {
 ctx := context.Background()
 mux := hosts.NewMux()

 g := workflow.NewGraph("Checkout")
 g.SetURL("Reserve", "inventory.reserve")
 g.SetURL("Charge", "billing.charge")
 g.AddTransition("Reserve", "Charge")
 g.AddTransition("Charge", workflow.End)
 mux.HandleGraph("checkout", g)

 mux.HandleTask("inventory.reserve", func(ctx context.Context, f *workflow.Flow) error {
 f.SetBool("reserved", true)
 return nil
 })
 mux.HandleTask("billing.charge", func(ctx context.Context, f *workflow.Flow) error {
 f.SetString("receipt", "r-123")
 return nil
 })

 eng := dwarf.NewEngineUnderTest(t.Name())
 defer eng.Shutdown(ctx)
 eng.SetHost(mux)
 eng.Startup(ctx)

 _, out, err := eng.Run(ctx, "checkout", map[string]any{"sku": "ABC"}, nil)
 if err != nil {
 t.Fatal(err)
 }
 if out.Status != workflow.StatusCompleted {
 t.Fatalf("status %s", out.Status)
 }
 if receipt := out.State.GetString("receipt"); receipt != "r-123" {
 t.Errorf("receipt %q", receipt)
 }
}

Mux.HandleGraph(workflowURL, graph) registers a graph under the URL you pass to Create or Run; HandleTask(taskURL, handler) registers a task under the URL bound with SetURL. The handler signature is the same func(ctx, *workflow.Flow) error you write in production.

Patterns

Mon, 01 Jan 0001 00:00:00 +0000

Patterns

For developers looking for how to express a common need. Each recipe is short and links to the guide that covers its building blocks.

An LLM Agent Loop

An agent thinks, calls tools, and repeats until it has an answer. That is a loop in the graph: Think decides, and a switch either runs the tools or ends the flow.

g := workflow.NewGraph("Agent")
g.SetURL("Think", "https://agent/think")
g.SetURL("UseTools", "https://agent/use-tools")
g.AddTransitionSwitch("Think", "UseTools", "hasToolCalls")
g.AddTransitionSwitch("Think", workflow.End, "!hasToolCalls")
g.AddTransition("UseTools", "Think")

graph LR
 classDef task fill:#32a7c1,color:#f4f2ef,stroke:#32a7c1
 classDef term fill:#e5f4f3,color:#434343,stroke:#32a7c1
 linkStyle default stroke:#32a7c1

 t0["Think"]:::task
 t0_switch{"switch"}:::term
 t1["UseTools"]:::task
 t1 --> t0
 t0 --> t0_switch
 t0_switch -->|"hasToolCalls"| t1
 _start(( )):::term --> t0
 t0_switch -->|"!hasToolCalls"| _end(( )):::term

Every turn of the loop is a durable step, so a crash resumes the agent where it was. For a conversation, end the flow when the agent has answered, and start the next turn with Continue: it carries the previous turn’s final state (the conversation so far) into a new flow. Your application keeps the conversation as a chain of flow keys, and continues the key the last Continue returned. To run several tool calls in parallel, have Think emit them as an array and fan out with forEach.