When to Reach for a Graph, Not a Chain

Most agent systems start as a chain. Step one feeds step two feeds step three, a tidy pipeline you can hold in your head and draw on a napkin. It works beautifully right up until the agent encounters something the napkin didn't anticipate — a tool that fails, a retrieval that comes back empty, a user who answers a question you didn't ask. A chain has no way to say wait, go back. It can only march forward.

The tell: when you start faking loops

You can usually feel the moment a chain has outgrown itself. It's when you catch yourself simulating a loop inside a single step — re-prompting, retrying, stuffing "if the last attempt failed, try again differently" into a prompt that is already too long. You're building a state machine by hand, in natural language, in the worst possible place to debug one.

A linear chain is a bet that nothing will surprise you. Production is the house, and the house always collects.

That's the signal to reach for a graph. Model the work as nodes and edges: each node does one thing, and the edges — not the prompt — decide what happens next. Now "retry with a different strategy" is an edge back to an earlier node, not a paragraph of pleading. The control flow lives in code you can test, instead of prose you can only hope about.

What graphs actually buy you

Two things, mostly. Cycles, so an agent can revisit, refine, and recover instead of failing forward. And explicit state, so context is something you carry deliberately between nodes rather than something you pray survives in the conversation history.

The cost is real — a graph is more machinery than a chain, and you shouldn't build one to answer a single question. But the moment your system needs to react to its own intermediate results, the graph stops being over-engineering and starts being the honest shape of the problem. The chain was always a graph. You were just hoping it wouldn't have to be.