The same functional core, imperative shell (5): an LLM call is an I/O effect of the shell. The core (parsing, routing, business rules) stays deterministic and is tested without the model; the model’s variability is measured separately, by evals, where it belongs.
The pattern predates LLMs: Gary Bernhardt’s talk “Boundaries” and his screencast “Functional Core, Imperative Shell” (2012) argued for pushing every effect — network, disk, clock, randomness — to a thin outer layer. A model call is that kind of effect, with one more property: the same request can return different output. Even temperature: 0 does not guarantee identical output from a hosted model (52). If model output is consumed in ten places, every one of them becomes flaky; if it is consumed once, parsed into a typed value (53) and handed to pure code, the flakiness has one address.
trait Completion { async fn complete(&self, req: Req) -> Result<Raw>; }
// prod: an HTTP client; test: recorded fixtures (golden traces), so the pipeline
// is tested deterministically; evals run the live model separately and less often
fn route(ticket: &Ticket, label: Label) -> Queue { /* pure: no model, no clock */ }A support-ticket pipeline: the shell fetches the ticket, calls the model for a label, and writes the result; the core maps (ticket, label) to a queue, applies SLAs and escalation rules. Core tests feed labels directly — thousands of cases a second, no network. Pipeline tests replay recorded model responses (golden traces) so a change to parsing or routing is caught deterministically. Evals (51) run the live model on a labelled set, on a schedule and on every prompt or model change, and report a score with its variance, not a pass/fail.
Limits. Recorded fixtures go stale: they prove the code handles last month’s outputs, not today’s model, so they are re-recorded on model changes. And some systems put the model inside the control flow — an agent choosing its next tool — where the fence is around each step, not around the whole.