Prompt = code: review, version, changelog, rollback. The model is pinned to an exact version — gpt-x-latest in production is an unpinned dependency (Hyrum’s law (16) for models: the behavior of any version is an observable contract, and a version change is a breaking change until proven otherwise).
The evidence that a model name is not a fixed thing: Chen, Zaharia and Zou, “How Is ChatGPT’s Behavior Changing over Time?” (2023), ran the same tasks against the March and June 2023 versions behind the same product names and found large changes in both directions — on some tasks accuracy dropped sharply, and the models’ willingness to follow formatting instructions shifted. Any pipeline that parsed that output broke without a single line of its own changing. A dated snapshot does not stop the provider from retiring it, but it turns a silent change into a scheduled migration you can test.
# golden set and eval results live alongside
model: claude-haiku-4-5-20251001 # dated snapshot; claude-haiku-4-5 is
# a family alias, i.e. an unpinned dependency
temperature: 0
prompt_sha: 8f3ab2
eval_baseline: { schema_ok: 0.998, amounts_ok: 0.974 } # below this, deploy is blockedThe file above is the unit of change. A prompt edit is a pull request that reruns the eval set (51) and shows the new scores next to eval_baseline; a model upgrade is the same pull request with one line changed. Production logs record the prompt hash and the model snapshot for every call (57), so a bad answer can be traced to the exact configuration that produced it, and rollback is a revert, not an archaeology session.
temperature: 0 does not give you determinism — don’t build on it. The whole setup is versioned: model snapshot and provider, system prompt, tool definitions, schemas, retrieval index version (58), runtime parameters and safety configuration. Pinning has a cost — snapshots are retired, and staying pinned too long means a forced migration under a deadline — so upgrades are scheduled, not avoided.