Determinism vs. Variance
The same input can generate different outputs. The right question isn’t “does it vary?” It’s “where can it vary?”
In one sentence
Agents are probabilistic: the same input can produce different outputs. The process needs to be designed with that in mind.
Without it
Treat everything as fixed: a rigid system — and still variable.
With it
Separate free-form fields from fixed fields: variation where it adds richness, validation where it cannot vary.
Analogy
Two excellent writers draft different letters from the same report. Neither one is wrong. Whoever hires them expects both to stay within the standard, not to write identical texts.
Example
Two generations for the same child produce letters that differ in structure and word choice. That is acceptable, and even desirable: 4,000 identical letters would sound like a form letter, which is exactly the opposite of what the program wants to convey.
What cannot vary: the facts, the data protected by LGPD, the language, and the size range.
The team explicitly separated the two lists: what is free (voice, order, emphasis) and what is fixed (facts, LGPD, format), and only the second became an eval criterion with a 100% requirement.
The common mistake
Treating all variation as a defect and trying to eliminate it by lowering temperature and making the prompt rigid. The result is a worse system that still varies.
The symmetrical mistake is more costly: tolerating variation where it is unacceptable. Variation in tone is acceptable; variation in "this child is still in the program" never is. That’s why the useful question is where the output can vary.
In practice
- Classify each output field as free or fixed. This list is a project artifact and deserves to be maintained as such.
- Fixed fields go into deterministic validation in code (schema, regex, check against the source), and never into the prompt.
- Add retry with validation, fallback, and escalation to a human when validation fails twice.
- Measure variance: run the same case 5 times and observe the spread. High variance in a fixed field is an architecture bug, and rewriting the instruction will not fix it.
How to make it tangible
A list of output fields classified as free or fixed; the fixed ones validated in code.
Connects with
Translated from Portuguese with AI assistance.