LLM
The engine. It receives text and predicts the most likely continuation. Memory, decision-making, and action are built around it.
In one sentence
A model that receives text and predicts the most likely continuation. It does not save anything between one call and the next, has no will, and does not execute anything on its own. Memory, tools, and loops are things someone builds around it.
Before
All the text goes in together
What it does
Predicts the most likely continuation
After
Text comes out
Analogy
It is the car’s engine. An engine by itself does not take anyone anywhere: it lacks a chassis, steering, brakes, and someone saying where to go. A more powerful engine still has no chassis and no brakes.
Almost every AI project that fails bought an engine and thought it had bought a car.
Example
You paste Ana’s letter and ask for a translation into Spanish. It comes back good. You close the window, open it again, and ask for “the same thing from the previous letter,” and it has no idea what you are talking about.
That is exactly how the model works. Each call starts from zero, and everything it needs to know has to go in the same request. Doing this well is called Context Engineering, and that work is on you.
The common mistake
Thinking that choosing the model is the project. The model is the easiest piece to swap out and the fastest decision to make. The expensive part is everything around it: accessible data, written rules, the definition of done, evaluation.
A company that spends six months comparing models and not a single day writing the process is optimizing the wrong part and will conclude that “AI does not work for our use case.”
The second mistake comes from the first: when the result is poor, moving up to a larger model. Most of the time, what was missing was context. A larger model with poor context gives the same wrong answer, at a higher cost.
In practice
- Pin the model and version in configuration, in a single place. A model chosen
by implicit default changes underneath you without anyone noticing.
- Measure cost per result. Token is the model’s unit of measure; result is
yours.
- When switching models, run the Golden Dataset before and after. Without that, you
only think it improved.
- Before moving up to a larger model, check whether the problem is missing context.
- The same question asked twice may produce different answers. That is
Determinism vs. Variance: it is part of how the model works and not a defect.
How to make it tangible
One line in the configuration stating which model and which version, with the key outside the code. If the model name is scattered across multiple files, it is not yet tangible.
Note
It is the replaceable piece of the system, and that is how it should be treated. Models change every few months; your Harness, your Evals, and your Ontology should not have to change with them.
If switching models forces you to rewrite the agent, the coupling is wrong, and that will get expensive the first time a better model appears.
Translated from Portuguese with AI assistance.