Orchestrator / Routing
The traffic manager: expensive model for difficult tasks, cheaper model for triage.
In one sentence
The component or agent that decides which agent and which model gets which task: expensive model for difficult tasks, cheaper model for triage. It’s the operation’s traffic manager.
Before
Task comes in
What it does
Routing by difficulty and cost
After
The right model for the right case
Example
A batch of 400 reports comes in. The orchestrator sends the complete ones to the cheaper model, the ones with a missing field or contradiction to the expensive model, and those for a child who left the program straight to a human.
The average cost per letter drops by half, and no one notices any difference in the outcome.
In practice
- Start with explicit rules instead of letting a model decide. Routing is business logic.
- Measure routing accuracy: how many tasks went to the expensive model unnecessarily.
- Every route needs a default destination, and the human also counts as a destination.
How to make it tangible
A routing function with explicit rules: this type of case goes to this model.
Connects with
Translated from Portuguese with AI assistance.