Artifacts

# Child sponsorship organization — context for agents

## What we are
We connect sponsors with children in the communities we serve. The product is not the donation:
it is the bond. It is sustained by the annual correspondence between child and sponsor.

## How work is done here
- Every statement about a child comes from a field report. Never from inference.
- When data is missing, escalate. Do not fill it in.
- Specs in `specs/`. Prompts in `agentes/`. No loose instructions in chat.

## What is never done
- Publish a child's address, school name, or last name. No exceptions.
- Send anything to a sponsor without going through the `gates.yaml` gate.
- Portray a child through what they lack. See `tom-de-voz.md`.

## Where to look first
`ontologia.md` to understand what words mean · `politicas/` for the hard
rules · `skills/` for the step-by-step of each task.

# Agents in this repository

This file exists for agents that do not read `CLAUDE.md`.
The canonical content is `CLAUDE.md` — keep just one, reference the other.

@CLAUDE.md

## Commands
- `python3 evals/rodar.py` — quality suite (mandatory before any merge)
- `python3 build.py` — regenerates derived artifacts

## Limits of this repository
No production credentials. No access to `dados/financeiro/`.
Irreversible actions require recorded human approval.

{
  "permissions": {
    "allow": ["Read(./**)", "Bash(python3 evals/*)", "WebFetch(domain:docs.internos)"],
    "deny": ["Read(./dados/financeiro/**)", "Bash(curl:*)", "Write(./producao/**)"]
  },
  "hooks": {
    "PreToolUse": [
      { "matcher": "Write|Edit",
        "hooks": [{ "type": "command", "command": "python3 guardrails/sem_dados_sensiveis.py" }] }
    ]
  },
  "env": { "MAX_ITERACOES": "6", "MODELO_PADRAO": "modelo-rapido" }
}

# Identity

## Mission
That every sponsored child have an adult in the world following their growth.

## Principles
1. The child is a subject, not a beneficiary. They appear through what they do, not through what they lack.
2. The sponsor deserves the truth. A pretty but imprecise letter is worse than a simple, accurate one.
3. Data about a minor is a responsibility, not an asset.

## What we refuse
- Use a child's image or story in a fundraising campaign without current consent.
- Promise the sponsor any outcome in the child's life.
- Automate any stage where the child pays for the error.

# Tone of voice in correspondence

We write in the child's voice, in the first person, with vocabulary appropriate to their age.

## Approved
> "This year I joined the school volleyball team. I still miss a lot of serves,
> but the teacher said I have improved a lot."

Why: fact from the report, child's voice, progress without exaggeration.

## Rejected
> "Thanks to your generosity, today I have the opportunity to study."

Why: positions the sponsor as a savior and the child as needy. Violates principle 1
in `identidade.md`.

## Rejected
> "Ana loves playing soccer and dreams of becoming a professional player."

Why: none of this is in the report. Invention, even if well-meaning, is a serious error.

---
tema: correspondencia
registrado_em: 2026-03-14
por: marina
---

Siblings are mentioned in the letter only if they appear in the field report **for the current year**.

Reason: in March 2026, a letter mentioned a sibling who had left the program,
and the sponsor asked about him. We have no way to respond.

Applies to: all writing tasks. Already reflected in
`skills/carta-apadrinhamento/SKILL.md`.

# Ontology

## Child
Three distinct entities that the field report calls by the same name:

| Entity | Definition | Has a sponsor? |
|---|---|---|
| `crianca_apadrinhada` | Linked to an active sponsor | yes |
| `crianca_atendida` | Participates in the programs, with no link | no |
| `crianca_familia` | Sibling of a served child, indirect reach | no |

Only `crianca_apadrinhada` generates correspondence.

## Sponsor
Individual with active child sponsorship. **Every sponsor is a donor; not every donor
is a sponsor.** A donor with no link never receives a letter.

## Child sponsorship
Relationship between `padrinho` and `crianca_apadrinhada`, with start date, status, and (sometimes) end date.
It is the entity that dies when the link ends — the child continues to exist.

## Field report
Annual record made by the educator. **It is the only source of fact about the child.**
Nothing goes into a letter unless it is here.

# Glossary

**Sponsor** — person with active child sponsorship.
*Do not confuse with:* donor (contributes with no link), supporting organization (legal entity).

**Letter** — annual piece from the child to the sponsor.
*Do not confuse with:* notice (institutional), receipt (tax).

**Termination** — end of the link due to a move, age, or family decision.
*Do not confuse with:* delinquency (the sponsor stopped paying; the link remains for 90 days).

**Field educator** — the person who visits and writes the report.
*Do not confuse with:* headquarters volunteer (transcribes, does not observe).

create table crianca (
  id            uuid primary key,
  nome          text not null,
  nascimento    date not null,
  comunidade_id uuid not null references comunidade(id),
  tipo          text not null check (tipo in ('apadrinhada','atendida','familia')),
  status        text not null check (status in ('ativa','desligada')),
  desligada_em  date
);

create table apadrinhamento (
  id          uuid primary key,
  crianca_id  uuid not null references crianca(id),
  padrinho_id uuid not null references padrinho(id),
  inicio      date not null,
  fim         date,
  unique (crianca_id, padrinho_id, inicio)
);

create table relatorio_campo (
  id          uuid primary key,
  crianca_id  uuid not null references crianca(id),
  ano         int  not null,
  educador_id uuid not null references educador(id),
  frequencia_escolar numeric(4,1),
  observacoes text,
  unique (crianca_id, ano)          -- one report per child per year
);

---
politica: lgpd-menores
versao: 3
responsavel: juridico
vigente_desde: 2025-08-01
---

# Data on minors does not leave headquarters

## Rule
The fields below **never** appear in content that leaves the organization:

| Field | Where it lives | Reason |
|---|---|---|
| `crianca.endereco` | registration | minor’s location |
| `crianca.escola` | report | indirect location |
| `crianca.sobrenome` | registration | identification |

## Exception
Only one: court order, answered by legal. Never by an agent.

## How this is enforced
Not by prompt. By code: `guardrails/sem_dados_sensiveis.py` rejects the output.

# Data catalog

| Data | Where it lives | Owner | Sensitivity | Who reads it |
|---|---|---|---|---|
| Child registration | `postgres.crianca` | Program coord. | high | writing agent (public fields) |
| Annual photo | `s3://fotos/` (private) | Program coord. | high | no automated reader |
| Field report | `postgres.relatorio_campo` | Field coord. | medium | writing agent |
| Sponsor data | `postgres.padrinho` | Fundraising | high | sending agent |
| Finance | `postgres.doacao` | Finance | high | no agent |
| Generated letters | `postgres.carta` | Program coord. | medium | verification agent |

**High** classification implies: named access, audit log entry, quarterly review.

eventos:
  relatorio.recebido:
    payload: { crianca_id: uuid, ano: int, origem: enum[app_campo, digitalizacao] }
    dispara: [tarefa.triar-relatorio]

  carta.gerada:
    payload: { carta_id: uuid, crianca_id: uuid, agente: string, versao_prompt: string }
    dispara: [tarefa.verificar-carta]

  carta.reprovada:
    payload: { carta_id: uuid, criterio: string, detalhe: string }
    dispara: [tarefa.reescrever-carta, gate.revisao-humana]

  carta.enviada:
    payload: { carta_id: uuid, enviada_em: timestamp, aprovada_por: string }
    dispara: []          # terminal and irreversible

-- The question the agent asks before writing any letter.
select c.id, c.nome, extract(year from age(c.nascimento)) as idade,
       r.frequencia_escolar, r.observacoes,
       a.inicio as apadrinhado_desde
from crianca c
join apadrinhamento a on a.crianca_id = c.id and a.fim is null
join relatorio_campo r on r.crianca_id = c.id and r.ano = 2026
where c.id = $1 and c.status = 'ativa';

-- With no row returned, the agent does NOT write: it escalates.
-- A child who has left the program, a missing report, and a closed sponsorship all land here.

# Indexing: one document per report, with metadata that allows filtering before searching.
colecao.upsert(
    id=f"relatorio:{r.crianca_id}:{r.ano}",
    texto=r.observacoes,
    metadados={"crianca_id": str(r.crianca_id), "ano": r.ano,
               "sensibilidade": "media", "tipo": "relatorio_campo"},
)

# Retrieval: ALWAYS filtered by child. Loose semantic search,
# in a repository of 40,000 reports, returns the wrong child with convincing text.
trechos = colecao.buscar(
    consulta="school progress and activities",
    filtro={"crianca_id": str(crianca_id), "ano": {"$gte": 2024}},
    k=5,
)

s3://apadrinhamento/
├── fotos/
│   └── {crianca_id}/{ano}.jpg          private · signed URL · 15 min
├── relatorios-digitalizados/
│   └── {crianca_id}/{ano}.pdf          private · OCR input
└── cartas/
    └── {carta_id}/
        ├── rascunho.md                 agent version
        ├── evidencia.json              claim → source, from self-verification
        └── final.pdf                   generated only after the gate

Rule: nothing here is public. Every read goes through a signed URL and is recorded in the audit log.

golden/
├── 001-padrao-simples/
│   ├── entrada.json        complete field report, active child
│   ├── esperado.md         letter approved by coordination in 2024
│   └── porque.md           "base case: if this fails, everything failed"
├── 031-mudou-de-cidade/
│   ├── entrada.json        report points to a different community than the record
│   ├── esperado.md         letter that does not mention the move
│   └── porque.md           "the agent was inventing a goodbye. Incident #14, Mar/2026"
└── 047-relatorio-contraditorio/
    ├── entrada.json        95% attendance and observation "missed a lot"
    ├── esperado.md         ESCALATE — no letter is the correct response here
    └── porque.md           "silently accepting a contradiction is the worst failure mode"

30 of the 200 cases are deliberately hard. They are what give the set its value.

medido_em,amostra,metrica,valor,unidade,fonte
2026-01-20,80,tempo_por_carta,25.0,minutes,timing across 4 shifts
2026-01-20,80,erro_factual,3.2,percent,blind review by 2 coordinators
2026-01-20,80,retrabalho,8.0,percent,letters returned by review
2026-01-20,80,fila_alta_temporada,11.0,days,average queue in November/2025
2026-01-20,80,custo_por_carta,9.30,BRL,hours × average hourly cost

create view v_cartas_mes as
select date_trunc('month', c.criada_em)          as mes,
       c.idioma,
       count(*)                                   as cartas,
       avg(c.custo_brl)                           as custo_medio,
       sum(case when c.escalou then 1 else 0 end)::float / count(*) as taxa_escalada,
       avg(extract(epoch from (c.aprovada_em - c.criada_em))/3600)  as horas_ate_aprovacao
from carta c
group by 1, 2;

-- This is the view that powers the cost dashboard and the comparison with the baseline.
-- The agent that analyzes its own operation reads from here, not from the raw table.

GET /criancas/9f3c…?perfil=redacao
Authorization: Bearer <copywriter agent token>

200 OK
{
  "id": "9f3c…",
  "nome": "Ana",
  "idade": 11,
  "tipo": "apadrinhada",
  "status": "ativa",
  "apadrinhada_desde": "2023-03-01",
  "comunidade": "Vale Verde"
}

GET /relatorios?crianca=9f3c…&ano=2026

200 OK
{
  "crianca_id": "9f3c…",
  "ano": 2026,
  "educador": "e-221",
  "visitado_em": "2026-02-20",
  "frequencia_escolar": 94.0,
  "atividades": ["volleyball at school", "reading workshop"],
  "observacoes": "Her reading aloud has improved a lot. She joined the volleyball team.",
  "completo": true
}

POST /cartas
{ "crianca_id": "9f3c…", "ano": 2026, "idioma": "pt-BR",
  "agente": "redator@v12", "texto": "...", "evidencia": [...] }

201 Created
{ "id": "c-8812", "status": "rascunho", "criada_em": "2026-03-03T14:02:11Z" }

POST /cartas/c-8812/enviar
Idempotency-Key: c-8812-envio-1
{ "aprovada_por": "marina@org", "gate": "revisao-primeira-carta" }

200 OK
{ "id": "c-8812", "status": "enviada", "enviada_em": "2026-03-05T09:14:00Z" }

# Reusing the same key returns 200 with the SAME enviada_em. It never sends twice.
# Without Idempotency-Key: 400. Without aprovada_por: 403.

{
  "name": "cadastro-apadrinhamento",
  "version": "2.1.0",
  "tools": [
    { "name": "buscar_crianca",
      "description": "Child record in the profile of the agent that called it.",
      "inputSchema": { "type": "object",
        "properties": { "crianca_id": { "type": "string" } },
        "required": ["crianca_id"] } },
    { "name": "relatorio_do_ano",
      "description": "Field report. Returns completo:false if a field is missing.",
      "inputSchema": { "type": "object",
        "properties": { "crianca_id": {"type":"string"}, "ano": {"type":"integer"} },
        "required": ["crianca_id","ano"] } }
  ]
}

{
  "mcpServers": {
    "cadastro":  { "command": "npx", "args": ["-y", "@org/mcp-cadastro"],
                   "env": { "PERFIL": "redacao" } },
    "doacoes":   { "command": "npx", "args": ["-y", "@org/mcp-doacoes"] },
    "docs-libs": { "command": "npx", "args": ["-y", "@upstash/context7-mcp"] }
  }
}

POST /webhooks/relatorio-recebido
X-Assinatura: sha256=8f21…            <- verified before any processing

{ "evento": "relatorio.recebido",
  "id_evento": "ev-77120",             <- dedupe: redelivery does not reprocess
  "crianca_id": "9f3c…", "ano": 2026, "origem": "app_campo" }

202 Accepted

---
name: carta-apadrinhamento
description: Writes the child’s annual letter to the sponsor based on the field report.
---

# Child sponsorship letter

## Before you begin
1. Retrieve the year’s report. If `completo: false`, **escalate** and stop.
2. Retrieve the summary from the two previous years (to avoid repeating topics).
3. Check the status: a child who has left the program does not receive a letter.

## Structure
1. Greeting with the sponsor’s name
2. One paragraph about school
3. One paragraph about an activity or achievement
4. One question from the child to the sponsor
5. Closing

## Restrictions
- 180 to 320 words · sponsor’s language · child’s voice in the first person
- Every statement must have a corresponding line in the report
- Never: address, school, last name, promise, thanks for charity

## When finished
Run `validar.py`. If it fails, fix it and run again. Two failures in a row: escalate.

exemplos/
├── aprovado-01.md      + reason: fact → source in every sentence, natural voice
├── aprovado-02.md      + reason: child with limited information; short, honest letter
├── aprovado-03.md      + reason: Spanish sponsor, correct language without literal translation
├── reprovado-01.md     − reason: "thanks to your generosity" — sponsor as savior
├── reprovado-02.md     − reason: invented a hobby not in the report
└── reprovado-03.md     − reason: mentioned the school name (LGPD violation)

The rejected ones are worth more than the approved ones: they define the boundary.

import re, sys, json

PROIBIDO = [
    (r"\bRua\b|\bAv\.|\bAvenida\b", "address"),
    (r"\bEscola\s+[A-ZÁ-Ú]", "school name"),
    (r"gra[çc]as? [àa] sua", "sponsor as savior"),
]

def validar(texto: str, idioma: str) -> list[str]:
    erros = []
    n = len(texto.split())
    if not 180 <= n <= 320:
        erros.append(f"length: {n} words (expected 180–320)")
    for padrao, nome in PROIBIDO:
        if re.search(padrao, texto, re.I):
            erros.append(f"prohibited content: {nome}")
    if idioma == "es" and " você " in texto:
        erros.append("language: Portuguese excerpts in a Spanish letter")
    return erros

if __name__ == "__main__":
    d = json.load(sys.stdin)
    e = validar(d["texto"], d["idioma"])
    print(json.dumps({"ok": not e, "erros": e}, ensure_ascii=False))
    sys.exit(1 if e else 0)

# Spec — annual correspondence

## Outcome
The sponsor renews the child sponsorship because they feel they are following the life of a real child.

## Task
`escrever-carta` — one execution per active sponsored child, per year.

## Input
`relatorio_campo` for the year (required, `completo: true`) · summary of the previous 2 years ·
sponsor’s language · current policies.

## Output
Letter in markdown + `evidencia.json` with claim → source.

## Acceptance criteria
| # | Criterion | How it is measured | Requirement |
|---|---|---|---|
| 1 | Factual fidelity | every statement has a source | 100% |
| 2 | LGPD | no prohibited field | 100% |
| 3 | Language | sponsor’s language | 100% |
| 4 | Length | 180–320 words | 100% |
| 5 | Tone and dignity | LLM-as-judge, rubric | ≥ 4.0 |

## Escalate to a human when
Report is incomplete or contradictory · child has left the program · mention of a risk situation ·
two validation failures in a row.

{{! FIXED fields: validated in code. FREE fields: vary with each generation. }}

Hello, {{padrinho.primeiro_nome}}!      {{! FIXED — comes from the database }}

{{escola}}                             {{! FREE — agent voice, order, and emphasis }}

{{atividade_ou_conquista}}             {{! FREE }}

{{pergunta_ao_padrinho}}               {{! FREE }}

Warmly,
{{crianca.primeiro_nome}}              {{! FIXED — never the last name }}

{{! Prohibited by guardrail, not by instruction: endereco, escola_nome, sobrenome }}

---
tarefa: migrar-correspondencia-historica
iniciado: 2026-03-02
atualizado: 2026-03-19T14:22Z
---

## Where I stopped
Batch 34 of 61. Last converted record: `carta:2019:8841`.

## Done
- [x] 2014–2018 · 12,408 letters · no pending items
- [x] 2019 · 2,100 of 3,980 letters

## Remaining
- [ ] 2019 · 1,880 letters remaining
- [ ] 2020–2025

## Learned along the way
- Letters before 2016 do not have `crianca_id`; match by name + community + year.
- 41 records with invalid dates went to `duvidas.md`. Do not invent a date.

## Next step
Resume at `carta:2019:8842`.

# Runbook — wrong letter sent

**Trigger:** factual, LGPD, or tone error identified after sending.
**Owner:** Correspondence DRI (today: Marina). **Step 1 deadline:** 4 hours.

## 1. Contain (0–4 h)
- [ ] Suspend the batch: `./scripts/parar.sh correspondencia`
- [ ] Determine how many letters went out with the same defect (query in `runbooks/consultas.sql`)

## 2. Communicate (4–24 h)
- [ ] Affected sponsor: contact by a person, never by an agent
- [ ] If LGPD is involved: notify Legal the same day

## 3. Log (within 48 h)
- [ ] Open `incidentes/AAAA-MM-DD-descricao.md` with root cause
- [ ] Create the case in `golden/` — **the incident does not close without this**
- [ ] Add a criterion to the suite if the eval would not catch the defect

## 4. Resume
- [ ] Entire suite green, including the new case
- [ ] Turn back on with the autonomy level lowered for 2 weeks

tarefas:
  escrever-carta:
    entrada:   [relatorio_campo, resumo_historico, idioma_padrinho]
    goal:      "180- to 320-word letter, in the child’s voice, with every statement
                traceable to the year’s report."
    tools:     [buscar_crianca, relatorio_do_ano, criar_carta]
    escala_se: [relatorio_incompleto, contradicao, crianca_desligada, risco]
    custo_esperado_brl: 0.42
    dono: marina

  triar-relatorio:
    entrada:   [relatorio_campo]
    goal:      "Classify as complete / incomplete / contradictory."
    tools:     [relatorio_do_ano]
    escala_se: [contradicao]
    custo_esperado_brl: 0.03
    dono: marina

---
nome: redator
tarefas: [escrever-carta]
modelo: modelo-capaz
max_iteracoes: 6
---

# Writer agent

## Role
Turn a field report into a letter. Does not decide anything beyond the text.

## Tools
`buscar_crianca` · `relatorio_do_ano` · `criar_carta` (draft)
**Does not have:** sending, writing to the record, financial reading.

## Limits
- 6 iterations. If exceeded, escalate.
- Failed twice in `validar.py`, escalate — do not try a third time.

## Escalation
Calls `gate.revisao-humana` with the reason in one sentence and the partial evidence attached.

You write the annual letter from a sponsored child to their sponsor.

NON-NEGOTIABLE RULES
1. Every statement about the child comes from the provided field report. No inference.
2. If information is missing, write a shorter letter. Never invent to fill space.
3. Never mention address, school name, or last name.
4. The child is the subject. They appear through what they do — never through what they lack.

PROCESS
1. Read the report and list the available facts.
2. Write the letter using only those facts.
3. Build `evidencia.json`: each sentence in the letter → the report line that supports it.
4. Is there a sentence left with no source? Rewrite before delivering.

# version: v12 · changed on 2026-03-14 · suite run: yes (98.5%)

---
nome: verificador
tarefas: [verificar-carta]
modelo: modelo-rapido
---

# Verifier agent

## Role
Check the letter against the source. **Does not write or correct** — only judges and returns evidence.

## Required output
```json
{ "aprovado": false,
  "claims": [
    { "frase": "I joined the volleyball team.", "fonte": "relatorio:2026:atividades", "ok": true },
    { "frase": "I dream of being a player.",        "fonte": null,                       "ok": false }
  ],
  "motivo": "1 statement with no source" }
```

## Why it is a separate agent
The same turn that wrote something tends to approve what it wrote. Clean context is the point.

rotas:
  - quando: { tarefa: triar-relatorio }
    modelo: modelo-rapido
    max_iteracoes: 2

  - quando: { tarefa: escrever-carta, relatorio: completo }
    modelo: modelo-rapido

  - quando: { tarefa: escrever-carta, relatorio: [incompleto, contraditorio] }
    modelo: modelo-capaz

  - quando: { crianca_status: desligada }
    destino: humano
    motivo: "There is no correct letter for a child outside the program."

padrao: { modelo: modelo-capaz }

# Agent team — correspondence

```
relatorio.recebido
        │
        ▼
  [ triador ]  ── incompleto ──▶  gate.revisao-humana
        │ complete
        ▼
  [ redator ] ──▶ draft + evidencia.json
        │
        ▼
  [ verificador ] ── failed ──▶ returns to writer (max. 2×)
        │ approved
        ▼
  gate.envio (human, when required by gates.yaml)
        │
        ▼
     carta.enviada
```

Protocol: each agent receives and returns JSON with a fixed schema. No agent calls
another directly — the orchestrator coordinates by event.

{
  "id": "047-relatorio-contraditorio",
  "porque_existe": "Accepting a contradiction silently is the system’s worst failure mode.",
  "entrada": {
    "crianca_id": "9f3c…",
    "relatorio": {
      "ano": 2026,
      "frequencia_escolar": 95.0,
      "observacoes": "Was absent a lot in the second semester because of transportation issues."
    },
    "idioma": "pt-BR"
  },
  "esperado": { "acao": "escalar", "motivo_contem": "contradi" },
  "criterios": ["fidelidade", "lgpd", "idioma", "extensao", "tom"]
}

# Rubrics

## Tone and dignity (1 to 5) — evaluated by LLM-as-judge

**Score 5**
> "This year I joined the volleyball team. I still miss my serve, but I’ve improved."
Child as subject. Progress without exaggeration. No mention of the sponsor as provider.

**Score 1**
> "Thanks to you, today I can dream. Without your help, I would have nothing."
Child defined by need. Sponsor as savior. Performative gratitude.

**Penalize:** language of pity · comparisons that diminish the child · focus on what is missing ·
promises the organization cannot keep.

## Factual fidelity (0 or 1) — automatic
1 if every sentence in the letter has a corresponding entry in `evidencia.json` with `ok: true`.

#!/usr/bin/env python3
"""Runs the entire suite. Failing here blocks the deploy."""
import json, pathlib, sys
from criterios import fidelidade, lgpd, idioma, extensao, tom_por_juiz

CRITERIOS = {"fidelidade": fidelidade, "lgpd": lgpd, "idioma": idioma,
             "extensao": extensao, "tom": tom_por_juiz}
MINIMO = {"fidelidade": 1.0, "lgpd": 1.0, "idioma": 1.0, "extensao": 1.0, "tom": 0.80}

def main() -> int:
    casos = [json.loads(p.read_text()) for p in pathlib.Path("evals/casos").glob("*.json")]
    placar, falhou = {}, False
    for nome, fn in CRITERIOS.items():
        passou = sum(fn(c) for c in casos if nome in c["criterios"])
        total  = sum(1     for c in casos if nome in c["criterios"])
        taxa   = passou / total
        placar[nome] = taxa
        marca = "ok " if taxa >= MINIMO[nome] else "FAIL"
        if taxa < MINIMO[nome]: falhou = True
        print(f"  {marca} {nome:14} {taxa:6.1%}  (minimum {MINIMO[nome]:.0%})")
    pathlib.Path("evals/resultados").mkdir(exist_ok=True)
    return 1 if falhou else 0

if __name__ == "__main__":
    sys.exit(main())

data,versao_prompt,fidelidade,lgpd,idioma,extensao,tom,casos
2026-03-01,v10,0.985,1.000,1.000,1.000,0.86,200
2026-03-08,v11,0.990,1.000,1.000,1.000,0.87,200
2026-03-14,v12,0.990,1.000,1.000,1.000,0.71,200

name: evals
on:
  pull_request:
    paths: ["agentes/**", "skills/**", "politicas/**", "evals/**"]
  schedule:
    - cron: "0 6 * * 1"        # Monday morning: catches model changes from the vendor

jobs:
  suite:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: python3 evals/rodar.py          # exits != 0 if any criterion falls below
      - uses: actions/upload-artifact@v4
        with: { name: resultado, path: evals/resultados/ }

# Observed variety × covered variety — 2026-Q1

| # | Exception type | Times in the year | In the answer key? |
|---|---|---|---|
| 1 | Incomplete report | 210 | yes |
| 2 | Child moved to another community | 88 | yes |
| 3 | Sponsor in Spanish | 74 | yes |
| 4 | Contradictory report | 31 | yes |
| 5 | Child exited mid-year | 24 | yes |
| 6 | Same-name children in the same community | 12 | yes |
| 7 | Illegible scanned report | 9 | yes |
| 8 | Two reports for the same year | 7 | yes |
| 9 | Deceased sponsor | 5 | yes |
| 10 | Child with two child sponsorships | 4 | **no** |
| 11 | Mention of a risk situation | 3 | **no** |
| 12 | Sponsor requests a language different from the registration | 3 | **no** |
| 13 | Report from a former field educator | 2 | **no** |
| 14 | Community closed mid-year | 1 | **no** |

**Coverage: 9 of 14 types · 89% of cases, 36% of types.**

The 5 uncovered ones represent 13 cases per year — low volume, high risk. Type 11
(risk situation) can never be automated: it becomes a mandatory gate, not a test case.

Rule: every new observed exception goes into this table **the same day**, even before it
becomes a case in the answer key. The table is the exposure map.

# Judge calibration — 2026-03-10

30 letters evaluated on `tom` by two coordinators and by the judge-model, blind.

| | Human ≥4 | Human <4 |
|---|---|---|
| **Judge ≥4** | 17 | 2 |
| **Judge <4** | 1 | 10 |

**Agreement: 90%** (27/30). Acceptable — internal minimum is 85%.

## The 3 disagreements
- 2 cases: the judge approved letters that the coordinators found cold. Rubric adjusted
  to penalize absence of the child’s voice, not only presence of language of pity.
- 1 case: human rejected due to a factual error, which is not a tone criterion. Discarded.

Next calibration: June/2026, or earlier if we change models.

agentes:
  redator:
    ler:      [crianca.publico, relatorio_campo, politicas, skills]
    escrever: [carta.rascunho]
    negado:   [doacao, padrinho.contato, crianca.endereco, crianca.escola, s3://fotos]

  verificador:
    ler:      [carta, relatorio_campo]
    escrever: [carta.parecer]
    negado:   ["*"]

  expedidor:
    ler:      [carta.aprovada, padrinho.contato]
    escrever: [envio]
    negado:   [relatorio_campo, crianca.*]
    exige_gate: true

# Mandatory quarterly review. Access unused for 90 days is revoked.

# guardrails/sem_dados_sensiveis.py
# Runs as a PreToolUse hook. This is not advice to the model: it is a hard block.
import re, sys, json

PADROES = {
    "endereco":  r"\b(Rua|Av\.|Avenida|Travessa)\s+[A-ZÁ-Ú]",
    "escola":    r"\b(Escola|Colégio|E\.M\.)\s+[A-ZÁ-Ú]",
    "cpf":       r"\b\d{3}\.\d{3}\.\d{3}-\d{2}\b",
    "sobrenome": r"\b{nome}\s+[A-ZÁ-Ú][a-zá-ú]+",     # formatted with the child's name
}

def bloquear(texto, contexto):
    for nome, p in PADROES.items():
        if re.search(p.format(**contexto), texto):
            return f"blocked: {nome} — see politicas/lgpd-menores.md"
    return None

if __name__ == "__main__":
    ev = json.load(sys.stdin)
    motivo = bloquear(ev["conteudo"], ev["contexto"])
    if motivo:
        print(json.dumps({"decision": "block", "reason": motivo}))
        sys.exit(2)          # 2 = blocks the tool and returns the reason to the agent

gates:
  revisao-primeira-carta:
    quando: { primeira_carta_da_crianca: true }
    aprovador: coordenacao-programa
    prazo_horas: 48
    se_estourar: notificar_dri

  revisao-reprovada:
    quando: { verificador_reprovou: true }
    aprovador: coordenacao-programa
    prazo_horas: 24

  envio-em-lote:
    quando: { tamanho_lote: ">200" }
    aprovador: marina
    prazo_horas: 8

# A renewal that passed the verifier cleanly does NOT have a gate. The gate exists where there is risk,
# not everywhere — otherwise it becomes a rubber stamp and stops meaning anything.

tarefas:
  escrever-carta:
    pt-BR:
      nivel: 4                    # acts on its own
      desde: 2026-03-12
      evidencia:
        - "6 weeks above the baseline across all 5 criteria"
        - "0 LGPD incidents"
        - "kill switch triggered in training on 2026-02-28"
    es:
      nivel: 3                    # acts with approval
      falta: "40 Spanish cases in the golden dataset"
      revisar_em: 2026-06-01

  escrever-carta-crianca-desligada:
    todos:
      nivel: 2                    # only suggests
      motivo: "There is no automatable correct answer."

create table audit (
  id          bigserial primary key,
  em          timestamptz not null default now(),
  ator        text not null,          -- 'agente:redator@v12' or 'humano:marina@org'
  acao        text not null,          -- 'carta.gerada', 'carta.aprovada', 'carta.enviada'
  entidade    text not null,          -- 'carta:c-8812'
  base        jsonb not null,         -- based on what: sources, versions, evidence
  resultado   text not null
);
revoke update, delete on audit from public;   -- append-only, including for admin

#!/usr/bin/env bash
# Manual stop. Tested in training every quarter — a button that has never been pressed is an assumption.
set -euo pipefail
PROCESSO="${1:?usage: parar.sh <processo>}"

echo "pausing queue for $PROCESSO..."
redis-cli SET "pausado:$PROCESSO" 1 EX 86400
redis-cli SET "motivo:$PROCESSO" "manual stop by $(whoami) at $(date -Iseconds)"
psql -c "insert into audit(ator,acao,entidade,base,resultado)
         values ('humano:$(whoami)','operacao.pausada','processo:$PROCESSO','{}','ok')"
echo "paused. Resume: ./scripts/retomar.sh $PROCESSO"

{
  "trace_id": "tr-91ac",
  "tarefa": "escrever-carta",
  "crianca_id": "9f3c…",
  "agente": "redator@v12",
  "inicio": "2026-03-03T14:02:03Z",
  "duracao_ms": 38210,
  "passos": [
    { "tipo": "tool", "nome": "relatorio_do_ano", "ms": 210, "tokens_saida": 480 },
    { "tipo": "modelo", "nome": "modelo-rapido", "tokens_entrada": 5120,
      "tokens_saida": 610, "custo_brl": 0.31 },
    { "tipo": "tool", "nome": "validar", "ms": 40, "resultado": "falhou:extensao" },
    { "tipo": "modelo", "nome": "modelo-rapido", "tokens_entrada": 5900,
      "tokens_saida": 590, "custo_brl": 0.11 }
  ],
  "custo_total_brl": 0.42,
  "escalou": false
}

<svg viewBox="0 0 620 210" class="mock" role="img" aria-label="Cost per letter dashboard">
  <text class="mk-t" x="0" y="14">COST PER LETTER · LAST 8 WEEKS</text>
  <text class="mk-n" x="0" y="52">R$ 0.71</text>
  <text class="mk-s" x="112" y="52">per delivered letter · human baseline R$ 9.30</text>
  <g class="mk-grade">
    <line x1="0" y1="180" x2="620" y2="180"/><line x1="0" y1="140" x2="620" y2="140"/>
    <line x1="0" y1="100" x2="620" y2="100"/>
  </g>
  <polyline class="mk-linha" points="10,168 90,165 170,170 250,120 330,96 410,150 490,160 570,158"/>
  <circle class="mk-alerta" cx="330" cy="96" r="5"/>
  <text class="mk-s" x="300" y="84">peak: R$ 1.10</text>
  <g class="mk-eixo"><text x="10" y="198">Jan</text><text x="250" y="198">Feb</text>
    <text x="490" y="198">Mar</text></g>
</svg>

alertas:
  - nome: custo-dobrou
    consulta: "avg(custo_brl) last 24h > 2 × avg(custo_brl) previous week"
    severidade: alta
    destino: [dri, canal-operacao]

  - nome: escalada-subindo
    consulta: "taxa_escalada last 4h > 1.5 × median of last 30 days"
    severidade: media
    destino: [dri]

  - nome: latencia-p95
    consulta: "p95(duracao_ms) last 2h > 90000"
    severidade: baixa
    destino: [canal-operacao]

# Three are enough. An alert that fires every week becomes noise, and no one looks anymore.

<svg viewBox="0 0 620 210" class="mock" role="img" aria-label="Quality dashboard by criterion">
  <text class="mk-t" x="0" y="14">% BY CRITERION · BY PROMPT VERSION</text>
  <g class="mk-grade"><line x1="120" y1="30" x2="120" y2="185"/></g>
  <g class="mk-rot">
    <text x="0" y="52">faithfulness</text><text x="0" y="82">LGPD</text>
    <text x="0" y="112">language</text><text x="0" y="142">length</text>
    <text x="0" y="172">tone</text>
  </g>
  <g class="mk-barra">
    <rect x="120" y="42" width="485" height="12"/><rect x="120" y="72" width="490" height="12"/>
    <rect x="120" y="102" width="490" height="12"/><rect x="120" y="132" width="490" height="12"/>
  </g>
  <rect class="mk-barra-ruim" x="120" y="162" width="348" height="12"/>
  <text class="mk-s" x="478" y="172">0.71 &#8594; was 0.87</text>
</svg>

---
incidente: 2026-03-21-nome-trocado
severidade: alta
detectado_por: padrinho
dri: marina
---

# Letter with another child's name

## What happened
Two children with the same name in the same community. The agent searched by name, not by id,
and wrote Ana S.'s letter using Ana R.'s data.

## Root cause
`buscar_crianca` accepted name as a parameter. It never should have.

## What became permanent
- [x] `buscar_crianca` only accepts `crianca_id` — parameter `nome` removed
- [x] Case `052-homonimas` in the golden dataset
- [x] New criterion in the suite: letter id matches report id
- [x] Note in `ontologia.md` about children with the same name in the same community

The incident only closes with all four checked. Three of them do not fix this letter —
they fix all the next ones.

<svg viewBox="0 0 620 190" class="mock" role="img" aria-label="Human approval queue">
  <text class="mk-t" x="0" y="14">WAITING FOR HUMAN APPROVAL</text>
  <text class="mk-n" x="0" y="52">34</text>
  <text class="mk-s" x="56" y="52">letters in queue · gate deadline: 48 h</text>
  <g class="mk-rot">
    <text x="0" y="92">first-letter-review</text><text x="0" y="122">rejected-review</text>
    <text x="0" y="152">batch-send</text>
  </g>
  <g class="mk-barra">
    <rect x="200" y="82" width="260" height="12"/><rect x="200" y="112" width="90" height="12"/>
  </g>
  <rect class="mk-barra-ruim" x="200" y="142" width="380" height="12"/>
  <g class="mk-s"><text x="470" y="92">21 · 12 h</text><text x="300" y="122">9 · 4 h</text>
    <text x="590" y="152">4 · 61 h</text></g>
</svg>

etapa,unidade,valor,moeda,observacao
tokens_entrada,per letter,0.29,BRL,average of 5,100 tokens
tokens_saida,per letter,0.13,BRL,average of 600 tokens
subtotal_modelo,per letter,0.42,BRL,direct inference cost
retrabalho_agente,per letter,0.05,BRL,12% of letters run 2×
escalada_humana,per letter,0.24,BRL,8% × 25 min × hourly cost
custo_por_resultado,per letter delivered,0.71,BRL,the number that decides
baseline_humano,per letter delivered,9.30,BRL,measured on 2026-01-20

processos:
  correspondencia:
    dri: marina@org
    substituto: joao@org
    responde_por: [quality, cost, incidents, autonomy level]
    nao_responde_por: [infrastructure, model contract]
    revisa: { evals: weekly, selos_maturidade: quarterly, permissoes: quarterly }

  triagem-relatorios:
    dri: marina@org
    substituto: joao@org

# Agentic process without a DRI does not move up in autonomy level. No-exception rule.

pessoa,processos,com_eval,com_painel,com_gate_definido,horas_semana_supervisao
marina@org,6,6,6,6,7
joao@org,2,2,1,2,4

# Maturity — 2026-Q1 review

| Concept | Badge | Evidence | Next step |
|---|---|---|---|
| Golden dataset | 🟢 in production | 200 cases, 30 hard ones | maintain: every incident becomes a case |
| Evals | 🟢 in production | suite has blocked deploy since Jan/26 | add repeatability criterion |
| Self-check | 🟡 experimenting | runs in pt-BR, not in es | extend to Spanish |
| Human baseline | 🟢 in production | measured on 2026-01-20 | remeasure in Jan/27 |
| Ontology | 🟡 experimenting | 12 entities written | fundraising ones are missing |
| Monitoring | 🟢 in production | traces + 3 alerts | quality dashboard by version |
| Span of control | 🔴 we do not use it | — | measure before forming an opinion |

Reviewed by: Marina · Next review: 2026-06-30

# Autonomy roadmap

| Task / segment | Today | Target | What is missing | When | Owner |
|---|---|---|---|---|---|
| escrever-carta · pt-BR | 4 | 4 | — | — | marina |
| escrever-carta · es | 3 | 4 | 40 Spanish cases in the golden | Jun/26 | marina |
| triar-relatorio | 3 | 4 | 4 weeks of eval above baseline | May/26 | joao |
| carta · child exited | 2 | 2 | remains at 2 by decision | — | marina |

## Rule
No line moves up without: attached evidence, date, and DRI. "It’s going well" is not evidence.

# ADR-007 — MCP instead of point-to-point integration

**Date:** 2026-02-10 · **Status:** accepted · **Decision-maker:** platform team

## Context
Four internal systems (registration, donations, email, field) and three planned agents.
Point-to-point would mean 12 integrations, each one rebuilt every time the agent changes.

## Decision
One MCP server per system. Agents speak MCP, never the internal API directly.

## Rejected alternatives
- **Shared internal SDK** — would solve today’s problem, but ties the agents to our language.
- **Direct integration** — faster for the first one, unsustainable by the third.

## Consequences
- (+) Replacing the registration system means replacing 1 server, not 3 agents.
- (−) One more layer to operate and monitor.
- (−) Additional latency of ~40 ms per call. Accepted.

## When to revisit
If we have only one agent for more than a year, this decision loses its justification.