Skip to content
BabelBabel Docs

What verification means

Babel’s design goal is that completion claims are checked by the harness, not taken from the model’s confidence.

The honesty gate

Section titled “The honesty gate”

In chat mode, a model “I’m done” answer is not accepted as success by itself:

  1. Execute tasks typically need successful writes plus policy-dependent verification.
  2. Only authoritative verifier commands — your project’s real test runners — may green completion. Package installs and ad-hoc _verify* scripts do not.
  3. Verifier-dependency tampering (repeated edits to tracked verifier scripts or package.json scripts) can block completion.
  4. A requested verified-complete outcome can be downgraded by the shared executor kernel, which is the terminal authority.

Outcome labels

Section titled “Outcome labels”
  • Verified — the required verification actually ran and passed in this session.
  • Unverified patch — changes exist; authoritative verification did not substantively run. Treat as unfinished.
  • Blocked — the harness stopped the session (isolation unavailable, tamper detection, budget limits). This is the policy working, not a crash.

Exit code honesty: exit 0 corresponds to verified/unverified-patch terminal outcomes per the product’s contract — a zero exit alone is not a proof of correct code.

What this does not prove

Section titled “What this does not prove”
  • A verified outcome is scoped to the checks that ran, in that environment. It is not a general guarantee about the code.
  • Scoped test passes are not a live-model quality score.
  • Ordinary coding-loop reliability remains under active qualification upstream; see the product’s status documentation.