The Review Gap: Verified Output Is the Scarce Resource

Generation got cheap in a way that hid a cost transfer. Every model release makes it a little easier to produce a pull request, a migration script, a support macro or a lesson plan, and pushes the hard part one step downstream, into reading the thing before it ships. The constraint is no longer producing output. It is the number of artifacts one person can actually check and sign off on.

Where the gap shows up

The gap appears in three places, and each fails differently.

Shipping. An agent that opens pull requests overnight changes the shape of your morning. The failure is rarely that the code is obviously bad. It is that review becomes a formality: a nine-hundred-line diff gets an LGTM because reading it properly would take an hour you do not have. You then ship code that has been seen but not verified. Those are different states. A reviewed artifact has been read by someone. A verified artifact has had at least one of its claims tested against something outside the model’s own output.

The same mechanism makes quiet failures worse: automations break while nobody is watching precisely because no check closed the loop. That is the subject of triage habits for automation that fails while you sleep, and the review gap is the upstream cause of most of that triage.

Course work. Learning has the same gap in a quieter form. A tutorial hands you a working demo and a feeling of progress; it rarely hands you the ability to say what would break the demo. People finish able to reproduce the video and unable to diagnose the run on their own machine, with their own data, behind their own proxy. The demo was generated. The competence was not verified. It is why the second project is harder than the first: the first is a transcription, the second requires you to review your own understanding.

Multi-agent output. The first fix people reach for is a second agent that reviews the first. It fails predictably. Give the reviewer the same context, the same tools and the same model family, and it inherits the generator’s blind spots; it will tell you the work looks good because from where it stands, it does. Context engineering is the discipline that decides what the reviewer sees, and if the answer is “the same thing the generator saw”, you have bought a second opinion from the same brain. Real review needs independence: a different source of truth, a different tool, or a mechanical check that can fail.

Why “generated” stopped being the scarce input

Generation is parallel and cheap. Ten drafts cost roughly the same as one because the expensive part, the model itself, is already running. Verification is serial. It needs ground truth, and ground truth lives in specific places: the schema, the test suite, the endpoint that does or does not return 200, the invoice, the person who owns the account. A model can propose a claim. Only a system or a human can close it.

That asymmetry is the whole story. Your throughput is not measured in drafts produced; it is measured in claims closed per day. Once you accept that, planning changes. You stop asking how to generate more and start asking how to make each artifact cheaper to check, which is a design problem rather than a prompting problem. It is also why a plain script that runs a real check often beats a builder that only moves text around; see workflow builder vs a plain script for where that line falls.

Make output reviewable by construction

Reviewing an opaque artifact is expensive because you are reverse-engineering intent. Reviewing a contract is cheap because you are confirming claims. The cheapest thing you can add to any agent task is a claims file that separates what was asserted from what was checked.

{
  "artifact": "migrations/2026-03-orders-currency.sql",
  "producer": "agent:migrate-orders",
  "claims": [
    {
      "claim": "orders.currency exists",
      "evidence": "psql -c '\\d orders' -> currency varchar(3)",
      "checked_by": "command"
    },
    {
      "claim": "no existing rows have a null currency",
      "evidence": "SELECT count(*) FROM orders WHERE currency IS NULL -> 0",
      "checked_by": "command"
    },
    {
      "claim": "pricing service tolerates the new column",
      "evidence": "not checked",
      "checked_by": "pending"
    }
  ]
}

Then fail the build while anything is pending:

import json
import sys

REQUIRED = {"claim", "evidence", "checked_by"}
CLOSED = {"command", "human", "test"}

def problems(doc):
    out = []
    for c in doc.get("claims", []):
        missing = REQUIRED - c.keys()
        if missing:
            out.append(f"{c.get('claim', '<unnamed>')}: missing {sorted(missing)}")
        elif c["checked_by"] not in CLOSED:
            out.append(f"{c['claim']}: unverified ({c['checked_by']})")
    return out

doc = json.load(open(sys.argv[1]))
found = problems(doc)
print("\n".join(f"UNVERIFIED: {p}" for p in found) or "all claims closed")
sys.exit(1 if found else 0)

The second half of reviewability is scope. A diff you can hold in your head gets read properly; a diff that spans twenty files gets skimmed. Splitting work is cheaper than heroic reading.

No row is right for everything. The point is that “I looked at it” is the weakest row on the list, and it is the default most teams fall back to when volume rises.

Enforce a review budget

A budget you do not enforce is a preference. Put the cap in configuration next to the generator so the machine stops producing when the human falls behind. This is a small file, not a platform.

# review-gate.yaml
queue:
  max_open: 5                 # artifacts awaiting a named verifier
  block_new_work_above: 5
artifacts:
  max_diff_lines: 400
  max_files: 12
  split_before_review: true
checks:
  - name: tests
    run: pytest -q
  - name: contracts
    run: python scripts/check_claims.py contracts/*.json
  - name: docs-links
    run: lychee --no-progress docs/
merge:
  require_named_verifier: true
  allow_self_merge: false

And a gate at the top of the generation job, so new work cannot be created while the queue is full:

pending=$(find .review-queue -name '*.json' 2>/dev/null | wc -l)
if [ "$pending" -ge 5 ]; then
  echo "review queue at $pending; generating more will only widen the gap"
  exit 1
fi
echo "queue at $pending; one slot free"

Two properties matter more than the exact numbers. The verifier is a name, not a checkbox, because “the team” cannot be accountable. And self-merge is off: if the same agent that produced the change can approve it, you have automated the gap instead of closing it. Pick the caps by measuring how many artifacts one person can genuinely check in a working day of your own output, then set them slightly below that.

What to do next

1. Count your open artifacts right now: PRs, drafts, agent reports, half-read docs. If the number is larger than what you could check in ninety minutes, stop generating until it drains.
2. Add a claims file to your next agent task and make the build fail on pending. One closed claim beats ten confident paragraphs.
3. Split any diff over 400 lines before you read it. Scope is the cheapest form of review you control.
4. Give your review agent a tool the generator did not have, such as a test runner, a linter or an HTTP client, or it is only agreeing with itself.
5. If your gap is at the start, where you can generate but not verify, build one small project end to end where you can run every check yourself. Ship With AI is a four-hour practical course that takes a non-coder from zero to a first working AI automation project; it is $39 USD one-time at https://slashmaster6.gumroad.com/l/mgtpcn.

Get Ship With AI

Ship With AI — $39, one-time payment, instant download. See the full breakdown on the review page.

Related reading

• The Ultimate Guide to AI Automation 2026
• How I Built a 10-Product Digital Business
• Building an AI Content Factory