Designing Agent Workflows You Can Actually Debug

Agent workflows rarely fail with a stack trace. A step returns a plausible-looking string, the next step treats it as valid, the run reports success, and the damage shows up three days later in a customer’s inbox. The fix is not a better prompt. It is treating the workflow like a distributed job system with contracts, logs and checkpoints.

None of that requires a platform. It requires deciding, before you write the prompt, what a step may accept and what it must produce.

Define every step as a job with an explicit contract

An agent workflow is a DAG of jobs. The model call is an implementation detail inside a step, not the step itself. If step 2 accepts free-form prose from step 1, you cannot tell whether step 2 failed or step 1 produced garbage, and you cannot test either one in isolation.

# workflow: invoice-triage@3
steps:
  - id: extract_invoice
    implementation: agent
    prompt_version: extract-invoice@7
    inputs:
      source_pdf: {type: string, required: true}
      run_id: {type: string, required: true}
    outputs:
      vendor: {type: string}
      amount_cents: {type: integer}
      due_date: {type: string}   # ISO 8601
    side_effects: none
    retries: 2

  - id: post_invoice
    implementation: tool
    inputs:
      amount_cents: {type: integer}
      vendor: {type: string}
    outputs:
      ledger_id: {type: string}
    side_effects: irreversible
    requires_approval: true

Three properties do the heavy lifting. inputs and outputs are typed, so a malformed handoff fails at the boundary instead of three steps later. side_effects tells your runner whether the step needs an idempotency key and a checkpoint. prompt_version is pinned, so a replay reproduces the run you actually had rather than the workflow you have today.

Your runner — a custom loop, a queue, n8n, Temporal — mostly decides how much of this you get for free. If you are still choosing one, the tradeoffs between the common options are covered in Zapier vs n8n vs AI Workflow Builder. The schema above is portable across all of them, which is the point.

A healthy workflow is mostly boring code. Only the fuzzy steps — parsing a messy PDF, classifying intent, drafting copy — should be agent calls. The rest should be deterministic functions you can unit test.

Make steps idempotent so retries are free

Retries are the normal case. Timeouts, rate limits and transient 500s happen, and a runner that cannot safely retry a step is a runner you will babysit. The distinction that matters is between computing a value and causing an effect. Computation is naturally repeatable; effects are not.

def post_invoice(invoice, run_id):
    idem_key = f"{run_id}:post_invoice"
    with db.transaction():
        prior = db.get_idempotency(idem_key)
        if prior:
            return prior.result          # retry becomes a no-op
        ledger_id = ledger.create_entry(
            vendor=invoice.vendor,
            amount_cents=invoice.amount_cents,
            idempotency_key=idem_key,
        )
        db.put_idempotency(idem_key, {"ledger_id": ledger_id})
        return {"ledger_id": ledger_id}

The key is derived from the run and the step, not generated fresh at call time. A random UUID per attempt defeats the entire mechanism.

Emit structured records, not chat transcripts

The transcript is a debugging aid, not a log. It is unstructured, it does not survive a prompt change, and you cannot query it. Write one JSON line per step attempt to a real log sink.

{"ts":"2026-02-11T09:14:22Z","run_id":"8f3a","step_id":"extract_invoice","attempt":1,"status":"error","error_class":"validation","inputs_hash":"b41c9e","prompt_version":"extract-invoice@7","model":"<pinned-model-id>","duration_ms":1830,"output_path":"runs/8f3a/steps/01_extract_invoice/output.json"}

The fields that pay for themselves: run_id and step_id so you can pull a single run, attempt so retry storms are visible, inputs_hash so you can tell whether a failure is bad data or bad code, and prompt_version plus model so a regression can be correlated with a change. Most useful is error_class. Split at least four ways:

• validation — the model broke the output contract.
• transport — timeout, rate limit, connection reset.
• tool — the downstream API rejected the call.
• policy — a checkpoint or guardrail refused the action.

A validation failure is a prompt problem. A tool failure is an integration problem. Logging both as error: something went wrong guarantees you will re-learn that distinction under pressure. If the boundary between what the model sees and what your code enforces is blurry, context engineering is the right frame for tightening it.

Put a checkpoint in front of anything irreversible

Ask one question per step: if this runs twice, or runs once with the wrong input, can it be undone? Money movement, outbound messages, deletions and production deploys cannot. Those get a checkpoint.

The pattern is propose-then-dispose. The agent step returns a proposal artifact and the runner halts in an awaiting_approval state. Approval is recorded as a durable row — approver, timestamp, hash of the artifact that was approved — not as a thumbs-up in a chat thread. When someone asks in six months why an invoice was posted, the row is the answer.

Because every step’s output is persisted, an approved run resumes at the checkpoint rather than restarting from the top. That matters when upstream steps cost money or touch rate-limited APIs. Restarting to re-post an already-approved invoice is exactly the duplicate you were trying to prevent.

Scope checkpoints tightly. A human reviewing a three-line proposal in five seconds is a working control. A human reviewing a wall of model reasoning is a rubber stamp, and a rubber stamp is worse than no checkpoint because it creates false confidence.

Keep run artifacts so you can replay the run

Replay is the only honest way to answer whether a change made things better or worse. Replay means re-running a recorded run with its recorded inputs and pinned versions, then diffing step outputs against the recording.

runs/2026-02-11T09-14-22Z-8f3a/
  workflow.yaml          # pinned copy of invoice-triage@3
  manifest.json          # model ids, prompt versions, tool schemas
  steps/
    01_extract_invoice/
      input.json
      output.json
      log.jsonl
  approvals/
    post_invoice.json    # approver, timestamp, artifact hash

Two rules keep replay honest. First, record inputs, not just outputs — without the exact input a step consumed, you can only re-run the whole workflow from the top, which drifts. Second, treat nondeterminism as expected: a replayed step that produces different but schema-valid output is a diff to inspect, not a bug to chase.

What to do next

1. Pick your most painful workflow and write the YAML step contracts before touching a prompt. Type the inputs and outputs, and mark each step side_effects: none, reversible or irreversible.
2. Add run_id and step_id to every log line and to every outbound call’s idempotency key. If your runner generates a fresh key per attempt, fix that first.
3. Move irreversible steps behind a record-based checkpoint with approver, timestamp and artifact hash stored in the database.
4. Start persisting one run artifact directory per production run, and set a retention window you can afford. Replay your last failed run before changing any prompt.
5. If you would rather start from definitions you can inspect and version than from a blank file, AI Workflow Builder ($99 one-time) turns plain-English prompts into validated multi-agent workflow definitions you can inspect, version and run — then add the log fields, idempotency keys and checkpoints described above.

Get AI Workflow Builder

AI Workflow Builder — $99, one-time payment, instant download. See the full breakdown on the review page.

Related reading

• The Ultimate Guide to AI Automation 2026
• How I Built a 10-Product Digital Business
• Building an AI Content Factory