Debug a Copied Prompt: Placeholders, Constraints, Model Version

A prompt you copied from a library came back mediocre, and the first instinct is to say the model got worse or the prompt was oversold. In most runs I have looked at, the template was fine and the run was not: a placeholder went in blank, a constraint was trimmed off the bottom, or the call landed on a model snapshot the author never tested against. Check those three things in that order, because each check is cheaper than the one after it.

Why “bad prompt” reports are usually half-filled prompt reports

A library prompt is a spec. A single API call is one instance of that spec, and a bad instance tells you very little about the spec until you can show the instance actually matched it.

Two things break the match.

The first is editing by memory. The task sentence — “summarise this changelog”, “review this function” — is the part people keep, because it is the interesting part. The parts that get dropped are the boring ones: the output format, the length cap, the do-not list, the definition of a term the model would otherwise guess at, the instruction to ask before assuming. Those are precisely the parts that make output usable. A prompt without them is not a weaker version of the original; it is a different prompt, and you now have no evidence about the original one.

The second is the invocation layer. The text you pasted says nothing about which role it went in as, what temperature applied, what the token cap was, or which model snapshot served the request. Two runs of identical text with different values on those four are not comparable. This is the split that context engineering makes between the words and the environment around them; the note on context engineering is worth reading if that distinction is new to you.

Step 1: Audit the placeholders you actually filled

Open the rendered prompt — the text you actually sent, not the template — and look for four specific failure shapes.

• Unfilled slots. {{TONE}} still sitting in the payload. Some providers will follow the literal braces and produce generic output without complaint.
• Shrug fills. The slot is technically filled with a category instead of a value: {{AUDIENCE}} becomes “general”, {{MAX_WORDS}} becomes “short”. The model now has to invent the constraint you thought you gave it.
• Collapsed blocks. A multi-line format example pasted into a field that strips newlines. Examples only anchor format if the line structure survives the trip.
• Eaten characters. Angle-bracket placeholders in a Markdown editor become HTML. Curly braces inside a JSON example can be read as template syntax. Both corrupt the format anchor.

Write the rendered prompt to disk before you send it, then check it mechanically rather than by eye:

import re, pathlib, sys

SLOT = re.compile(r'\{\{([A-Z0-9_]+)\}\}')
template = pathlib.Path('prompt.md').read_text()
values = {
    'TOPIC': 'retry policy for a rate limiter',
    'AUDIENCE': 'backend developers',
    'FORMAT': 'JSON with keys: risk, fix, effort',
    'MAX_WORDS': '250',
}
filled = SLOT.sub(lambda m: values.get(m.group(1), m.group(0)), template)

left = sorted(set(SLOT.findall(filled)))
if left:
    sys.exit('unfilled placeholders: ' + ', '.join(left))

pathlib.Path('rendered.md').write_text(filled)

Keep rendered.md next to the response it produced. If you cannot reproduce the exact text behind a bad answer, you are guessing, and every step after this one is guesswork too.

Step 2: Find the constraints you dropped

If the fills are clean, diff your rendered prompt against the template you copied from. This catches the second failure mode, silent deletion during retyping.

git diff --no-index --word-diff library/template.md runs/rendered.md

Read the output for removed lines rather than changed ones. A dropped constraint usually looks like noise in a diff and explains the entire result.

The scope guard and grounding rules get cut first because they read like boilerplate. They are also the ones that decide whether you can trust the output, which is a separate question from whether it reads well. If the prompt sits inside a pipeline, the same discipline applies to automation you cannot watch; the habits in triage for automation that fails while you sleep transfer directly.

The broader point: a constraint you did not write down is not a constraint. It is a hope that the model shares your taste.

Step 3: Confirm the model version you pointed it at

Only after steps 1 and 2 come back clean is the model worth questioning.

Requested model and served model are different fields. Log the model string the API returns in the response object, not the one in your config. Floating aliases get repointed; dated snapshots do not, or at least not without notice. Beyond the name, three settings change results more than people expect:

• Temperature and top_p. A prompt written for deterministic extraction at temperature 0 behaves differently at 0.7, and several client libraries default higher than authors assume.
• Max output tokens. Truncation reads like the model gave up halfway. Check the finish reason before you rewrite a single sentence.
• Role placement. Instructions sent as a system message and the same text sent as a user message are not interchangeable across providers.

Put every field that changes output into one reviewable file:

# run.yaml — the whole invocation, written down
model: gpt-4o-2024-08-06        # dated snapshot; not a floating alias
temperature: 0
top_p: 1
max_output_tokens: 1200
messages:
  - role: system
    content_file: prompts/system.md
  - role: user
    content_file: runs/rendered.md
response_format:
  type: json_schema
  json_schema:
    name: review
    strict: true
    schema_file: schemas/review.json

Keep this file in version control beside the rendered prompt. When a prompt supposedly stops working, the diff between the last good run and the bad one is a two-file diff instead of an argument. The same reasoning applies when a template has several owners; versioning a template so buyers do not lose their customisations covers the file-level side of it.

What to do next

1. Write the rendered prompt to disk on every call, timestamped, and store it next to the raw response. Reproducibility first; everything else depends on it.
2. Add the placeholder check above to your test suite so an unfilled slot fails the build instead of reaching a customer.
3. Run the word-diff against the source template whenever output shape surprises you, and read the removed lines before the added ones.
4. Replace floating model aliases in config with dated snapshots, and log the served model string on every response so a silent alias change shows up as data, not as a vibe.
5. If you would rather start from templates that already carry their placeholders and constraints explicitly, the AI Prompt Library is a curated, job-organised set of copy-paste prompts for writing, coding, research and ops, sold as a one-time $39 download at https://slashmaster6.gumroad.com/l/diwoc. It does not fix a run you never logged, but it does give you a template whose constraint section is written down instead of implied.

Get AI Prompt Library

AI Prompt Library — $39, one-time payment, instant download. See the full breakdown on the review page.

Related reading

• The Ultimate Guide to AI Automation 2026
• How I Built a 10-Product Digital Business
• Building an AI Content Factory