Getting reliable structured output
When your code consumes the model's output, you need it to conform to a schema — a hand-parsed free-text response is a bug waiting to happen.
- Provider structured outputs — OpenAI's
response_format: {type: "json_schema", strict: true}constrains decoding to guarantee schema-valid JSON; Anthropic supports structured output / a forced tool call. Prefer these over "please return JSON" prompting. - If you must prompt for it — show the exact schema, use
additionalProperties: falsethinking (no extra fields), make every field required or explicitly nullable, and give one example. Then validate on your side and retry on failure.
Delimiters and prompt anatomy
Structure every non-trivial prompt: [role/system] → [instructions] → [examples] → [context, clearly delimited] → [the input] → [output format]. Use XML-style tags or headings so the model knows which part is which. Distinguish instructions from data — otherwise text inside the data can be read as an instruction (the seed of prompt injection, covered in the AI Security course).
Managing context
- More context is not always better. Irrelevant context distracts the model and triggers "lost in the middle." Include what's needed, ordered with key material first/last.
- Long inputs — put the question after the long document (models attend better to the end), and consider summarizing or chunking rather than dumping everything.
- Token budget — context costs money and latency on every call. Trim, summarize, and cache stable prefixes (provider prompt caching) to keep prompts lean.
Templates & variables
Production prompts are templates with typed variables ({user_question}, {retrieved_context}), versioned in code — not strings pasted together. This makes them testable, diff-able, and safe to iterate (next chapter).