Free lesson · AI Engineering

Anatomy of an Effective Prompt

Easy11 min readaillmrag

Anatomy of an Effective Prompt

The Prompt That Taught Me Structure

Early in my prompt engineering education - before I would have called it that - I wrote a system prompt for a code review assistant. It was a single paragraph: "You are a senior code reviewer. Review the code the user submits and provide feedback on bugs, performance issues, and style violations. Be thorough but constructive."

The output was... fine. It found some bugs. It mentioned some style issues. It was roughly what a junior engineer would produce in a casual Slack message. Not useless, not impressive, not reliable.

A month later, after studying the prompts behind some of Meta's internal tools, I rewrote it. The new prompt was structured - role, then context, then specific instructions, then output format, then examples, then constraints. It was 800 tokens instead of 50. The output was unrecognisable: categorised feedback, severity ratings, specific line references, and a summary recommendation. Same model. The structure of the prompt was the entire difference.

This topic gives you that structure. Not as a rigid template - rigid templates produce rigid, formulaic prompts - but as a set of building blocks that effective prompts are assembled from, with the design reasoning for each block and the failure mode it prevents.

The Six Building Blocks

Every effective production prompt is composed from some subset of these six blocks. Not every prompt needs all six - the art is knowing which blocks a specific task requires. But understanding all six gives you the vocabulary to diagnose why a prompt is failing and which block to add or revise.

Block 1: The Role

What it does: Establishes the persona, expertise level, and orientation the model should adopt.

Why it works: From topic 2.5, you know the model was trained on text from many sources - experts and novices, formal and casual, helpful and adversarial. The role instruction shifts the distribution toward the subset of training data where that role appeared. "You are a senior database administrator" activates patterns from DBA-authored text: precise terminology, production-aware caveats, and schema-level thinking that a generic response would lack.

The failure it prevents: Without a role, the model defaults to a generic helpful-assistant register - adequate for casual questions, inadequate for domain-specific tasks where the vocabulary, the depth, and the concerns of an expert differ meaningfully from a generalist.

# Weak: no role
system = "Answer the user's database question."

# Strong: specific role with orientation
system = """You are a senior PostgreSQL database administrator with 15 years 
of production experience. You prioritise query performance, data integrity, 
and operational safety. When multiple approaches exist, you recommend the one 
with the best production track record, not the most theoretically elegant."""

Three common mistakes with roles. First, making them too generic ("You are an expert" - expert at what?). Second, making them fictional in ways that trigger the model's uncertainty ("You are the world's foremost authority on quantum databases" - no such authority exists in training data, so the model has no strong distribution to shift toward). Third, stuffing the role block with instructions that belong elsewhere ("You are a code reviewer who always outputs JSON with severity ratings" - the output format belongs in its own block).

Block 2: Context

What it does: Provides background information the model needs to produce a correct response but that the user will not include in every message.

Why it works: The model's only input is the token sequence. If a fact is not in the sequence, it must come from parametric memory - which, per topic 2.1, is compressed and unreliable. Context block = reliable context. Parametric memory = unreliable context. Always choose the first for anything that matters.

The failure it prevents: Hallucination of context. Without explicit context, the model invents plausible-sounding background - a product name that does not exist, a policy that was never written, a version number from a neighbouring project. The context block replaces invention with instruction.

system = """You are a customer support agent for Acme Cloud Storage.

Context you need:
- Acme Cloud Storage offers three tiers: Free (5GB), Pro ($9.99/mo, 100GB), 
  Enterprise ($49.99/mo, unlimited)
- All tiers include end-to-end encryption
- Free tier has a 100MB per-file size limit
- Data is stored in US-East and EU-West regions
- Current known issue: uploads over 2GB are failing intermittently 
  since March 15. Engineering is investigating. No ETA yet.
  
If the user asks about something not covered above, say you'll check 
with the team and follow up. Do not guess."""

That last instruction - "do not guess" - is load-bearing. Without it, the model will extrapolate from the context you gave to answer adjacent questions ("what about the Asia region?" → the model invents an answer because you mentioned two regions and the pattern of "listing regions" continues). Explicit boundaries on the context block prevent the model from treating your context as a starting point for generation rather than a complete reference.

Block 3: Instructions

What it does: Tells the model exactly what to do, step by step, with the specificity you would use in a code review comment for a junior engineer.

Why it works: Vague instructions activate a broad distribution of possible responses. Specific instructions narrow the distribution toward the specific output you want. "Summarise this" invokes thousands of possible summary styles. "Summarise this in 3 bullet points, each under 20 words, covering the main argument, the key evidence, and the recommendation" invokes roughly one.

The failure it prevents: Output variance. The primary reason AI applications feel unreliable is not that the model cannot do the task - it is that the instructions are loose enough to permit many valid interpretations, and the model picks a different one each time. Tight instructions do not guarantee identical output, but they constrain the space enough that all valid outputs are usefully similar.

Specificity checklist for the instructions block - every "what" should be answered:

Tip: The negation trap: "Do not include personal opinions" is a weaker instruction than "State only verifiable facts from the provided context." Positive instructions (what to do) anchor the model more reliably than negative instructions (what not to do), because the model processes the entire instruction - including the thing you told it not to do - and the banned concept is now activated in the distribution. Reframe negatives as positives wherever possible. This is not universal (some negatives are unavoidable), but as a default orientation it measurably improves compliance.

Block 4: Output Format

What it does: Specifies the exact structure, schema, or format of the response. This is topic 1.3's territory, elevated from "getting JSON out" to a general prompt-design principle.

Why it works: Format specification removes an entire category of variance. Without it, the model decides whether to use bullet points or prose, whether to include headers, whether to wrap JSON in markdown fences. Each decision is a coin flip in the distribution. Specifying format replaces all those coin flips with deterministic constraints.

# Format block for the code review assistant
format_block = """Respond in this exact JSON format:
{
  "summary": "one-sentence overall assessment",
  "issues": [
    {
      "severity": "critical" | "warning" | "info",
      "line": number or null,
      "description": "what the issue is and why it matters",
      "suggestion": "how to fix it"
    }
  ],
  "approval": "approve" | "request_changes" | "needs_discussion"
}"""

When the output format is JSON and the consumer is a program, combine this block with topic 1.3's Structured Outputs or Pydantic validation. The format block tells the model what shape to generate; the validation layer guarantees it arrived.

When the output is prose for a human, the format block still does work: "Respond in 2–3 short paragraphs. Start with the most important finding. End with a specific next step." constrains variance without imposing a schema.

Block 5: Examples (Few-Shot)

What it does: Shows the model concrete input-output pairs that demonstrate the desired behaviour.

Why it works: Examples are the most powerful single technique in prompt engineering, and the reason is architectural. From topic 2.3, the model attends over its entire context. An example is not just an instruction about what to do - it is the output itself, sitting in the context, with the exact format, tone, length, and level of detail you want. The model attends to it and pattern-completes accordingly. Showing is stronger than telling because the thing being shown is already in the right token space.

The failure it prevents: Ambiguity in instructions. No matter how specific your instructions are, a well-chosen example resolves ambiguities that language cannot. "Severity: critical means the code will fail in production" leaves room for interpretation. An example that shows a SQL injection classified as critical and a missing log statement classified as info demonstrates the calibration.

examples_block = """
Example 1:
Input: "SELECT * FROM users WHERE id = " + user_input
Output: {"severity": "critical", "line": 3, 
  "description": "String concatenation in SQL query creates injection vulnerability",
  "suggestion": "Use parameterised queries: cursor.execute('SELECT * FROM users WHERE id = %s', (user_input,))"}

Example 2:
Input: results = db.query("SELECT * FROM orders")
Output: {"severity": "warning", "line": 7,
  "description": "SELECT * retrieves all columns; production queries should select only needed columns",
  "suggestion": "Replace with specific columns: SELECT id, status, total FROM orders"}
"""

Three rules for examples. Diversity over quantity: two examples showing different categories of behaviour teach more than five examples of the same category. Edge cases earn their place: if your task has a common failure mode, one example showing the correct handling of that case is worth more than three examples of the easy path. Examples cost tokens: at 100–300 tokens per example, three examples add 300–900 tokens to every call. At production volume, that is a cost decision (topic 3.1's recurring-expense lens). Include the minimum number that demonstrates the required behaviour; add more only when evaluation data shows the model is still ambiguous on a specific case.

Block 6: Constraints and Guardrails

What it does: Defines boundaries - what the model must not do, what it should do when uncertain, and how it should handle edge cases.

Why it works: The model is trained to be helpful, which means it will always try to produce something. Without explicit constraints, "something" might include fabricated answers to questions outside its knowledge, responses that violate domain-specific rules, or outputs that are technically correct but contextually inappropriate. Constraints define the walls the model should not cross.

The failure it prevents: Boundary violations. The customer support agent that answers questions about competitors. The medical triage system that provides diagnoses instead of routing to a doctor. The code reviewer that rewrites the code instead of reviewing it.

constraints_block = """Constraints:
- If the code is in a language you cannot confidently review, say so 
  rather than guessing.
- Never suggest changes that would alter the code's external behaviour 
  without flagging them as "behavioural change."
- If the code has no issues, return an empty issues array and 
  approval: "approve". Do not invent issues.
- Maximum 10 issues per review. If more exist, note "additional issues 
  exist" in the summary and list the 10 most severe."""

The "do not invent issues" constraint deserves emphasis. Without it, I have measured a consistent pattern where the model produces 1–3 spurious findings per review - plausible-sounding issues that are not actually bugs. The model's training includes abundant examples of code reviews finding things, so "finding nothing" is a low-probability output. The explicit constraint shifts the distribution toward the less-probable-but-correct "no issues found" when that is the right answer.

Assembling the Blocks

The blocks go in a specific order, and the order matters - because of the attention patterns from topic 2.3.

Position 1 (strongest attention): Role + Context. The model processes system prompts first and attends to them throughout. Who you are and what you know go here.

Position 2 (strong attention): Instructions + Format. What to do and how to structure the output. These are the core operational directives.

Position 3 (moderate attention): Examples. Concrete demonstrations of desired behaviour. Placed after instructions so the model has the frame before seeing the instances.

Position 4 (end of system prompt, strong attention): Constraints. Boundaries and edge-case handling. Placed last because they are the "but do not do X" corrections on the instructions, and recency in the context gives them emphasis.

This ordering also maximises prompt caching (topic 2.3): the role, context, instructions, and format are stable across requests. Examples are mostly stable but may rotate. Constraints are fully stable. The user message - the only variable part - comes after, preserving the cached prefix.

graph TD
    A[Block 1: Role<br/>Who is the model?] --> B[Block 2: Context<br/>What does it need to know?]
    B --> C[Block 3: Instructions<br/>What should it do?]
    C --> D[Block 4: Output Format<br/>What shape is the response?]
    D --> E[Block 5: Examples<br/>Show, do not just tell]
    E --> F[Block 6: Constraints<br/>What must it NOT do?]
    F --> G[User Message<br/>The variable input]
    A -.->|"strongest attention<br/>+ cache-friendly prefix"| F

This diagram shows the assembly order of prompt building blocks. Block 1, the Role, is at the top, followed by Context, Instructions, Output Format, Examples, and Constraints. The user message comes last as the variable input. A dotted line from Role to Constraints indicates that this ordering creates the strongest attention positions for the most important blocks and produces a cache-friendly stable prefix.

Interview: The interview pattern here is: "Walk me through how you would design the prompt for [specific task]." Interviewers are not looking for the perfect prompt - they are looking for a structured approach. Candidates who say "I'd tell it to do X" get a different score than candidates who say "I'd start with the role and context, then specific instructions with inclusion and exclusion criteria, then the output schema, then 2–3 diverse examples, and then constraints for edge cases - and I'd test against an eval set before shipping." The structure is the signal.

Exercise: Rebuild a Prompt From Blocks

Exercise: Take your best-performing Module 1 prompt and rebuild it using the six-block framework:

1. Decompose your existing prompt into whichever blocks it currently uses. Most Module 1 prompts have 2–3 blocks implicitly; identify which are missing.
2. Add the missing blocks. Write a role, context, explicit instructions, format specification, at least one example, and constraints.
3. Run both versions (old and new) against your evaluation test cases from Module 1. Measure accuracy, consistency (how similar are 5 runs?), and token count.
4. Challenge: deliberately remove one block at a time and re-run your eval. Which block's absence hurts quality most? That block is doing the most work in your prompt - protect it, test it, and never remove it without evidence.

What You Should Take From This Lesson

An effective prompt is assembled from six building blocks - role, context, instructions, output format, examples, and constraints - each preventing a specific failure mode: generic voice, hallucinated background, output variance, format instability, instruction ambiguity, and boundary violations. The blocks assemble in an order that respects attention strength and prompt caching: stable identity and directives first, variable user input last. Not every prompt needs all six - the skill is knowing which blocks a specific task requires - but understanding all six gives you the vocabulary to diagnose failures and the structure to fix them systematically.

The next topic takes individual techniques - chain-of-thought, self-consistency, role stacking, and others - and shows you when each one transforms a good prompt into a significantly better one, and when it is wasted tokens.

Back to AI Engineering