Free lesson · AI Engineering

What Is Prompt Engineering?

Easy11 min readaillmrag

What Is Prompt Engineering?

The Discipline That Sounds Like a Joke and Pays Like a Skill

When I first heard the term "prompt engineering" in 2023, my reaction was the one most engineers had: that is not a real thing. Engineering is building systems, writing code, managing infrastructure. Typing instructions at a chatbot is not engineering. It is asking nicely.

I was wrong, and the correction cost me two weeks.

My team was building an internal tool that classified support tickets into priority categories - P0 through P3 - based on the ticket description. The model was GPT-4, the prompt was a paragraph I had written in about three minutes ("You are a support ticket classifier. Read the ticket and assign a priority from P0 to P3"), and the accuracy was 71%. Good enough for a prototype, not good enough to ship.

My engineering instinct said: get more training data, fine-tune, deploy a custom model. That was the "real" engineering solution. A colleague - someone who had spent time in prompt engineering before it had a name - asked to try something first. She spent two days rewriting the prompt. She did not change the model, the data, or the architecture. She changed the text.

The accuracy went to 89%.

Two days of prompt work accomplished what I had estimated would take four weeks of fine-tuning pipeline development. The "not real engineering" had out-engineered my engineering plan by 6x on time-to-result and achieved a higher accuracy than my fine-tuning estimate.

That was the day I stopped thinking of prompt engineering as "typing instructions" and started thinking of it as what it actually is: programming in natural language, with the same demands for precision, testing, and iteration as any other kind of programming - and with its own set of failure modes, debugging techniques, and best practices that take time to learn.

What Prompt Engineering Actually Is

Prompt engineering is the practice of designing, testing, and iterating the text inputs to a language model to reliably produce desired outputs. The word "reliably" is doing most of the work in that sentence. Anyone can write a prompt that works once. The discipline is writing prompts that work at production scale - across thousands of inputs, edge cases, adversarial inputs, and the natural drift of model behaviour over time.

From Module 2, you know why this discipline exists. The model is a function from token sequence to probability distribution. Your prompt is the input to that function. Change the prompt and you change the distribution. Every word, every instruction, every example you include reshapes the probability landscape the model samples from. Prompt engineering is the practice of shaping that landscape intentionally rather than accidentally.

Three properties make this harder than it sounds:

Natural language is ambiguous. In code, sort(array, reverse=True) has exactly one interpretation. In a prompt, "list the items in order of importance" has many: whose importance? By what criteria? Ascending or descending? Including or excluding ties? The model will resolve every ambiguity somehow - by picking the most probable interpretation given its training data. If that interpretation differs from yours, the output looks wrong. It is not wrong. Your instruction was ambiguous, and the model resolved the ambiguity differently than you would have.

The feedback loop is probabilistic. In code, a bug manifests as a wrong output for a specific input, and you can trace it deterministically. In prompt engineering, a "bug" manifests as a distribution shift - the output is wrong 8% of the time instead of 3%, or it is wrong in a new way on a new class of inputs. Debugging requires statistical thinking: run the prompt on a test set, measure accuracy across categories, identify the failure patterns, hypothesise about what in the prompt caused them, revise, re-measure.

Small changes have large effects. Moving one instruction from the middle of a prompt to the beginning can change accuracy by 10%. Adding "Think step by step" before a reasoning task can change accuracy by 15–20%. Rephrasing "do not include" as "exclude" can change behaviour because the tokens are different and the associated probability landscapes differ. This sensitivity is what makes prompt engineering a discipline rather than a one-shot activity: the space of possible phrasings is enormous, and the performance surface over that space is rugged.

Note: Prompt engineering is the cheapest, fastest lever for improving LLM output quality. It requires no infrastructure, no training data, no deployment changes, and no additional cost per call (the prompt tokens are a rounding error compared to the quality gain). When output quality is not good enough, prompt engineering is always the first thing to try, and it is often the last thing needed.

The Three Levels of Prompt Engineering

I organise prompt engineering into three levels, because the techniques, the testing requirements, and the payoff profiles are different at each level.

Level 1: Instructional Prompting

The most basic level: telling the model what to do in clear, specific natural language. This is where most people stop, and where most quality gains live.

The ticket classifier that went from 71% to 89% - the improvement was almost entirely Level 1 work. My original prompt was vague: "assign a priority." The revised prompt specified what each priority level meant, gave examples of each, specified the output format, and listed edge cases with their correct classifications. It did not use any advanced technique. It was just clear.

You have already done Level 1 work. Your topic 1.1 system prompt, your topic 1.3 extraction schema, your topic 1.5 summarisation prompt for conversation compaction - these were all instructional prompting. This module's job is to make you deliberately good at what you have been doing intuitively, by giving you a structural framework for prompt design.

Level 2: Technique-Based Prompting

The intermediate level: applying specific, named techniques that reliably shift model behaviour in known directions. Chain-of-thought, few-shot examples, self-consistency, role assignment, output formatting constraints - these are patterns with documented effects and known failure modes.

Level 2 is where prompt engineering starts to feel like a craft. Each technique has a cost (in tokens, in latency, sometimes in unexpected side effects) and a benefit (in accuracy, in consistency, in handling of edge cases). Knowing which technique to reach for when, and when not to reach for any, is the skill. Topic 4.3 covers these in depth.

Level 3: Systematic Prompt Optimization

The advanced level: treating the prompt as a parameter to be optimised against a metric, using evaluation datasets, automated search, and sometimes tools like DSPy that optimise prompts programmatically. This level treats prompt engineering as machine learning over the prompt space - searching for the input text that maximises accuracy on your evaluation set.

Level 3 is the least common in production today but is growing fast, and it is where the field is heading. Topic 4.7 covers it. For most applications, Levels 1 and 2 are sufficient.

Prompt Engineering as Programming

Let me push the programming analogy further, because it is the mental model that makes prompt engineering stick for engineers.

The analogy is not just pedagogical - it is operational. The teams I have seen succeed at prompt engineering are the ones that treat prompts with the same discipline they apply to code: they version them, they test them against evaluation sets before deploying changes, they review them, and they write post-mortems when a prompt change causes a quality regression.

The teams that fail treat prompts as disposable text - written once, never tested, never versioned, changed ad hoc by whoever is debugging the feature that day. Six months later, nobody knows why the prompt says what it says, what the alternatives were, or what quality threshold it was tested against. That is not a prompt engineering failure. It is a software engineering failure applied to prompts.

Important: Every principle from topic 1.3 (Structured Output) was prompt engineering. The three-level enforcement architecture - schema-shaped prompting, provider-enforced structure, application-side validation - was a prompt engineering strategy, dressed in Python. Topic 1.5's summarisation prompt was prompt engineering. Your conversation memory's system prompt was prompt engineering. This module does not introduce a new activity. It names, structures, and systematises what you have been doing.

Prompt Engineering in the Toolkit: Sharper Boundaries

Let me make the comparison with fine-tuning more concrete, because the boundary between "prompt-engineer it" and "fine-tune it" is the single most expensive wrong call in AI Engineering, and this module is the place to learn where the line sits.

Prompt engineering changes what you ask. Every call sends your instructions fresh; the model's weights are untouched. You can iterate in minutes, A/B test in hours, and roll back in seconds. The cost is per-call token overhead.

Fine-tuning changes what the model is. The weights themselves shift, permanently (until you retrain). Iteration takes hours to days per cycle. Rollback means redeploying the previous model version. The cost is training compute plus the operational overhead of model management.

The practical test I use: can the desired behaviour be demonstrated in 5 examples? If yes, few-shot examples in a prompt are almost certainly sufficient and are faster to iterate. If the behaviour requires hundreds of examples to learn - a highly specific output format, a nuanced domain-specific tone, a classification boundary that resists verbal description - fine-tuning is the tool.

At Meta, we used fine-tuning for exactly two categories: specialised output formats that no amount of prompting could make consistent (a particular internal ticketing schema with 40+ fields), and domain-specific classification tasks where the label definitions were too subtle for natural-language instruction (distinguishing between 12 types of infrastructure alerts where the distinction was in specific log-message patterns). Everything else - and I mean everything - was prompt engineering. The ratio was roughly 95% prompt engineering, 5% fine-tuning, by count of production features.

That ratio is not universal, but it is representative. Most engineers reach for fine-tuning too early. This module exists so you exhaust prompt engineering's capacity first.

What Prompt Engineering Cannot Do

Intellectual honesty requires drawing the boundary. Prompt engineering is powerful, cheap, and fast - but it has limits, and knowing those limits prevents you from spending weeks on a prompt when the problem is somewhere else.

Prompt engineering cannot add knowledge the model does not have. If the answer requires information from after the training cutoff, from your proprietary database, or from a niche domain the model was not trained on, no prompt will produce it reliably. That is retrieval's job (Module 6) or fine-tuning's job (Module 9).

Prompt engineering cannot guarantee correct output. It shifts the probability distribution toward desired outputs. A well-engineered prompt might achieve 95% accuracy. The remaining 5% requires validation, fallback paths, and human review - the engineering you built in topic 1.3.

Prompt engineering cannot overcome model limitations. If the model cannot do multi-step arithmetic reliably (topic 2.1), no prompt makes it reliable - the limitation is architectural, not instructional. Use tools (Module 7).

Prompt engineering has diminishing returns. The first hour of prompt work on a new task typically produces the largest quality gain. The twentieth hour produces a marginal improvement that fine-tuning, better retrieval, or a larger model could have delivered in the first hour. Knowing when to stop prompting and switch to a different lever is as important as prompting skill itself.

The decision tree for "how do I improve my AI application's quality?" is:

1. First: prompt engineering (cheap, fast, no infrastructure)
2. Second: retrieval / RAG (adds knowledge without retraining)
3. Third: fine-tuning (changes behaviour, not knowledge)
4. Fourth: larger model (more capability, more cost)

Try them in order. Most applications never need to go past step 2.

The Cost of Prompts

One more dimension, because this module just taught you to see costs. Prompts have a token cost - your system prompt goes into every single API call as input tokens. A 200-token system prompt costs you 200 input tokens per call. A 2,000-token system prompt costs you 2,000. At 100,000 calls per day on GPT-4o ($2.50/M input tokens), the difference between a 200-token and a 2,000-token system prompt is:

• 200 tokens: 100K × 200 = 20M tokens/day → $50/day →
  ,500/month
• 2,000 tokens: 100K × 2,000 = 200M tokens/day → $500/day →
  5,000/month

The 1,800 extra tokens in your system prompt cost

Prompt caching (topic 2.3) softens this - your system prompt is typically the stable prefix that qualifies for cached rates. But even at 50% discount, a bloated system prompt is still paying for tokens that might be doing no work.

Interview: The interview tells on this topic are binary. Either the candidate treats prompt engineering as a disciplined practice - versioned, tested, iterated against evaluation data - or they treat it as "I tried a few things and picked the one that seemed best." The first candidate ships reliable AI features. The second ships features that break on edge cases nobody tested for. When an interviewer asks "How do you develop and maintain your prompts?", they are sorting you into one of these two bins.

Exercise: Audit What You Have Already Written

Exercise: Before learning new techniques, audit the prompts you have already built:

1. Collect every system prompt and instruction you wrote across Module 1 (extraction, conversation, summarisation). Print each one with its token count.
2. For each prompt, identify one ambiguity - a place where a different reader could interpret the instruction differently from you. Rewrite it to eliminate the ambiguity. Measure whether the rewrite changes the output on your test cases.
3. For each prompt, identify one instruction that could be removed without changing quality. Remove it and test. If quality holds, you just saved tokens - at production volume, real dollars.
4. Challenge: version your prompts. Create a prompts/ directory, save each system prompt as a named text file with a version number, and write a one-line changelog for each change you make from this point forward. This is the smallest possible prompt-management system, and it is what separates teams that improve over time from teams that drift.

What You Should Take From This Lesson

Prompt engineering is programming in natural language - with the same demands for precision, testing, and version control as any other programming. It operates at three levels: instructional clarity, named techniques, and systematic optimisation. It is always the first lever to try for quality improvement: cheaper than retrieval, faster than fine-tuning, simpler than model upgrades. Its limits are knowledge (retrieval's domain), guarantees (validation's domain), and architectural capability (tools' domain). Prompts have a recurring token cost at production volume, so every token must earn its place. And the discipline distinction is operational: teams that version, test, and review their prompts ship reliable features; teams that do not, ship fragile ones.

The next topic builds the structural framework: the anatomy of an effective prompt, section by section, with the design reasoning for each part and the failure mode each part prevents.

Back to AI Engineering