Free lesson · AI Engineering

What AI Engineering Actually Is

Easy11 min readaillmrag

What AI Engineering Actually Is

The Role That Did Not Exist Three Years Ago

Here is a job posting I saw on Meta's internal board in early 2024: "AI Engineer - Build production applications using foundation models. Experience with LLM APIs, RAG architectures, and prompt engineering required. ML research background not required."

That last line is the one that matters. ML research background not required. Three years earlier, every AI-related role at Meta - every single one - required a machine learning background. You needed to understand backpropagation, gradient descent, loss functions, and model training. If you could not explain the difference between L1 and L2 regularisation in an interview, you were not getting hired for anything with "AI" in the title.

That changed, and it changed fast. The emergence of powerful foundation models - GPT-4, Claude, Gemini, Llama, Mistral - created a new kind of work. Suddenly, building AI-powered applications did not require building AI models. It required knowing how to use them well. How to integrate them into production systems. How to handle their failure modes. How to manage their costs. How to evaluate their outputs.

That work needed a name. The industry settled on AI Engineering.

Note: AI Engineering is the discipline of building production applications that use AI models - primarily large language models - as a core capability. An AI Engineer does not typically train models from scratch. They select, integrate, orchestrate, and operate models that others have built, combining them with traditional software engineering to create applications that are reliable, scalable, and cost-effective.

If that sounds like "what a backend engineer does, but with LLMs instead of databases" - you are closer than you think.

What AI Engineering Is Not

The boundaries of AI Engineering become clearest when you see what sits on either side. Three adjacent roles share territory with AI Engineering but are fundamentally different disciplines.

AI Engineering vs Machine Learning Engineering

A Machine Learning Engineer builds and trains models. They work with training data, model architectures, hyperparameter tuning, and training infrastructure. When Meta trains a new version of Llama, that is ML Engineering work. When someone takes the trained Llama model and builds a document search system with it, that is AI Engineering work.

The analogy I use is this: a database engine developer and a backend engineer both work with data. But a database engine developer writes the storage engine, the query planner, and the replication protocol. A backend engineer uses the database to build an application. They need to understand how the database works - well enough to write good queries, design good schemas, and debug performance issues - but they do not build the database itself.

AI Engineering has the same relationship to ML Engineering. You need to understand how LLMs work. You need to know enough about transformer architecture, tokenisation, and generation to make informed decisions. But you are not training the model. You are building with it.

AI Engineering vs Data Science

Data Science is about extracting insights from data. A Data Scientist builds dashboards, runs analyses, trains predictive models, and communicates findings to stakeholders. Their primary output is understanding - "here is what the data tells us."

An AI Engineer's primary output is a working application. The overlap is narrow: both use Python, both work with data, both need statistical intuition. But a Data Scientist's week looks like Jupyter notebooks, visualisation libraries, and stakeholder presentations. An AI Engineer's week looks like API integrations, deployment pipelines, and production monitoring.

If you come from a Data Science background, AI Engineering will feel familiar in its tooling but different in its concerns. You will stop asking "what does the data show?" and start asking "how do I build a system that reliably uses this model to serve users at scale?"

AI Engineering vs Traditional Software Engineering

This is the boundary that matters most, because most AI Engineers come from traditional software engineering backgrounds. Including me.

Traditional software engineering is deterministic. You write a function, it produces the same output for the same input every time. Your tests can assert exact equality. Your deployment is considered successful when the new version behaves identically to the tested version.

AI Engineering is probabilistic. You send the same prompt to an LLM twice and you may get different responses. Your tests cannot assert exact equality - they have to assess quality on a spectrum. Your deployment is considered successful when the new version is probably at least as good as the old version, based on evaluation metrics you have chosen to track.

That single difference - deterministic versus probabilistic - ripples through everything. Testing, deployment, monitoring, incident response, user experience design, error handling. Every practice you know from traditional software engineering needs to be adapted for a world where your core component does not give the same answer twice.

Warning: The most common mistake experienced software engineers make when entering AI Engineering is treating LLM outputs like database query results - deterministic, reliable, and cacheable. They are none of those things by default. The entire production stack above the LLM exists to manage that non-determinism.

The Three Layers of the AI Stack

Every AI application has three layers. Understanding these layers is the single most useful mental model for the entire course, because every concept, every technique, and every architectural decision maps to one of them.

Layer 1 - The Model Layer

This is where the foundation models live. GPT-4, Claude, Gemini, Llama, Mistral - the large language models themselves. This layer includes the model weights, the inference infrastructure that runs them, and the APIs that expose them.

Most AI Engineers do not work at this layer directly. You consume it through APIs. But you need to understand it well enough to make informed decisions: which model to choose, what each model is good at, how generation parameters affect output, why some prompts work and others do not.

The model layer is also where fine-tuning lives. When you fine-tune a model, you are modifying this layer - adapting the model's behaviour for your specific use case. Fine-tuning is the deepest intervention an AI Engineer typically makes at the model layer.

graph TD
    A[Layer 1: Model Layer] --> B[Foundation Models]
    A --> C[Model APIs]
    A --> D[Fine-Tuning]
    B --> B1[GPT-4, Claude, Gemini]
    B --> B2[Llama, Mistral, Qwen]
    C --> C1[OpenAI API]
    C --> C2[Anthropic API]
    C --> C3[Self-Hosted Inference]
    D --> D1[LoRA / QLoRA]
    D --> D2[Supervised Fine-Tuning]

This diagram shows the Model Layer broken into three branches: Foundation Models (including closed models like GPT-4, Claude, Gemini and open models like Llama, Mistral, Qwen), Model APIs (OpenAI, Anthropic, and self-hosted inference), and Fine-Tuning (LoRA/QLoRA and supervised fine-tuning).

Layer 2 - The Orchestration Layer

This is where most AI Engineering work happens. The orchestration layer sits between the model and the application. It manages how the model is used - what context it receives, what tools it can call, how its outputs are validated, and how multiple model calls are chained together.

Prompt engineering lives here. RAG lives here. Tool use and function calling live here. Agent loops live here. Guardrails and content filtering live here. Evaluation pipelines live here.

If the model layer is the engine, the orchestration layer is the transmission, the steering, the brakes, and the navigation system. It is the layer that turns a raw model into a useful, reliable, safe application component.

This is the layer where AI Engineers spend 70-80% of their time. It is also the layer where the most impactful architectural decisions are made. A bad model choice can be fixed by swapping models. A bad orchestration architecture requires a rewrite.

Layer 3 - The Application Layer

This is the user-facing application itself. The chat interface, the search bar, the document processor, the code assistant - whatever the user interacts with. This layer handles user input, presents model outputs, manages sessions, handles authentication, and provides the experience that makes the AI useful.

The application layer is where AI Engineering meets traditional software engineering most directly. You are building APIs, rendering UIs, managing state, and deploying services. The difference is that one of your backend components is non-deterministic, variable-latency, and expensive per call.

graph TD
    subgraph "Layer 3: Application"
        APP[User Interface] --> API[API Layer]
        API --> AUTH[Auth & Sessions]
    end
    subgraph "Layer 2: Orchestration"
        ORCH[Prompt Management] --> RAG[Retrieval / RAG]
        ORCH --> TOOLS[Tool Use / Agents]
        ORCH --> GUARD[Guardrails]
        ORCH --> EVAL[Evaluation]
    end
    subgraph "Layer 1: Model"
        MODEL[Foundation Models]
        FT[Fine-Tuned Models]
    end
    API --> ORCH
    RAG --> MODEL
    TOOLS --> MODEL
    GUARD --> MODEL
    FT --> MODEL

This diagram shows the three-layer AI stack. At the top, the Application layer contains the user interface, API layer, and auth. In the middle, the Orchestration layer contains prompt management, RAG retrieval, tool use and agents, guardrails, and evaluation. At the bottom, the Model layer contains foundation models and fine-tuned models. Arrows show the flow from user interface down through orchestration to models.

Important: The three-layer model is your navigation system for this course. Every module maps to one or more layers. When you learn prompt engineering, you are working at Layer 2. When you learn about transformer architecture, you are understanding Layer 1. When you learn about building AI-powered UX, you are working at Layer 3. Keeping track of which layer you are working at helps you understand how concepts relate to each other.

Where AI Engineering Lives in an Organisation

At Meta, AI Engineers sit within product teams, not within the AI research organisation. This is a critical distinction. AI researchers advance the state of the art. AI Engineers apply the state of the art to products. The two groups collaborate, but they have different goals, different metrics, and different workflows.

In most companies I have worked with or advised, AI Engineering is emerging as a specialisation within software engineering, not as a separate function. Here is how it typically maps:

In a startup (fewer than 50 engineers): AI Engineering is not a separate role. It is a set of skills that some backend engineers develop. The same person who builds the API also integrates the LLM, writes the prompts, and deploys the RAG pipeline. This is actually the best training ground, because you see the entire stack.

In a mid-stage company (50-500 engineers): AI Engineering becomes a recognisable specialisation. You might have an "AI team" or "ML platform team" that includes 3-10 AI Engineers alongside ML Engineers. The AI Engineers own the orchestration layer and the integration between models and products. The ML Engineers own model training and evaluation.

In a large tech company (500+ engineers): AI Engineering is a formal role with its own career ladder, interview process, and performance criteria. At Meta, we have AI Engineers at every level from E3 to E7. The interview loop includes system design questions specific to AI applications, LLM evaluation problems, and architecture trade-off discussions.

Interview: What interviewers look for in AI Engineering candidates:

1. Can you design an AI system architecture and explain your trade-offs?
2. Do you understand the cost, latency, and reliability implications of your design choices?
3. Can you evaluate whether an AI system is working well, and what "well" means for the specific use case?
4. Do you know when NOT to use AI - when a traditional approach would be simpler, cheaper, or more reliable?

These are exactly the skills this course builds.

The Skills You Need (and the Ones You Do Not)

Let me be blunt about what you need to know before starting this course, what this course will teach you, and what this course deliberately does not cover.

What You Need Before Starting

Python at an intermediate level. You can write classes, use decorators, work with async/await, and use type hints. If you need a refresher, the appendix module covers the Python patterns AI Engineers use most. You do not need to be a Python expert, but you cannot be a Python beginner.

Basic software engineering fundamentals. You understand APIs, HTTP, JSON, databases, version control, and deployment at a conceptual level. You have built and deployed at least one non-trivial application. You do not need experience at scale, but you need to understand the basic mechanics of shipping software.

No machine learning background required. I will teach you everything you need to know about how LLMs work. You do not need to know what a gradient is, how backpropagation works, or what a loss function does. If you do have ML background, the relevant module will feel like a review, and that is fine.

What This Course Teaches

AI application architecture. How to design systems that use LLMs as a core component. Patterns, trade-offs, and failure modes.

The full AI Engineering toolkit. Prompt engineering, RAG, embeddings, function calling, MCP, fine-tuning, agents. Not just how each works, but when to use each and how to combine them.

Production operations. Deployment, monitoring, evaluation, cost management, safety, incident response. The operational discipline that separates demos from products.

Engineering judgement for AI. When to use AI. When not to. When to use a big model versus a small one. When to fine-tune versus prompt-engineer. When to build versus buy. These judgement calls are the hardest thing to teach and the most valuable thing to learn.

What This Course Does Not Cover

Training models from scratch. If you want to train a custom LLM, you need an ML Engineering course, not this one.

ML research. Attention mechanisms, novel architectures, and advancing the state of the art are not what this course is about.

Data Science workflows. Exploratory data analysis, statistical modelling, and insight communication are adjacent but different.

Specific framework tutorials. I will reference LangChain, LlamaIndex, and other frameworks where appropriate, but this is not a "build with LangChain" course. Frameworks change every six months. Principles do not. I teach principles.

A Field That Is Still Being Defined

I want to be transparent about something: AI Engineering as a discipline is roughly two years old. The term gained mainstream usage after Swyx's "Rise of the AI Engineer" essay in mid-2023. The first dedicated AI Engineering conference happened in late 2023. The first formal job postings with "AI Engineer" in the title appeared at scale in early 2024.

That means two things.

First, some of what I teach in this course will be obsolete within 18 months. Specific tools will be replaced. Specific techniques will be superseded. Specific best practices will be revised. I will flag these areas clearly when we encounter them, and I will teach you the principles underneath the specific techniques so that you can adapt when things change.

Second, there is genuine disagreement in the industry about what AI Engineering includes. Some people define it narrowly - LLM API integration and prompt engineering. Others define it broadly - everything from model fine-tuning to MLOps to AI product management. My definition sits in the middle: AI Engineering is building production applications using AI models, which requires understanding the models, the orchestration patterns, the production infrastructure, and enough product thinking to make good decisions about when and how to use AI.

This course covers that full scope. Some lessons will feel very close to traditional software engineering. Others will feel very close to ML Engineering. The AI Engineer's job is to live at the intersection, fluent in both, belonging exclusively to neither.

Tip: The field is moving fast, but the fundamentals are stabilising. The concepts of prompt engineering, RAG, embeddings, agents, and evaluation are not going away. The specific tools and implementations will change - but the architectural patterns and the engineering judgement you build in this course will remain relevant for years. That is what I have optimised for: durable knowledge over trendy tools.

What Comes Next

You now know what AI Engineering is, where it sits relative to adjacent disciplines, and what the three-layer AI stack looks like. In the next topic, we will map the current AI landscape - the models, the providers, the tools, and the infrastructure ecosystem you will be working with throughout this course.

After that, we will set up your development environment and I will walk you through the full learning roadmap for the course.

Then we start building.

Back to AI Engineering