Free lesson · AI Engineering

What Are AI Agents?

Medium11 min readaillmrag

What Are AI Agents?

The Assistant That Started Thinking for Itself

For nine modules, you have been building systems where the LLM responds to a request: the user asks, the system retrieves, the model answers. The model is reactive - it waits for input and produces output. It does not decide what to do next. It does not formulate its own questions. It does not take a goal and break it into steps. It is a very capable employee who does exactly what you tell it to do and nothing more.

Then, in late 2024, I watched a demo at Meta that changed how I thought about what we were building. An engineer gave a system a single instruction: "Investigate why the checkout flow's conversion rate dropped 8% this week and recommend a fix." The system did the following without further human input: queried the analytics dashboard for the conversion funnel breakdown, identified that the drop concentrated on the payment step, searched error logs for payment-related exceptions, found a spike in 503 errors from the payment gateway, checked the deployment history and identified a gateway SDK upgrade deployed Monday, retrieved the SDK changelog, identified a breaking change in the timeout configuration, and produced a report: "The payment gateway SDK update on Monday changed the default timeout from 30s to 5s, causing 503 errors on slow-network transactions. Recommendation: revert the timeout to 30s and add an explicit timeout configuration to prevent future defaults from breaking the flow."

Nobody told the system to check the deployment history. Nobody told it to read the changelog. It decided those were necessary steps based on what it found at each stage. Each observation led to a new question, each question led to a tool call, and each tool result refined the system's understanding until it had enough to answer.

That is an AI agent. Not a chatbot that answers questions. Not a pipeline that follows a fixed sequence. A system that takes a goal, reasons about how to achieve it, takes actions, observes results, and adapts its plan based on what it learns.

The Agent Loop: Perceive → Reason → Act → Observe

Every AI agent, from the simplest to the most complex, follows the same core loop:

graph TD
    A[Goal<br/>from user or system] --> B[PERCEIVE<br/>Observe current state:<br/>context, tool results,<br/>conversation history]
    B --> C[REASON<br/>Decide what to do next:<br/>which tool to call,<br/>what question to ask,<br/>or whether to stop]
    C --> D[ACT<br/>Execute the decision:<br/>call a tool, generate text,<br/>ask the user a question]
    D --> E[OBSERVE<br/>Process the result:<br/>what changed?<br/>what did I learn?]
    E --> F{Goal<br/>achieved?}
    F -->|No| B
    F -->|Yes| G[Generate final<br/>response to user]

This diagram shows the core agent loop. Starting from a goal provided by the user or system, the agent perceives its current state by examining context, tool results, and conversation history. It then reasons about what to do next - which tool to call, what question to ask, or whether to stop. It acts by executing the decision. It observes the result, processing what changed and what it learned. If the goal is not yet achieved, the loop returns to perception. Once the goal is achieved, the agent generates a final response.

The loop is the same structure as topic 6.8's agentic RAG and topic 7.3's multi-tool orchestration - because those were agents, just scoped to specific tasks. What changes in this module is the generality: the agent's reasoning is not limited to "should I retrieve more documents?" or "which tool next?" It is open-ended: "given everything I know and everything I can do, what is the best next step toward achieving this goal?"

What Makes an Agent Different From a Pipeline

The distinction matters architecturally, because pipelines and agents have different failure modes, different testing strategies, and different cost profiles.

A pipeline follows a fixed sequence of steps defined by the developer. Step 1 always happens before Step 2. The order is deterministic. The developer decided the flow; the LLM executes within each step. Topic 7.4's structured workflows are pipelines - the state machine controls the flow.

An agent determines its own sequence of steps at runtime. The order depends on the goal, the available tools, and the results of previous steps. The LLM decides the flow; the developer provides the tools, the constraints, and the stopping conditions.

Important: Most production AI applications should be pipelines, not agents. The checkout investigation demo was impressive. It was also expensive (11 LLM calls, 7 tool calls), unpredictable (a different investigation might have taken 5 steps or 20), and difficult to test (how do you write a test for "investigate this problem"?). Agents are the right tool for tasks where the path cannot be predicted in advance. Pipelines are the right tool for everything else. This module teaches you to build agents; it also teaches you when not to build them.

The Agent Architecture

Under the demo's intelligence is a specific technical architecture that you can build with the components you already have.

graph TD
    subgraph "Agent Core"
        LLM[LLM<br/>The reasoning engine<br/>Modules 1-4]
        SP[System Prompt<br/>Goal, constraints,<br/>available tools,<br/>stopping criteria]
    end
    
    subgraph "Agent Capabilities"
        TOOLS[Tools<br/>Function calling +<br/>MCP servers<br/>Modules 7-8]
        MEMORY[Memory<br/>Context window +<br/>conversation history +<br/>vector store<br/>Topics 1.5, 5.1-5.5]
        RETRIEVAL[Retrieval<br/>RAG pipeline<br/>Module 6]
    end
    
    subgraph "Agent Infrastructure"
        LOOP[Agent Loop<br/>Orchestration +<br/>iteration limits<br/>Topic 7.3]
        EVAL[Evaluation<br/>Task completion metrics<br/>Topic 10.9]
        SAFETY[Safety Boundary<br/>Execution validation +<br/>human-in-the-loop<br/>Topics 7.1, 10.6]
    end
    
    SP --> LLM
    LLM --> TOOLS
    LLM --> MEMORY
    LLM --> RETRIEVAL
    TOOLS --> LOOP
    MEMORY --> LOOP
    RETRIEVAL --> LOOP
    LOOP --> EVAL
    LOOP --> SAFETY

This diagram shows the complete agent architecture. The agent core consists of the LLM as the reasoning engine and a system prompt defining the goal, constraints, tools, and stopping criteria. The agent's capabilities include tools from function calling and MCP, memory from the context window and vector stores, and retrieval from the RAG pipeline. The agent infrastructure includes the agent loop with orchestration and iteration limits, evaluation for task completion metrics, and the safety boundary with execution validation and human-in-the-loop patterns. Each component maps to a specific module earlier in the course.

Every component maps to something you have already built. The LLM is Module 1. The system prompt is Module 4. The tools are Modules 7–8. The memory is topic 1.5 plus Module 5. The retrieval is Module 6. The orchestration loop is topic 7.3. The execution boundary is topic 7.1. The evaluation is topic 6.9's pattern applied to agent tasks.

An AI agent is not a new technology - it is a composition of technologies you already know. The new part is the reasoning layer - the system prompt and loop structure that lets the LLM decide what to do with these components, rather than the developer deciding. That reasoning layer is what this module builds.

The Spectrum of Agent Autonomy

Not all agents are equally autonomous. The spectrum runs from tightly constrained (almost a pipeline) to fully autonomous (the checkout investigation demo), and the right level of autonomy depends on the task's risk, predictability, and value.

Level 1: Structured agent (low autonomy). The agent follows a predefined set of steps but has flexibility within each step. "First search the knowledge base, then check the metrics dashboard, then summarise findings." The steps are fixed; the search queries, the metrics to check, and the summary are LLM-determined. This is topic 7.4's structured workflow with more flexibility per step.

Level 2: Guided agent (moderate autonomy). The agent has a goal and a set of available tools, with heuristic guidance about tool ordering. "Investigate the performance issue using the monitoring tools and log search tools. Check at least two data sources before concluding." The agent decides which tools to use and in what order, but the guidance constrains the search space.

Level 3: Autonomous agent (high autonomy). The agent has a goal and a set of tools, with minimal constraints beyond safety guardrails. "Investigate why conversion dropped and recommend a fix." The agent determines the entire investigation strategy. This is the most powerful level and the most expensive, unpredictable, and difficult to test.

My production recommendation: start at Level 1, earn your way to Level 2, and deploy Level 3 only for tasks where the value justifies the cost and the risk tolerance allows unpredictability. Most production agents I have deployed at Meta operate at Level 1 or Level 2. Level 3 is reserved for internal investigation tools where the cost of a long agent run is justified by the cost of the problem being investigated.

The Agent System Prompt

The system prompt for an agent is more complex than for a single-turn application, because it must communicate not just what the agent does but how it should think about doing it.

AGENT_SYSTEM_PROMPT = """You are an infrastructure debugging assistant. Your goal 
is to investigate production issues and recommend fixes.

AVAILABLE TOOLS:
- query_metrics: Get current or historical metrics for a service
- search_logs: Search error and access logs by time range and query
- get_deployments: List recent deployments for a service
- search_docs: Search internal documentation
- create_ticket: Create a Jira ticket with findings

INVESTIGATION PROCESS:
1. Start by understanding the symptom - what is the user reporting?
2. Gather data from at least two independent sources before forming a hypothesis
3. Test your hypothesis against the data before concluding
4. If your hypothesis is wrong, gather more data - do not force-fit

CONSTRAINTS:
- Maximum 10 tool calls per investigation
- Never create a ticket until you have a clear diagnosis
- If you cannot determine the cause after 10 tool calls, report what you found 
  and recommend manual investigation
- State your confidence level: high (strong evidence), medium (partial evidence), 
  low (insufficient data)

When you have enough evidence, present:
1. Root cause (with evidence)
2. Impact assessment
3. Recommended fix
4. Confidence level"""

Four elements distinguish an agent system prompt from a standard application prompt:

Available tools are listed explicitly. The agent needs to know its capabilities to reason about what steps to take. This overlaps with the tool definitions from Module 7 but frames them in terms of the agent's investigation strategy, not just their API signatures.

A reasoning process is described. "Gather data from at least two sources before hypothesising" is a metacognitive instruction - it tells the agent how to think, not just what to do. These instructions shape the agent's trajectory: without them, agents tend to converge too quickly on a hypothesis from a single data point.

Stopping conditions are explicit. Without a clear stopping condition, agents loop indefinitely - the "need one more tool call" problem from topic 7.3. The 10-call limit and the "report what you found" fallback are guardrails against unbounded execution.

Confidence calibration is requested. Asking the agent to state its confidence creates a signal you can act on: high-confidence conclusions ship as recommendations, low-confidence conclusions trigger human review. This is the grounding-score concept from topic 6.4, applied to agent reasoning.

The Cost of Agents

Agent costs are fundamentally different from pipeline costs, and understanding this before you build prevents the "why is this so expensive?" conversation after deployment.

Pipeline cost is predictable. A pipeline with 3 LLM calls and 2 tool calls has a fixed cost per request - you can compute it with Module 3's model before writing a line of code. You know the token count, the call count, and the latency before the first request arrives.

Agent cost is variable. An agent investigating a simple issue might take 3 tool calls; a complex issue might take 15. The same goal can take different paths on different runs. The cost distribution matters more than the average - a P99 cost of $2.00 with a P50 of $0.20 means 1 in 100 requests costs 10x the typical request. Your cost model must account for this variance with iteration limits (topic 7.3), cost budgets (a maximum spend per agent interaction), and the tiered autonomy approach from this topic's spectrum.

The cost multiplier is real. A simple factual question that a Module 1 LLM call answers in $0.001 becomes a $0.10–

My production rule: set a per-interaction cost budget before deploying any agent. "This agent may spend up to $0.50 per investigation." When the budget is reached, the agent generates its best answer from available evidence and stops. Without a budget, a single pathological interaction can cost more than an entire day of pipeline-based traffic.

Interview: The agent interview question is increasingly common: "Design an AI agent that helps engineers debug production incidents." Strong candidates describe: the agent loop (perceive → reason → act → observe), the tool suite (metrics, logs, deployments, docs), the system prompt with reasoning guidance and stopping conditions, the execution boundary with risk classification on each tool, and the cost/reliability trade-offs of agent-based versus pipeline-based investigation. Candidates who describe only the tool calls without the reasoning layer, stopping conditions, or cost analysis are describing a multi-tool pipeline, not an agent.

Exercise: Build Your First Agent

Exercise: Compose your Module 7–8 tools into an agent:

1. Write an agent system prompt for a task you care about (research assistant, code reviewer, data analyst). Include: goal description, available tools, reasoning guidance, stopping conditions, and output format.
2. Implement the agent loop from topic 7.3, but with the key difference: the model decides which tool to call based on its reasoning, not a fixed sequence.
3. Give the agent a specific goal and watch it work. Log every step: what it perceived, what it reasoned, what it did, what it observed. This trace is your debugging tool.
4. Count the tool calls. Compute the cost. Compare to what a pipeline (fixed sequence of the same tools) would cost for the same task.
5. Challenge: give the agent the same goal twice. Does it take the same path? If not, why not? You are observing the non-determinism of agent reasoning - the fundamental challenge this module addresses.
6. Challenge 2: give the agent a goal that cannot be achieved with the available tools. Does it recognise its limitation and report it, or does it hallucinate a solution? Adjust your stopping conditions until it handles this case gracefully.

What You Should Take From This Lesson

An AI agent is a system that takes a goal, reasons about how to achieve it, takes actions through tools, observes results, and adapts its plan - repeating until the goal is achieved or a stopping condition fires. The agent loop (perceive → reason → act → observe) is the core architecture. Agents are compositions of components you already know: LLMs (Module 1), prompts (Module 4), RAG (Module 6), tools (Modules 7–8), memory (topic 1.5 + Module 5). The new contribution is the reasoning layer - a system prompt and loop structure that lets the LLM decide what to do, rather than the developer deciding. Three autonomy levels (structured, guided, autonomous) trade off predictability against flexibility. Most production applications should be pipelines or structured agents, not fully autonomous agents - the cost, unpredictability, and debugging complexity of autonomous agents is justified only for tasks where the investigation path genuinely cannot be predicted in advance.

Next: the memory problem. Topic 10.2 tackles how agents remember what they have done, what they have learned, and what context persists across sessions - the infrastructure that turns a stateless loop into a system that accumulates understanding.

Back to AI Engineering