AI engineering

An LLM is a probabilistic text function: tokens in, a distribution over the next token out. Chatbots, RAG systems, coding agents and support bots are all the same loop with different context and tools. AI engineering is the craft of making that loop dependable enough to ship — choosing models, shaping prompts and context, retrieving the right evidence, calling tools safely, measuring quality and controlling cost.

The machine learning course covers how models are trained. These pages assumes you will mostly call models, and focuses on what changes when the model is a black-box API (or a self-hosted checkpoint) inside a product.

Who this is for

You are a software engineer who needs to put LLMs into real systems: an API route that answers from company docs, a workflow that extracts fields from PDFs, an agent that can open tickets and escalate. You write TypeScript or Python comfortably. You do not need a deep learning background — when a CS idea matters (token, logit, embedding, attention), lessons define it inline.

If you want GPU kernels, quantization math and serving fleets, that is the inference course. If you want transformers from first principles, start in ML and come back here for the product layer.

How to study it

Read in unit order the first time. Foundations (tokens, transformers, sampling, APIs) make every later page less mysterious; skip them and you will fight cost and latency without a model of why.

For each technique page, do one small end-to-end exercise: a prompt with three golden cases, a tiny RAG corpus, a tool-calling loop with one function, an eval set of twenty examples. The Further learning links are reference depth, not homework — pull them when you are stuck on a decision.

When you change a prompt, a model or a retrieval setting, re-run the same eval set. That habit matters more than any single architecture pattern.

Expect the fundamentals unit to take a focused afternoon if you already call LLM APIs daily, or two or three sittings if the token/attention vocabulary is new. Retrieval and agents each deserve a weekend project of their own. Production pages repay a second pass after you have something live — the latency and caching notes make more sense once you have a bill.

Unit roadmap

The numbered list is a shorter path. The sections below are every lesson: 155 in all, about 6 hours of reading.

LLM fundamentals

What the model is doing when you call it, and how to choose one.

• The LLM landscape and model choice — frontier vs open-weight, reasoning models, a selection process driven by your evals
• Tokens and context windows — tokenization, cost math, budgeting the window
• How transformers work (practitioner view) — attention, prefill/decode, KV cache, why position matters
• Sampling and decoding — temperature, top-p, penalties, when to leave defaults alone
• Working with LLM APIs — messages, streaming, stop reasons, retries, batch APIs

Prompting and context

What you put in the window.

• Prompting — instructions, examples and structure as specification
• Structured outputs — schemas, constrained decoding, validation
• Context engineering — ordering, compaction, memory, long context vs retrieval
• Multimodal inputs and outputs — images, PDFs, audio and video

Retrieval

Grounding answers in your data.

• Embeddings and similarity — vectors, similarity metrics, choosing an embedder
• Vector search and vector databases — ANN algorithms and store choices
• Retrieval-augmented generation — ingest, chunk, retrieve, generate with citations
• Advanced RAG — hybrid search, reranking, query rewriting, retrieval metrics

Agents and tools

Letting the model act.

• Tool use and function calling — schemas and the tool loop
• Agents and agent architectures — workflows vs agents, ReAct, stopping
• Model Context Protocol (MCP) — the open standard for tools and data
• Multi-agent systems — orchestrators, isolation, coordination cost
• Agent infrastructure — sandboxes, durable execution, permissions
• Voice agents — STT→LLM→TTS vs speech-to-speech, latency budgets

Quality and safety

Knowing whether it works and keeping it safe.

• Evaluating LLM apps — datasets, error analysis, metrics that drive changes
• LLM-as-a-judge — grader design and calibration
• Guardrails, security and safety — injection, exfiltration, jailbreaks
• Fine-tuning vs prompting vs RAG — when each lever is worth it

Production and case studies

Shipping under latency, cost and UX constraints.

• Latency and cost optimization — streaming, routing, cascades, batching
• Caching for LLM apps — prompt, exact and semantic caches
• LLMOps and observability — tracing, prompt management, online monitoring
• Designing AI product experiences — trust, control, failure states
• Case study: customer support agent
• Case study: coding agent
• Case study: document extraction pipeline

Course resources

Chip Huyen's AI Engineering is the closest book-length companion to this syllabus — systems view, not research papers.

AI Engineering (Chip Huyen, O'Reilly)oreilly.com

Karpathy's intro talk is the best one-hour mental model of what an LLM is and is not. Watch it before the fundamentals unit if the topic is new.

His longer deep dive is useful when you want training and post-training intuition without committing to a full ML course.

For live quality and cost comparisons across providers, keep Artificial Analysis and LMArena bookmarked — then build your own eval set before you trust either for a product decision.

Provider docs you will live in: Claude, OpenAI.