Machine learning

Sections: 6
Pages: 187
Reading time: ~7 h
Take first: Foundations

What you will be able to do

Read ML papers and code fluently: shapes, gradients, losses, probabilities and optimizers make sense.
Frame a business problem as an ML problem, pick a baseline, and evaluate it with the right metrics without leakage.
Implement backpropagation from scratch and train neural networks in PyTorch, debugging loss curves systematically.
Explain self-attention and every component of a modern decoder-only transformer, and count its parameters and FLOPs.
Describe how frontier LLMs are pretrained and post-trained (SFT, RLHF, DPO, RL with verifiable rewards) and what scaling laws predict.
Estimate the memory and compute needed to train a model and choose a parallelism strategy.
Design a two-stage recommender or ranking system and run it in production with monitoring for drift.

Models are programs that learn their parameters from data. A spam filter, a ranking model, a recommender, and a frontier LLM are all the same shape underneath: take inputs, compute a prediction, measure how wrong it was, and nudge parameters so the next prediction is less wrong. This topic is the mental model for that loop — the math, the classical baselines, the neural nets and transformers, and the systems that train and serve them. AI engineering and inference engineering both assume you can read shapes, gradients, losses and FLOPs; this is where that fluency comes from.

Why it matters for the rest of the degree

You cannot debug a training run, size a GPU budget, or argue about RAG vs fine-tuning without knowing what the model is doing. Attention and the transformer explain why context length and KV caches dominate serving cost. Scaling laws and distributed training explain why bigger models need more data and more GPUs in fixed ratios. Post-training explains why the same base weights become a chat model, a coding model or a reasoning model. On the product side, embeddings, RAG and evals are applied ML with a different packaging. Skip these pages and those pages become recipes; with it, they become consequences.

Who it's for

Written for a junior software engineer aiming at SDE2: comfortable in Python, shipping features, but new to training models or reading papers. You do not need a math degree. The first unit rebuilds the linear algebra, calculus, probability and optimization you need, with plain-language concepts and worked arithmetic. If you already know that toolkit, skim it and start at framing ML problems.

How to study it

Pace. 187 lessons, about 7 hours of reading. Math toolkit and Classical ML pages are the shortest; Deep learning and Transformers run longer. Do not race past gradients and shapes — every later page reuses them.

Run code. Prefer a laptop with a recent Python and, if you have one, an NVIDIA GPU or Apple Silicon. Reproduce the NumPy linear layer, the tiny autograd engine and the PyTorch training loop. Plot loss curves. When something fails, change one thing (learning rate, batch size, init) and watch the curve respond.

Projects that stick. Pick one or two:

Train a tabular model (logistic regression → LightGBM) on a real dataset with honest validation and a short write-up of metrics and leakage checks.
Implement a tiny decoder-only transformer and count its parameters and FLOPs by hand, then verify with code.
Fine-tune a small open model with LoRA on a narrow instruction set and measure before/after with a fixed eval set.

Further learning. Videos at the end of each lesson are there when prose is not enough — especially 3Blue1Brown for intuition and Karpathy for implementation. Books below are references, not prerequisites.

Roadmap

The numbered list is a shorter path. The sections below are every lesson in the topic.

Math toolkit

The language every model is written in.

Linear algebra for ML: vectors, matrices, batching, SVD intuition and FLOP counting for matmul.
Calculus and gradients: derivatives, the chain rule, common activation derivatives and gradient checking.
Probability, statistics and information: likelihood, Bayes, entropy, cross-entropy and why losses look the way they do.
Optimization and gradient descent: loss landscapes, learning rates, mini-batches, momentum and Adam.

Classical machine learning

Baselines, metrics and tabular workhorses.

Framing ML problems: supervised / unsupervised / RL, splits, baselines and when not to use ML.
Linear and logistic regression: fitting lines, predicting probabilities and regularization.
Evaluating models: precision, recall, AUC, calibration and honest validation.
Features and data preparation: encoding, scaling, imbalance and leakage-safe pipelines.
Trees and gradient boosting: decision trees, forests and XGBoost/LightGBM.
Unsupervised learning: clustering, dimensionality reduction and anomaly detection.

Deep learning

From a neuron to representations that transfer.

Neural networks: layers, activations and why depth helps.
Backpropagation from scratch: a tiny autograd engine in the spirit of micrograd.
PyTorch fundamentals: tensors, autograd, modules and a clean training loop.
Training neural networks well: optimizers, init, normalization and debugging recipes.
Convolutional networks: convolutions, ResNets and ideas that leaked into the rest of DL.
Sequence models and the road to attention: RNNs, LSTMs, seq2seq and why attention won.
Embeddings and representation learning: word2vec, contrastive learning and CLIP.

Transformers and language models

The architecture and training story behind modern LLMs.

Self-attention in depth: Q, K, V, scaled dot-product, masking and multi-head.
The transformer architecture: a decoder-only stack with parameter and FLOP counts.
Tokenization: BPE, vocabularies and weird model behavior from token boundaries.
Pretraining a language model: data, objective and compute for a base model.
Scaling laws: loss vs parameters, data and compute; Chinchilla and test-time compute.
Modern LLM architectures: MoE, GQA/MLA, long context and SSMs.
Generative models beyond text: diffusion, VAEs and autoregressive image/audio.

Scale and post-training

How frontier models are trained and aligned.

Distributed training: data/tensor/pipeline parallelism, ZeRO/FSDP and mixed precision.
Supervised fine-tuning and PEFT: instruction tuning, LoRA and QLoRA.
Reinforcement learning basics: MDPs, policies and policy gradients.
RLHF and preference optimization: reward models, PPO, DPO and AI feedback.
RL for reasoning models: verifiable rewards, GRPO and test-time compute.

Applied ML systems

Recommenders, ranking and shipping models.

Recommender systems: collaborative filtering, two-tower retrieval and multi-stage pipelines.
Ranking and learning to rank: CTR, LTR losses and ranking metrics.
ML in production: feature stores, training-serving skew, drift and monitoring.

What you'll be able to do

Read papers and training code: shapes, gradients, losses and optimizers make sense.
Frame a business problem as an ML problem, pick a baseline and evaluate without leakage.
Implement backprop from scratch and train nets in PyTorch, debugging loss curves systematically.
Explain self-attention and a decoder-only transformer, and count parameters and FLOPs.
Describe pretraining and post-training (SFT, RLHF, DPO, RL with verifiable rewards) and what scaling laws predict.
Estimate memory and compute for training and choose a parallelism strategy.
Design a two-stage recommender or ranker and monitor a deployed model for drift.

Course resources

3Blue1Brown's Essence series is the fastest way to rebuild geometric intuition for linear algebra and calculus before the toolkit pages.

Vectors | Chapter 1, Essence of linear algebra3Blue1Brown

Andrej Karpathy's Neural Networks: Zero to Hero (especially micrograd and makemore) is the implementation companion for the deep learning unit.

The spelled-out intro to neural networks and backpropagation: building microgradAndrej Karpathy

Mathematics for Machine Learning — free textbook covering the linear algebra, calculus, probability and optimization this topic compresses.

Dive into Deep Learning — interactive book with runnable code; strong on classical DL and modern architectures.

The Annotated Transformer — line-by-line walkthrough of the original transformer paper in PyTorch.

Math toolkit

Vectors and dot products

A vector is a list of numbers. The dot product measures how well two vectors line up, and it drives similarity, retrieval and linear models.

Matrices and matrix multiplication

A matrix maps vectors to new vectors. Matrix multiplication is the operation behind every linear layer, and its shape rule catches most bugs.

Linear algebra for ML

Vectors, matrices and matrix multiplication, the language every model is written in. Learn the shapes, then batching, FLOP counts and low-rank tricks.

Batching and broadcasting

Stacking examples into a matrix turns many small multiplies into one large one, which is what GPUs are built for. Broadcasting adds biases and masks, and it fails silently when shapes line up by accident.

Eigenvectors, SVD and low-rank approximation

Eigenvectors are directions a matrix only stretches. The SVD extends the idea to any matrix, and keeping its strongest parts gives a compact approximation that powers PCA and LoRA.

FLOP counting for matrix multiplies

A FLOP is one floating-point operation. Counting the FLOPs of a matrix multiply, using 2mnk, lets you estimate the cost of a layer, a model or a training run before you run it.

Orthogonality and weight initialization

Orthogonal vectors point in independent directions. The scale of random starting weights decides whether a signal vanishes or explodes as it passes through layers.

Derivatives as slopes

A derivative is the slope of a function at one point, and you can estimate it by nudging the input. Autograd computes the exact value, and finite differences check it.

Partial derivatives and gradients

A partial derivative is the slope along one input while the others stay fixed. The gradient collects all of them into one vector that points uphill.

Calculus and gradients

Derivatives, partial derivatives, the chain rule and gradients. This is the math that turns a loss number into parameter updates.

The chain rule on a graph

When one quantity depends on another through a chain of operations, their slopes multiply. Applied on a graph, the chain rule gives every gradient in one backward pass.

Jacobians and vector-Jacobian products

When a layer maps a vector to a vector, its slopes form a matrix called the Jacobian. Training only needs one vector times that matrix, which a single backward sweep gives.

Derivatives of common functions

The slopes of sigmoid, tanh, ReLU, and softmax with cross-entropy. Knowing them by heart makes backprop code easier to read and check.

Second derivatives and curvature

The second derivative says whether the slope is rising or falling. It describes the curvature of the loss, and curvature decides how large a safe learning step can be.

Random variables and distributions

A random variable is a value you cannot know until you observe it. A distribution says how likely each value is.

Conditional probability and Bayes

How learning one fact changes the probability of another, and how Bayes' rule runs that change in reverse. Base rates often matter more than test accuracy.

Probability, statistics and information

The probability ideas behind ML losses, Bayes updates and honest uncertainty. Start here for the map of the section.

Maximum likelihood

Maximum likelihood picks the parameters that make the data you saw most probable. Squared error and cross-entropy are both this one idea.

Softmax and temperature

Softmax turns raw model scores into a probability distribution. Temperature controls how sharp or flat that distribution is.

Entropy and cross-entropy

Entropy measures how surprising a distribution is on average. Cross-entropy measures how well a model's guesses match reality, and it is the classification loss.

Uncertainty and sample size

A metric from a finite test set is an estimate, not a fact. The law of large numbers, the central limit theorem and the bootstrap tell you how much to trust it.

Loss functions and loss landscapes

A loss function scores how wrong a model is. Its graph over the parameters is the loss landscape. Convex landscapes have one bottom, while deep networks have many valleys and flat saddle regions.

Optimization and gradient descent

How models learn, from loss landscapes and gradient descent to learning rates, mini-batches, momentum and Adam.

Gradient descent and the learning rate

Gradient descent moves the parameters a small step against the gradient. The learning rate sets the step size, and a wrong value makes training either crawl or blow up.

Mini-batch SGD and batch size

Estimating the gradient from a random mini-batch is cheap and noisy, and that noise is useful. Batch size trades gradient quality, hardware efficiency and generalization.

Momentum and Adam

Momentum averages recent gradients to speed up consistent directions. Adam also tracks each parameter's gradient size, which makes it the default for many neural networks.

Conditioning, warmup and learning-rate schedules

Features on very different scales create narrow valleys that slow or break gradient descent. Scaling reshapes the valley, and warmup, decay and early stopping control the learning rate over a run.

Classical machine learning

Overfitting, underfitting and bias–variance

A model that memorizes its training data looks brilliant until it sees new data. A model too simple for the pattern fails on both. Two learning curves tell you which problem you have.

Framing ML problems

Learning types, business framing, data splits, baselines and when not to use ML at all.

Types of learning

Supervised learning has labels, unsupervised learning has inputs only, self-supervised learning builds labels from the data, and reinforcement learning learns from rewards. The feedback you have decides which one fits.

Framing a business problem as a prediction

Turn a business decision into a prediction target, a label source, the decision it drives and the cost of each kind of error. Write these four sentences before any training.

Train, validation and test splits

Fit on train, tune on validation, and report once on test. Random splits leak the future and repeated users, so split the way the model will be used.

Baselines and when not to use ML

Build the simplest predictor that already drives the decision, measure it on the same split, and only then train a model. Sometimes a rule or an LLM call is the better tool.

Linear and logistic regression

The two models under almost everything in ML. They fit lines, predict probabilities and stay honest through regularization.

Linear regression and least squares

Linear regression predicts a number as a weighted sum of features. Least squares picks the weights that make the squared errors as small as possible.

Logistic regression and log-odds

Logistic regression turns a linear score into a probability with the sigmoid and trains with log loss. Its coefficients read easily as odds multipliers.

Regularization with L1, L2 and elastic net

Regularization adds a penalty for large weights. L2 shrinks weights smoothly, L1 drives some to exactly zero, and the penalty strength is tuned on validation data.

Feature scaling and leakage-safe pipelines

Linear models and neural nets need features on comparable scales. Learn the scaling from training data only, and keep the scaler inside a pipeline so serving applies the same transform.

Precision, recall and F1

A classifier can make two kinds of mistakes, false alarms and misses. Precision and recall measure each one, and F1 combines them. Accuracy hides both when positives are rare.

Evaluating models

Choose metrics that match the decision, set thresholds from costs, check calibration, and guard against leakage. A high offline score means little until it holds up on real decisions.

Thresholds and costs

A classifier's score becomes a yes or no only after you pick a threshold. The right threshold comes from the costs of each mistake, or from the review capacity you have.

ROC-AUC, PR-AUC and intervals

ROC-AUC and PR-AUC summarize a ranking across all thresholds, and they can disagree sharply on rare positives. A bootstrap interval shows how much a single AUC number can move.

Calibration

A calibrated model's probabilities match reality. When it says 0.8, about 80% of those cases are positive. Check with a reliability table before you use scores as probabilities.

Data leakage

Leakage is when a model learns from information it would not have at prediction time. Offline scores look excellent, then production falls apart. Here is how to find it.

Scaling and transforms

Linear models, nearest neighbors and neural nets are sensitive to the size of numbers. Standardizing, log-transforming skewed values and using ratios puts features on comparable footing.

Encoding categories and text

Categories and words become numbers through one-hot columns, hashing and bag-of-words counts. Each method has a cost that shows up with many levels or a large vocabulary.

Features and data preparation

Turning raw data into model inputs without leaking the future. Covers scaling, encoding, missing values, time windows, target encoding and class imbalance, and keeping training and serving identical.

Missing values and pipelines

A gap in the data can mean something, so flag it and fill it. The fill value must be learned on training data only, which is why imputers belong inside a pipeline.

Point-in-time features

A feature is valid only if it could have been computed at prediction time. Time windows are the most common source of leakage, and a parity checklist keeps training and serving identical.

Target encoding without leakage

Target encoding replaces a category with its average label. Computed naively, it memorizes the training labels. Out-of-fold statistics and smoothing make it safe.

Class imbalance

When positives are 0.1% to 5% of the data, naive training and a 0.5 threshold both fail. Fix the metric and threshold first, then try class weights, and use resampling only as a last resort.

Trees and gradient boosting

Decision trees, random forests and XGBoost/LightGBM: still the best default for tabular data.

Decision trees

A tree partitions the feature space with axis-aligned splits. Each split is chosen to purify the child nodes, and depth is the main capacity knob.

Bagging and random forests

Bagging trains many trees on bootstrap samples and averages them. A random forest also tries a random subset of features at each split, which decorrelates the trees.

Gradient boosting

Boosting adds shallow trees one after another, each fitting the residual of the ensemble so far. Shrinkage and early stopping keep the sum from hugging noise.

XGBoost, LightGBM and CatBoost

Three libraries implement gradient boosting with different engineering. LightGBM is a fast default, CatBoost handles categoricals well, and features still matter more than the logo.

Unsupervised learning

Clustering, dimensionality reduction and anomaly detection without labels.

k-means clustering

k-means assigns each point to the nearest of k centroids, then moves each centroid to the mean of its points. Scale features first, and pick k with a plot plus a use case.

DBSCAN and hierarchical clustering

DBSCAN grows dense regions and labels sparse points as noise. Hierarchical clustering builds a tree of merges you can cut after you see the structure.

PCA for machine learning

PCA finds orthogonal directions of maximum variance. Keep the top components to compress, denoise or plot. t-SNE and UMAP are for eyes, not for features.

Anomaly detection

When the positive class is "anything weird" and labels are rare, flag outliers with thresholds, isolation forests or reconstruction error. The operating point still needs a human cost.

Deep learning

The neuron and why it needs a nonlinearity

A neuron takes a weighted sum of its inputs, adds a bias and applies a nonlinear function. Without the nonlinearity, stacking layers gains nothing.

Neural networks

From a single neuron to multilayer perceptrons: layers, activations, output heads and how to count parameters.

Multilayer perceptrons and the forward pass

A multilayer perceptron stacks fully connected layers. Each layer is a matrix multiply plus a bias and an activation, and a whole batch moves through the stack as matrices.

Activation functions and vanishing gradients

Activations decide how each neuron bends. Sigmoid and tanh flatten out and starve deep networks of gradient. ReLU and its relatives keep the signal flowing, with a few failure modes to watch.

Output heads and loss functions

The last layer and the loss must match the question. Regression uses a linear output with MSE, binary classification uses one logit with binary cross-entropy, and multiclass uses softmax with cross-entropy.

Parameter counting, width and depth

Counting parameters tells you the memory cost before training starts. Depth builds layered features, width adds capacity in parallel, and the widest matrix usually dominates both memory and compute.

Backpropagation from scratch

How training computes the gradient of the loss for every weight in one backward pass. Covers computational graphs, reverse-mode autodiff and the bugs that break training.

Computational graphs and reverse mode

A forward pass records each operation as a node in a graph. The backward pass walks that graph in reverse and applies one local rule per node, so one sweep gives every gradient.

A tiny autograd class

A small Value class that remembers how it was made and runs the local rules backward. It is enough to compute every gradient for a scalar expression.

Gradients for linear layers and softmax

The backward formulas for a batched linear layer, and why softmax with cross-entropy has the simple gradient "probabilities minus the label."

A worked training step

One full forward pass, backward pass and weight update for a tiny two-unit network, with every number shown. It ends with a dead ReLU unit that receives no gradient.

PyTorch tensors

Tensors carry a shape, a dtype and a device. Views, broadcasting and strides explain most of the surprises you will hit.

PyTorch autograd

Autograd records the operations you run and computes gradients with backward(). It is the backprop algorithm, done for you.

PyTorch fundamentals

Tensors, autograd, modules, data loaders and a clean training loop. The daily-driver API for deep learning. Start here for the map.

PyTorch modules and data

nn.Module holds the weights and the forward pass. Dataset and DataLoader feed batches to the model. Both have a few traps.

The PyTorch training loop

The canonical loop repeats zero_grad, forward, loss, backward and step. The order matters, and so does switching between train and eval mode.

Mixed precision and checkpoints

Mixed precision runs most math in a smaller number format to save memory and time. Checkpoints save enough state to resume a run exactly.

Training neural networks well

Optimizers, schedules, initialization, normalization, regularization and a practical recipe for debugging loss curves.

BatchNorm, LayerNorm and RMSNorm

Normalization rescales activations so each layer sees well-scaled inputs. BatchNorm uses the batch, LayerNorm uses one example's features, and RMSNorm drops the mean.

Debugging loss curves

Check loss at initialization, overfit one batch, then read train versus val systematically. Most "broken training" is a shape, LR or data bug, not a new optimizer.

Gradient clipping and accumulation

Clipping caps rare gradient spikes. Accumulation simulates a larger batch when memory is tight. Get the loss averaging right or the effective learning rate silently changes.

Regularization for neural nets

Weight decay, dropout, augmentation and early stopping keep a net from memorizing. AdamW applies decay outside the adaptive update, which is why it replaced Adam plus L2.

Convolution as a sliding filter

A convolution slides a small kernel across the input and takes a weighted sum at each position. The same weights are reused everywhere, which keeps the layer small and makes it shift-aware.

Convolutional networks

How CNNs use small shared filters, stacked layers and residual connections to learn images, and why transfer learning from pretrained models is the default.

Conv output shapes and parameter counts

Kernel size, stride and padding set the output size of a convolution. Counting weights shows where a CNN keeps its parameters, which is often in the last dense layer.

Residual connections and ResNet

A residual block adds its input back to its output, so each block only needs to learn a change. That keeps very deep networks trainable and gives gradients a path that skips the weights.

Transfer learning for vision

Reuse a network trained on a large image dataset and adapt it to your own labels. Early layers carry over, and the head and normalization usually need to change.

Pooling and cheap convolutions

Pooling shrinks the grid by taking a summary of each window. 1x1 and depthwise separable convolutions cut the weights and compute a standard convolution would need.

Sequence models and the road to attention

RNNs, LSTMs and seq2seq — the problems they hit, and how attention emerged to fix them before transformers took over.

RNNs and backprop through time

A recurrent net compresses the past into a hidden state and reuses the same weights at every step. Training unrolls that loop and runs ordinary backprop on the chain.

Vanishing gradients and LSTMs

Unrolling an RNN multiplies the same Jacobian many times, so gradients vanish or explode. LSTMs and GRUs add gated highways so a signal can travel further.

Seq2seq and the fixed-vector bottleneck

Early translation models encoded a whole sentence into one vector, then decoded from it. Long sentences overflowed that vector, which is the problem attention was invented to fix.

Bahdanau attention

Instead of crushing a sentence into one vector, the decoder looks at all encoder states at each step. That soft alignment is the parent of self-attention.

Embedding lookup tables

An embedding layer is a learned table with one row per ID. Looking up a row turns a discrete ID into a vector that training can shape.

Embeddings and representation learning

Learning vectors that capture meaning, from word2vec and contrastive training to CLIP and retrieval. Start here for the map of embedding concepts.

word2vec

word2vec trains one vector per word so that words used in similar contexts end up close together. Skip-gram with negative sampling is the version to know.

Contrastive learning and InfoNCE

Contrastive learning pulls positive pairs together and pushes negatives apart. InfoNCE is the loss that does it with a softmax over a batch, and temperature controls how sharp it is.

Bi-encoders and cross-encoders

A bi-encoder embeds the query and the document separately, so search is fast. A cross-encoder reads them together, so it is more accurate but too slow for the whole corpus.

CLIP and zero-shot classification

CLIP trains an image encoder and a text encoder so matching image and caption pairs land close together. That lets you classify images by writing the class names as text.

Transformers and language models

Softmax and dot products

A dot product scores how well two vectors agree. Softmax turns any list of scores into weights that are positive and sum to 1.

Self-attention

The operation at the core of every modern language model. Each position builds a weighted average of the others, with weights set by query-key matches.

Queries, keys and values

Self-attention scores every pair of positions with a query and a key, then mixes value vectors using those scores. The √d_k factor keeps the softmax well behaved.

Attention masks

A mask blocks some query-key pairs before softmax. Causal masks stop a language model from reading future tokens, and padding masks ignore filler in a batch.

Multi-head attention

Multi-head attention runs several attention patterns in parallel, each on a slice of the model width, then merges them with one output matrix.

Attention cost and long context

Attention compares every position with every other, so its cost grows with the square of the sequence length. Here are the numbers and the variants that reduce them.

The transformer architecture

Every component of a modern decoder-only LLM, from embeddings to the LM head, with parameter and FLOP counts.

Embeddings and rotary positions

A token id becomes a vector from a table. Attention has no sense of order unless you inject position, and RoPE does that by rotating queries and keys.

The transformer block

A modern decoder block is pre-norm, attention, residual, pre-norm, MLP, residual. The residual stream is the highway. Sublayers read it and write an additive update.

Transformer parameter and FLOP counts

Non-embedding parameters are about 12 L d² for a classic decoder block. Forward FLOPs are about 2N per token. Training is about 6N.

Encoder vs decoder transformers

BERT looks both ways, T5 has an encoder and a decoder, and GPT looks only left. Decoder-only won for generative LLMs because one stack serves pretraining, SFT and chat.

Tokenization

Byte-pair encoding in depth: how vocabularies are built and why tokenization causes strange model behavior.

Why subword tokenization

Characters make sequences too long. Words make the vocabulary unbounded. Subwords sit in the middle, with a fixed vocabulary and a workable sequence length.

Byte-pair encoding

BPE builds a vocabulary by greedily merging the most frequent adjacent pairs, starting from bytes so every Unicode string is representable.

WordPiece, SentencePiece and special tokens

WordPiece and Unigram are cousins of BPE with different training criteria. Pre-tokenizers, specials and chat templates decide the actual ID sequence you train on.

Tokenizer pathologies

Many "the model can't spell" demos are tokenizer artifacts. Arithmetic, letter counting, trailing spaces and non-English fertility all come from how text was cut into IDs.

Next-token prediction

A language model learns by guessing the next token of real text. The text supplies the label, so no human annotation is needed, and the guesses reward learning grammar, facts and patterns.

Pretraining a language model

Data, objective and compute: how a base model is trained on trillions of tokens.

The pretraining data pipeline

Frontier pretraining is mostly data engineering. Raw web pages become clean, deduplicated, weighted text through extraction, filtering, deduplication and mixing. The data recipe moves quality as much as the architecture does.

Compute budget and the 6ND rule

A training run costs about six FLOPs per parameter per token. Multiply parameters by tokens, divide by realistic throughput, and you have a napkin estimate in GPU-days.

Training dynamics, schedules and checkpoints

A long pretraining run needs a learning-rate schedule, a plan for loss spikes and frequent checkpoints. Warmup, cosine or WSD decay, clipping and rollbacks keep a multi-week run alive.

Evaluating a pretraining run

Validation loss shows whether a base model is healthy. Downstream benchmarks add signal but can be noisy or contaminated, so check for leaks and compare runs at equal token budgets.

Scaling laws

How loss scales with parameters, data and compute, and what Chinchilla and test-time compute changed.

Kaplan power laws

For transformer language models, loss falls as a smooth power law in parameters, data and compute, which is why labs can budget a large run from smaller ones.

Chinchilla and compute-optimal training

For a fixed training FLOP budget, scale parameters and tokens roughly equally. A stuck rule of thumb is about 20 tokens per parameter.

Over-training for inference

Chinchilla minimizes training loss per FLOP. Products often train a smaller model on far more tokens so each served token is cheaper.

Extrapolating scaling laws

Labs fit loss on small proxy runs and aim a large run at a target. The fit is only as good as the match in data, architecture and metrics.

KV cache and attention variants

During generation, a model keeps keys and values for every past token. MQA, GQA and MLA shrink that cache, which often limits how many long conversations fit on one GPU.

Modern LLM architectures

Mixture of experts, GQA and MLA, long context, state space models and multimodal transformers.

Mixture of experts

A mixture-of-experts layer holds many expert networks and sends each token to only a few of them. Total parameters are large, while the compute per token stays closer to a smaller dense model.

Long-context techniques

Long context needs positions that stretch past the training length, attention that stays affordable, and training data that teaches the model to use distant text. A large context number on a spec sheet guarantees none of these.

State space models and hybrids

State space models carry a compact running state along a sequence, so their cost grows linearly with length. Hybrids mix these blocks with occasional attention layers to keep sharp recall.

Multimodal transformers

Images and audio become sequences of vectors that share one transformer with text. Early fusion trains them together, while late fusion attaches a vision encoder to a frozen language model.

Families of generative models

Autoregressive models, VAEs, GANs, diffusion and flow matching all learn to sample new data. They differ in how they generate, and in what they make easy and what they make hard.

How diffusion learns to denoise

A diffusion model learns to undo gradual noise. Training adds noise to clean images and teaches a network to predict that noise. Sampling starts from pure noise and removes a little at a time.

Generative models beyond text

How images, video and speech are generated. Compares the main model families, builds diffusion from first principles, and explains why diffusion inference costs so differently from chat.

Latent diffusion, guidance and conditioning

Latent diffusion runs the denoising in a small compressed space, and classifier-free guidance controls how closely the output follows the prompt. Conditioning covers the other inputs a denoiser can take.

Fewer steps, samplers and distillation

Textbook diffusion needs hundreds of denoising steps. Better samplers, noise schedules and distilled models produce good images in a few steps, which makes interactive products practical.

Diffusion inference cost

Serving an image model is a different systems problem from serving an LLM. Each image runs the denoiser many times over the whole latent, so the cost is steps times denoiser cost, not one pass per token.

Speech and video generation

Speech and video use the same generative families with different bottlenecks. Speech often becomes discrete audio tokens, and video multiplies the work by the number of frames.

Scale and post-training

Memory math for training

Training memory is much more than the weights. Mixed-precision Adam needs about 16 bytes per parameter before any activations, which is why a 7B model needs over 100 GB.

Mixed precision

Mixed precision runs most matrix multiplies in 16-bit floats and keeps a 32-bit master copy of the weights. fp16 needs loss scaling to avoid underflow. bf16 usually does not.

Distributed training

How to train models too big for one GPU, using data, sharding, tensor, pipeline and context parallelism. Covers the memory math that forces the choice and how to judge the result.

Data parallelism and all-reduce

Data parallelism gives each GPU a full copy of the model and a different slice of the batch. The GPUs then sum their gradients with an all-reduce, so every copy applies the same update.

ZeRO and FSDP

ZeRO and FSDP remove the copies that data parallelism makes of optimizer state, gradients and weights. Each GPU keeps only a shard and gathers the rest when a layer needs it.

Tensor, pipeline and context parallelism

Model parallelism splits the network itself. Tensor parallelism splits weight matrices, pipeline parallelism splits layers into stages, and context parallelism splits the sequence. Each one needs a different kind of network link.

Planning a training run

Choose a parallel layout that fits in memory, use checkpointing and accumulation to fit activations, and judge the result by MFU. Worked through a 70B model on 64 GPUs.

Supervised fine-tuning and PEFT

Instruction tuning, chat formats, data quality, LoRA/QLoRA math, and the tooling that turns a base model into a useful assistant.

Instruction tuning and loss masking

Supervised fine-tuning teaches a base model to follow instructions. Gradients should flow through the assistant tokens, not the user's prompt, and train and serve must share one chat template.

LoRA

LoRA freezes the base weights and trains a pair of thin matrices whose product is the update. Trainable parameters drop by orders of magnitude.

QLoRA

QLoRA freezes the base model in 4-bit NormalFloat and trains LoRA adapters in higher precision, which is why a 70B model can be fine-tuned on one 48 GB GPU.

PEFT vs full fine-tuning

Full fine-tuning updates every weight. PEFT freezes the trunk and trains a small adapter. Default to LoRA for product work, and go full only when you need a large shift.

MDPs, returns and values

The vocabulary of reinforcement learning. An agent acts, the environment returns a state and a reward, and returns and values measure how good those actions are over time.

Reinforcement learning basics

MDPs, returns, policy gradients, advantages and PPO. The minimum RL needed to read modern LLM post-training papers. Start here for the map.

Policy gradients and advantages

Policy gradients adjust the policy directly. Make the actions that led to high returns more likely. Baselines and advantages cut the noise in that update.

PPO and the KL leash

Proximal Policy Optimization (PPO) limits how far one update can move the policy. The clipped ratio and a KL penalty keep language models close to where they started.

RL for language models

How the RL vocabulary maps onto a language model. The model is the policy, tokens are actions, and a completion is the episode. Credit assignment and exploration are the hard parts.

Preference pairs and the Bradley-Terry reward model

People find it easier to compare two answers than to write a perfect one. Preference pairs train a reward model with a simple logistic loss.

RLHF and preference optimization

How language models learn from human and AI preferences. Supervised fine-tuning, a reward model, a KL-leashed RL step, and DPO as a single-loss alternative. Start here for the map.

The KL leash in RLHF

The third RLHF stage trains the policy against a reward model, with a KL penalty to the supervised model. The penalty weight decides whether the model improves or games its reward.

Direct preference optimization (DPO)

DPO optimizes the same KL-constrained goal as RLHF with a single loss on preference pairs. It needs no separate reward model and no sampling loop.

RLAIF and Constitutional AI

RLAIF uses a model's judgments in place of human labels, and Constitutional AI makes the rubric an explicit list of principles. Both scale feedback, and both need human checks.

Chain of thought and test-time compute

Chain of thought is intermediate reasoning written before the answer. Test-time compute is the extra inference work spent on a query, and it can buy accuracy on hard problems.

Verifiable rewards

Reasoning RL works best when a checker can say whether an answer is right. Exact answers, unit tests and proof checkers give the cleanest reward a language model can get.

RL for reasoning models

How reasoning models learn to think before they answer, using verifiable rewards, GRPO and test-time compute. Start here for the map.

GRPO

Group Relative Policy Optimization samples several completions per prompt and uses their average reward as the baseline. It drops the value network that PPO needs.

The R1 training recipe

DeepSeek-R1 combined pure RL, cold-start supervised data, more RL and distillation. The transferable pattern is verifiable RL, light supervised polish, then distill to smaller models.

Process vs outcome rewards

An outcome reward scores only the final answer. A process reward scores each step. Outcomes are simpler and the default, while process rewards give finer credit at a higher labeling cost.