Classical machine learning
Framing ML problems
Learning types, business framing, data splits, baselines and when not to use ML at all.
Most machine learning projects fail before the first model is trained. The failure is usually in the framing: the wrong prediction target, labels that do not match the decision, a split that leaks the future, or a problem that a rule would have solved more cheaply. Framing turns a business decision into a learnable prediction, and it decides whether learning is the right tool at all.
Where it fits
Framing comes before everything else in classical ML. The choice of model, the features and the metric all depend on it. Get the framing right, and the later steps have a clear target. Get it wrong, and a better algorithm only makes the wrong answer more precise.
Take a question like "Will this customer leave next month?" It looks like a model problem, but it is really a decision problem. Someone must decide who gets a retention offer, so the framing has to say what the label is, when the prediction is made, and what a wrong call costs. Each later page in this topic answers one of those questions.
Concepts
Read the table in order. Each row is a question that the next row depends on.
| Step | Concept | What it answers |
|---|---|---|
| 1 | Overfitting, underfitting and bias–variance | What do the two failure modes look like on the learning curves? |
| 2 | Types of learning | Do you have labels, and is the target a number or a category? |
| 3 | Framing a business problem | What is the target, the label, the decision and the cost of errors? |
| 4 | Train, validation and test splits | How should the data be divided so the score is honest? |
| 5 | Baselines and when not to use ML | What simple predictor must the model beat, and when is ML the wrong tool? |
Common mistakes
- Random splits on time-ordered or grouped data. The score looks better than the deployed model will be.
- Judging by accuracy on imbalanced classes. A model that predicts the majority class can score well and do nothing useful.
- Looking at the test set while tuning. The reported score then becomes optimistic.
- Skipping the baseline. A score of 0.82 means nothing until you know what a simple rule scores on the same split.
Key takeaways
- Write the target, label source, decision and error costs before training.
- Match the learning type to the labels you actually have.
- Split the data the way the model will be used, by time or by group.
- Beat a baseline on the same split. If you cannot, fix the framing before you tune the model.
Further learning
Martin Zinkevich's practical engineering rules are the default checklist before you train anything fancy:
Rules of Machine Learning (Google)Rules of Machine Learning: | Google for DevelopersGoogle for Developers
The free ISL book covers supervised learning framing, bias–variance, and train/test thinking with clear examples:
An Introduction to Statistical LearningAn Introduction to Statistical LearningGoogle's free introduction is a fast, interactive pass over framing, splits and generalization:
Machine Learning Crash Course (Google)Machine Learning | Google for DevelopersGoogle for Developers
Related
Evaluating models
Choose metrics that match the decision, set thresholds from costs, check calibration, and guard against leakage. A high offline score means little until it holds up on real decisions.
Linear and logistic regression
The two models under almost everything in ML. They fit lines, predict probabilities and stay honest through regularization.
Fine-tuning vs prompting vs RAG
When to change weights instead of the prompt or the retrieved context, and how to decide with evals.