Classical machine learning
Models you can fit, read and check
- Beginner
- 10 chapters
- 154 min read
The models that still earn their place: nearest neighbours, linear and logistic regression and decision trees, plus the habits that keep them honest — feature scaling and encoding, train and test splits, baselines, precision and recall, thresholds, calibration, coefficient and permutation explanations, and a model card with real numbers.
What You'll Learn
- Frame a supervised problem as a hypothesis class, a loss and an optimisation
- Engineer features that match the model and the problem
- Split data without leaking the test set
- Choose k and a distance metric for nearest neighbours
- Fit and interpret a linear regression
- Read logistic regression coefficients as log-odds
- Build a decision tree and limit its depth
- Choose a threshold from precision, recall and cost
- Read coefficients and permutation importance without claiming cause
- Write a model card with honest metrics and limits
Prerequisites
- Comfortable with Python lists, functions and loops
- Basic arithmetic: averages, squares and square roots
- No ML library is needed — every example is plain Python
Vocabulary
The terms the chapters assume, in plain words.
- accuracy
- The share of predictions that are correct, counting every class the same.
- additive
- Built by summing separate contributions, so each part’s effect can be read on its own.
- baseline
- A reference score from a rule so plain it needs no learning, such as always naming the most common answer.
- features
- The input columns for one example — the numbers a model is allowed to read before it predicts.
- fitting
- Adjusting a model’s numbers so its predictions get closer to the answers in the training data.
- generalisation
- How well a model performs on examples it never saw while being fitted.
- hypothesis class
- The family of shapes a model is allowed to use, such as a straight line or a tree.
- label
- The correct answer for one example — what you want the model to predict.
- loss
- A single number measuring how wrong a prediction is; smaller means better.
- metric
- A single number used to judge predictions, such as the share answered correctly.
- noise
- Random variation in data that carries no real signal, which a model can mistake for a pattern.
- optimiser
- The procedure that changes a model’s numbers to make its loss smaller.
- overfitting
- When a model memorises accidental details of its training examples and does worse on new ones.
- parameters
- The numbers inside a model that fitting adjusts, as opposed to the inputs it reads.
- regularisation
- Any penalty or limit that discourages a model from chasing noise in its training examples. The aim is better performance on new data.
- supervised learning
- A setting where each example carries the answer you want the model to predict.
- test set
- Examples kept out of fitting, used to estimate how the model will do on data it has not seen.
- training set
- The examples a model is fitted on; a score there is not an estimate of performance on new data.
- weights
- The numbers a model multiplies each input by before adding the results together.
Course Content
- 01Learning from examplesNot started
Supervised learning is curve fitting with a budget. You choose a hypothesis class, a loss and an optimisation, and the data decides the numbers.
- 02Train, validation and testNot started
Three sets answer three questions: can the model fit, which setting should you choose, and how will it do on data it has never seen?
- 03Nearest neighboursNot started
The simplest learner stores the data and answers by looking at the closest examples. Distance, k and scaling decide everything it does.
- 04Linear regressionNot started
Fit a line by minimising squared error. The closed form is exact, and the gradient loop is the same idea in miniature.
- 05Logistic regressionNot started
A linear score passed through a sigmoid gives a probability. The loss is log loss, and the decision boundary is still a line.
- 06Decision treesNot started
A tree asks a sequence of yes/no questions. Gini or entropy picks each split, and depth is the regularisation that keeps it honest.
- 07Precision, recall and thresholdsNot started
A classifier outputs a score; a threshold turns it into a decision. Precision, recall and the confusion matrix tell you what that decision costs.
- 08Feature engineeringNot started
A model can only use what its columns expose. Scaling, encoding, missing values and ratios decide whether the algorithm ever sees the pattern you meant.
- 09Interpreting a modelNot started
A coefficient is a conditional statement about the model, not about the world. Read it in its units, check it against a held-out set, and never mistake importance for cause.
- 10Capstone: an honest model cardNot started
Train a small classifier, evaluate it once on held-out data, and write the numbers a reader needs: dataset, split, metrics, subgroups, calibration and limits.