Preface¶
A course in five units
This book is a textbook for a first university course in machine learning. It follows the five units of the course: the learning problem, data preprocessing and dimensionality reduction, regression, classification, and clustering. It assumes a pass in Class XII mathematics and some programming in Python; the mathematics used is school algebra, a little calculus (derivatives and partial derivatives) and a little linear algebra (vectors, matrices, eigenvalues of a 2 × 2 matrix), and each is recalled where it is first needed.
Who the book is for¶
The book is written for the student who will sit the examination and for the teacher who must plan the lectures. The student will find each idea explained in plain words before any notation, every formula derived or at least motivated, and every calculation of the course worked out line by line. The teacher will find the units broken into chapters of about one to two lectures each, with worked examples ready for the board and exercises with answers for tutorial sessions.
How the book maps to the course¶
| Unit | Topics | Hours | Chapters |
|---|---|---|---|
| 1. Introduction | Basic definitions and concepts, key elements, supervised and unsupervised learning, introduction to reinforcement learning, applications of ML | 5 | 1 |
| 2. Preprocessing | Feature scaling, feature selection methods, dimensionality reduction (principal component analysis), class balancing, outlier detection and removal | 8 | 2, 3 |
| 3. Regression | Linear regression with one variable and with multiple variables, gradient descent, over-fitting, regularisation, regression evaluation metrics | 12 | 4, 5 |
| 4. Classification | Decision trees, naive Bayes, logistic regression, k-nearest neighbours, perceptron, multilayer perceptron, neural networks, back-propagation, support vector machines, classification evaluation metrics | 12 | 6 to 12 |
| 5. Clustering | Approaches for clustering, distance metrics, k-means clustering, hierarchical clustering | 8 | 13, 14 |
The course also includes twelve practical programs. They are listed below with the chapter that supplies the theory behind each; the practical work itself, done in Python on public data sets, is outside this book.
| Practical | Theory in chapter |
|---|---|
| Naive Bayes classifier | 7 |
| Simple linear regression; multiple linear regression | 4 |
| Polynomial regression; lasso and ridge regression | 5 |
| Logistic regression | 8 |
| Artificial neural network | 10 |
| k-NN classifier | 9 |
| Decision tree classification | 6 |
| SVM classification | 11 |
| K-means clustering; hierarchical clustering | 13, 14 |
| Evaluation protocol: train/test split, k-fold cross-validation, MSE and \(R^2\) for regression, accuracy, precision, recall, specificity, F1 and AUC for classification | 5, 12 |
How the chapters are built¶
Every chapter opens with a short introduction and, for the first chapter of a unit, the learning outcomes of that unit. The numbered sections that follow explain the topic in continuous prose. Four kinds of box recur:
- Definitions and notes state a term precisely.
- Key ideas pick out the one thing in a section that must not be forgotten.
- Cautions mark the traps that cost marks and accuracy.
- Examples work a problem completely, with every step shown. Derivations of the important results are presented in the same form.
Each chapter ends with a list of common mistakes, a summary of what you should now be able to do, and exercises. Every exercise has its answer in a box that stays closed until you open it: attempt the exercise on paper first.
How to use the book¶
Read a chapter once for the idea, then read it again with pencil and calculator, redoing every worked example before looking at its steps. The examples deliberately use small numbers, so that the arithmetic can be done by hand and the structure of the method is not hidden by the numbers. Then do the exercises and check them. Three habits pay for themselves throughout the course:
- State, before computing anything, what is being predicted and how the result will be judged.
- Keep the test data sealed. Anything that is learned from data (a scaler, a selected feature, a threshold, a number of clusters) is learned from the training part alone.
- Compare every model with a trivial baseline before believing it.
Notation¶
The symbols are the same throughout. \(m\) is the number of examples and \(n\) the number of features; \(x^{(i)}\) is the \(i\)-th example and \(x_j\) its \(j\)-th feature; \(y\) is the target and \(\hat{y}\) the model's prediction; \(\theta\) (or \(w\) and \(b\)) are the parameters, learned from data, while \(\alpha\), \(\lambda\), \(k\) and \(C\) are hyper-parameters, chosen on validation data. The appendix Formula Sheet and Notation lists every symbol with the usual pitfalls, followed by the formulas of each unit for revision, and a glossary closes the book.
A note on units and chapters¶
The word unit always refers to one of the five units of the course; the book is divided into chapters, which are shorter. Where the text says "Unit 3" it means the regression material of Chapters 4 and 5.