Skip to content

Preface

A course in five units

This book is a textbook for a first university course in machine learning. It follows the five units of the course: the learning problem, data preprocessing and dimensionality reduction, regression, classification, and clustering. It assumes a pass in Class XII mathematics and some programming in Python; the mathematics used is school algebra, a little calculus (derivatives and partial derivatives) and a little linear algebra (vectors, matrices, eigenvalues of a 2 × 2 matrix), and each is recalled where it is first needed.

Who the book is for

The book is written for the student who will sit the examination and for the teacher who must plan the lectures. The student will find each idea explained in plain words before any notation, every formula derived or at least motivated, and every calculation of the course worked out line by line. The teacher will find the units broken into chapters of about one to two lectures each, with worked examples ready for the board and exercises with answers for tutorial sessions.

How the book maps to the course

Unit Topics Hours Chapters
1. Introduction Basic definitions and concepts, key elements, supervised and unsupervised learning, introduction to reinforcement learning, applications of ML 5 1
2. Preprocessing Feature scaling, feature selection methods, dimensionality reduction (principal component analysis), class balancing, outlier detection and removal 8 2, 3
3. Regression Linear regression with one variable and with multiple variables, gradient descent, over-fitting, regularisation, regression evaluation metrics 12 4, 5
4. Classification Decision trees, naive Bayes, logistic regression, k-nearest neighbours, perceptron, multilayer perceptron, neural networks, back-propagation, support vector machines, classification evaluation metrics 12 6 to 12
5. Clustering Approaches for clustering, distance metrics, k-means clustering, hierarchical clustering 8 13, 14

The course also includes twelve practical programs. They are listed below with the chapter that supplies the theory behind each; the practical work itself, done in Python on public data sets, is outside this book.

Practical Theory in chapter
Naive Bayes classifier 7
Simple linear regression; multiple linear regression 4
Polynomial regression; lasso and ridge regression 5
Logistic regression 8
Artificial neural network 10
k-NN classifier 9
Decision tree classification 6
SVM classification 11
K-means clustering; hierarchical clustering 13, 14
Evaluation protocol: train/test split, k-fold cross-validation, MSE and \(R^2\) for regression, accuracy, precision, recall, specificity, F1 and AUC for classification 5, 12

How the chapters are built

Every chapter opens with a short introduction and, for the first chapter of a unit, the learning outcomes of that unit. The numbered sections that follow explain the topic in continuous prose. Four kinds of box recur:

  • Definitions and notes state a term precisely.
  • Key ideas pick out the one thing in a section that must not be forgotten.
  • Cautions mark the traps that cost marks and accuracy.
  • Examples work a problem completely, with every step shown. Derivations of the important results are presented in the same form.

Each chapter ends with a list of common mistakes, a summary of what you should now be able to do, and exercises. Every exercise has its answer in a box that stays closed until you open it: attempt the exercise on paper first.

How to use the book

Read a chapter once for the idea, then read it again with pencil and calculator, redoing every worked example before looking at its steps. The examples deliberately use small numbers, so that the arithmetic can be done by hand and the structure of the method is not hidden by the numbers. Then do the exercises and check them. Three habits pay for themselves throughout the course:

  1. State, before computing anything, what is being predicted and how the result will be judged.
  2. Keep the test data sealed. Anything that is learned from data (a scaler, a selected feature, a threshold, a number of clusters) is learned from the training part alone.
  3. Compare every model with a trivial baseline before believing it.

Notation

The symbols are the same throughout. \(m\) is the number of examples and \(n\) the number of features; \(x^{(i)}\) is the \(i\)-th example and \(x_j\) its \(j\)-th feature; \(y\) is the target and \(\hat{y}\) the model's prediction; \(\theta\) (or \(w\) and \(b\)) are the parameters, learned from data, while \(\alpha\), \(\lambda\), \(k\) and \(C\) are hyper-parameters, chosen on validation data. The appendix Formula Sheet and Notation lists every symbol with the usual pitfalls, followed by the formulas of each unit for revision, and a glossary closes the book.

A note on units and chapters

The word unit always refers to one of the five units of the course; the book is divided into chapters, which are shorter. Where the text says "Unit 3" it means the regression material of Chapters 4 and 5.