Skip to content

Glossary

The terms of the book, with the chapter where each is explained

A–C

Accuracy
The fraction of a classifier's predictions that are correct. Misleading when one class is much more common than the other (Chapter 9).
Agglomerative clustering
Hierarchical clustering that starts with every example as its own cluster and repeatedly merges the two closest clusters, according to a linkage rule (Chapter 11).
Aggregation
Combining many values into one, such as a sum, mean, count or maximum (Chapters 2, 3 and 5).
Alignment
pandas' matching of values by index label, rather than by position, when two Series or DataFrames are combined (Chapter 4).
AUC
Area under the ROC curve: the probability that a classifier scores a randomly chosen positive example above a randomly chosen negative one; 0.5 is random guessing, 1 is perfect (Chapter 10).
Axis
A dimension of an array. In an aggregation, the axis named is the one that disappears: axis=0 gives one result per column (Chapter 3). In matplotlib, an Axes object is one plot within a figure (Chapter 6).
Bag of words
A representation of a document by the counts of the words it contains, ignoring their order (Chapter 12).
Baseline
A trivial model, such as predicting the mean or the most common class, against which real models are compared (Chapters 8 and 9).
Bias (statistical)
A systematic difference between an estimate and the true value. In modelling, the error that comes from a model too simple to capture the pattern (underfitting) (Chapters 7 and 8).
Bias (in data)
Systematic distortion in how data was sampled, measured or labelled, which a model trained on it can reproduce (Chapter 13).
Binning
Turning a numerical variable into ordered categories, with pd.cut or pd.qcut (Chapter 5).
Bootstrap
Estimating the sampling variation of a statistic by recomputing it on many resamples drawn with replacement from the data (Chapter 7).
Box plot
A chart of a distribution showing the median, the quartiles, whiskers, and points beyond 1.5 interquartile ranges as possible outliers (Chapter 6).
Broadcasting
NumPy's rule for combining arrays of different shapes by stretching dimensions of length 1 (Chapter 3).
Categorical variable
A variable whose values are categories: nominal (no order) or ordinal (ordered) (Chapter 1).
Central limit theorem
The result that the mean of many independent observations is approximately normally distributed, whatever the shape of the population (Chapter 7).
Centroid
The mean of the examples in a cluster; k-means represents each cluster by its centroid (Chapter 11).
Chained assignment
Selecting from a DataFrame twice and assigning to the result, as in df[mask]["col"] = value; it never changes the original under copy-on-write (Chapter 4).
Classification
Supervised learning in which the target is a category (Chapter 9).
Clustering
Grouping examples so that those in a group are more similar to each other than to those in other groups, without using labels (Chapter 11).
Coefficient of determination (\(R^2\))
One minus the ratio of a model's squared error to that of predicting the mean; 1 is perfect, 0 is no better than the mean (Chapter 8).
Confidence interval
A range of plausible values for a parameter, computed by a method that captures the true value in a stated share (such as 95%) of samples (Chapter 7).
Confusion matrix
A table counting true positives, false positives, false negatives and true negatives (Chapter 9).
Copy-on-write
pandas' rule (always on from version 3) that a selection behaves as an independent copy, so changing it never changes the original table (Chapter 4).
Correlation
Pearson's \(r\), a measure from \(-1\) to \(1\) of the strength of a straight-line relationship between two numerical variables (Chapter 6).
Cosine similarity
The cosine of the angle between two vectors; used to compare documents represented as TF-IDF vectors (Chapter 12).
Cross-validation
Estimating performance by training on part of the data and validating on the rest, repeatedly, so that every example is used for validation once (Chapter 10).
CSV
Comma-separated values: plain text with one record per line and fields separated by commas (Chapters 2 and 4).

D–H

Data leakage
The use, during training or evaluation, of information that would not be available when the model is used, making the evaluation falsely optimistic (Chapter 10).
Data minimisation
The principle of collecting and keeping only the personal data that a purpose needs (Chapter 13).
DataFrame
pandas' table: a set of Series (columns) sharing one index (Chapter 4).
Decision boundary
The border between the regions of feature space that a classifier assigns to different classes (Chapter 9).
Decision tree
A model that predicts by a sequence of yes/no questions about the features, grown by choosing splits that make groups purer (Chapter 9).
Dendrogram
A tree diagram showing the order and distances at which hierarchical clustering merged clusters (Chapter 11).
Distribution
The pattern of how often each value of a variable occurs; for a random variable, the probabilities of its values (Chapters 6 and 7).
dtype
The type of the values in a NumPy array or a pandas column, such as int64, float64 or str (Chapters 3 and 4).
Expectation
The long-run average value of a random variable, also called its mean (Chapter 7).
Exploratory data analysis (EDA)
Getting to know a dataset through summaries and charts before modelling it (Chapter 6).
F1 score
The harmonic mean of precision and recall (Chapter 9).
Feature
An input variable used by a model to make predictions (Chapter 8).
Feature importance
A measure of how much a model relies on each feature; see permutation importance (Chapters 9 and 14).
Gini impurity
\(1 - \sum_c p_c^2\), a measure of how mixed the classes in a group are; decision trees choose splits that reduce it (Chapter 9).
Gradient boosting
An ensemble method that adds decision trees one after another, each fitted to correct the errors of those before (Chapter 14).
Grid search
Trying every combination in a grid of hyperparameter values and keeping the one with the best cross-validated score (Chapter 10).
Groupby
pandas' split-apply-combine operation: split rows into groups by a key, apply a function to each, and combine the results (Chapter 5).
Histogram
A chart of a numerical variable's distribution, with bars counting the values in each interval (bin) (Chapter 6).
Hyperparameter
A setting chosen before training, such as \(k\) in k-NN or the depth of a tree, rather than learned from the data (Chapters 8 to 10).
Hypothesis test
A procedure for judging whether data is consistent with a null hypothesis, usually summarised by a p-value (Chapter 7).

I–O

Imputation
Filling in missing values, for example with the median of a column (Chapters 5 and 14).
Index
The labels of the rows of a pandas Series or DataFrame (Chapter 4).
Inertia
The sum of squared distances from each point to its cluster's centroid, which k-means minimises (Chapter 11).
Interquartile range (IQR)
The distance between the first and third quartiles; the box of a box plot (Chapters 5 and 6).
k-anonymity
The property that every combination of quasi-identifier values in a table is shared by at least \(k\) records (Chapter 13).
k-fold cross-validation
Cross-validation with the data divided into \(k\) folds, each used once for validation (Chapter 10).
k-means
A clustering algorithm that alternates assigning points to the nearest centroid and moving centroids to the mean of their points (Chapter 11).
k-nearest neighbours (k-NN)
A classifier that predicts by a majority vote of the \(k\) closest training examples (Chapter 9).
Lag
A copy of a time series shifted so that each time point sees an earlier value; created with shift (Chapter 12).
Lasso
Linear regression with a penalty on the sum of the absolute values of the coefficients, which can set some coefficients exactly to zero (Chapter 8).
Least squares
Fitting a model by minimising the sum of squared residuals (Chapter 8).
Logistic regression
A classification model that passes a linear combination of the features through the sigmoid function to give a probability (Chapter 9).
Mean absolute error (MAE)
The average absolute difference between predictions and true values (Chapter 8).
Median
The middle value of sorted data; half the values lie below it (Chapters 1 and 6).
Missing value
A value that was not recorded, shown by pandas as NaN, NaT or <NA> (Chapters 3 and 5).
Normal distribution
The symmetric bell-shaped distribution fixed by its mean and standard deviation (Chapter 7).
Null hypothesis
In a hypothesis test, the "no effect" or "no difference" position that the test assesses (Chapter 7).
One-hot encoding
Representing a categorical variable by one 0/1 column per category (Chapter 8).
Outlier
A value far from the rest of the data, which may be genuine or an error (Chapters 5 and 6).
Overfitting
Fitting the noise of the training data as well as the pattern, so that performance on new data suffers (Chapter 8).

P–S

p-value
The probability, if the null hypothesis were true, of a result at least as extreme as the one observed (Chapter 7).
Permutation importance
The increase in a model's error when one feature's values are randomly shuffled (Chapter 14).
Permutation test
A hypothesis test that compares an observed statistic with its values after many random reshufflings of the group labels (Chapter 7).
Pipeline
A chain of preprocessing steps and a model that is fitted and used as one estimator, so that every step learns from training data only (Chapters 8 to 10).
Population
The whole set of cases about which conclusions are wanted; a sample is drawn from it (Chapter 7).
Power
The probability that a hypothesis test detects a real effect of a given size (Chapter 7).
Precision
Of the examples predicted positive, the fraction that truly are positive: \(TP/(TP + FP)\) (Chapter 9).
Principal component analysis (PCA)
A method that finds orthogonal directions of greatest variance in the data, used to reduce the number of dimensions (Chapter 11).
Pseudonymisation
Replacing an identifier with a code that cannot be traced back without separately held information, such as a secret key (Chapter 13).
Quasi-identifier
A field, such as date of birth or postal code, that does not identify a person alone but can in combination with others (Chapter 13).
Random forest
An ensemble of decision trees, each grown on a bootstrap sample with random subsets of features, whose predictions are averaged or voted (Chapter 9).
Recall
Of the truly positive examples, the fraction predicted positive: \(TP/(TP + FN)\); also called sensitivity or the true positive rate (Chapter 9).
Regression
Supervised learning in which the target is a number (Chapter 8).
Regularisation
Adding a penalty on the size of a model's coefficients to the loss, to reduce overfitting (Chapter 8).
Reproducibility
The property that the same data and code give the same results when the analysis is rerun by anyone (Chapter 13).
Resampling (time series)
Changing the frequency of a time series, for example from daily to monthly, with resample (Chapter 12).
Residual
The difference between a true value and a model's prediction, \(y - \hat{y}\) (Chapter 8).
Ridge regression
Linear regression with a penalty on the sum of squared coefficients (Chapter 8).
ROC curve
A plot of the true positive rate against the false positive rate over all classification thresholds (Chapter 10).
Sampling distribution
The distribution of a statistic over all possible samples of a given size (Chapter 7).
Seed
The starting value of a pseudo-random number generator; fixing it makes random results repeatable (Chapters 2, 3 and 13).
Series
pandas' one-dimensional labelled array: values plus an index (Chapter 4).
Sigmoid
The function \(1/(1 + e^{-z})\), which maps any number to the range 0 to 1 (Chapter 9).
Silhouette score
The average over all points of \((b - a)/\max(a, b)\), comparing distances within a point's cluster (\(a\)) and to the nearest other cluster (\(b\)) (Chapter 11).
Skew
Asymmetry of a distribution; a right-skewed distribution has a long tail of large values (Chapter 6).
Standard deviation
The square root of the variance; a measure of spread in the units of the data (Chapters 3 and 7).
Standard error
The standard deviation of a statistic's sampling distribution; for a mean, \(\sigma/\sqrt{n}\) (Chapter 7).
Standardisation
Rescaling a variable to mean 0 and standard deviation 1 by subtracting its mean and dividing by its standard deviation (Chapters 3, 8 and 9).
Stratified split
A split or set of folds that keeps the same proportion of each class in every part (Chapters 9 and 10).
Supervised learning
Learning to predict a target from examples where the target is known (Chapter 8).

T–Z

Target
The variable a model predicts, also called the label or response (Chapter 8).
Test set
Data held back and used once, at the end, to estimate how a final model performs on new data (Chapters 8 and 10).
TF-IDF
Term frequency–inverse document frequency: word counts reweighted so that words common to many documents count less (Chapter 12).
Threshold
The predicted probability above which a classifier predicts the positive class; 0.5 by default (Chapter 9).
Tidy data
A table with one variable per column and one observation per row (Chapter 1).
Time series
Measurements of a quantity taken in time order (Chapters 6 and 12).
Tokenisation
Splitting text into units, usually words, called tokens (Chapter 12).
Training set
The data used to fit a model's parameters (Chapter 8).
Type I error
Rejecting a true null hypothesis: a false positive (Chapter 7).
Type II error
Failing to reject a false null hypothesis: a missed effect (Chapter 7).
Underfitting
Using a model too simple to capture the pattern in the data, so that it does poorly on both training and new data (Chapter 8).
Unsupervised learning
Finding structure in data without a target, as in clustering and dimensionality reduction (Chapter 11).
Validation set
Data used to make choices about a model, such as its hyperparameters, separate from the test set (Chapter 10).
Variance (of a random variable)
The average squared distance of a random variable from its mean (Chapter 7).
Variance (in modelling)
The error that comes from a model's sensitivity to the particular training sample; high in overfitted models (Chapter 8).
Vectorisation
Writing operations on whole arrays or columns at once instead of looping over elements (Chapter 3).
z-score
The number of standard deviations by which a value lies above or below the mean: \((x - \mu)/\sigma\) (Chapter 7).