Glossary¶
The terms of the book, with the chapter where each is explained
A–C¶
- Accuracy
- The fraction of a classifier's predictions that are correct. Misleading when one class is much more common than the other (Chapter 9).
- Agglomerative clustering
- Hierarchical clustering that starts with every example as its own cluster and repeatedly merges the two closest clusters, according to a linkage rule (Chapter 11).
- Aggregation
- Combining many values into one, such as a sum, mean, count or maximum (Chapters 2, 3 and 5).
- Alignment
- pandas' matching of values by index label, rather than by position, when two Series or DataFrames are combined (Chapter 4).
- AUC
- Area under the ROC curve: the probability that a classifier scores a randomly chosen positive example above a randomly chosen negative one; 0.5 is random guessing, 1 is perfect (Chapter 10).
- Axis
- A dimension of an array. In an aggregation, the axis named is the one that disappears:
axis=0gives one result per column (Chapter 3). In matplotlib, anAxesobject is one plot within a figure (Chapter 6). - Bag of words
- A representation of a document by the counts of the words it contains, ignoring their order (Chapter 12).
- Baseline
- A trivial model, such as predicting the mean or the most common class, against which real models are compared (Chapters 8 and 9).
- Bias (statistical)
- A systematic difference between an estimate and the true value. In modelling, the error that comes from a model too simple to capture the pattern (underfitting) (Chapters 7 and 8).
- Bias (in data)
- Systematic distortion in how data was sampled, measured or labelled, which a model trained on it can reproduce (Chapter 13).
- Binning
- Turning a numerical variable into ordered categories, with
pd.cutorpd.qcut(Chapter 5). - Bootstrap
- Estimating the sampling variation of a statistic by recomputing it on many resamples drawn with replacement from the data (Chapter 7).
- Box plot
- A chart of a distribution showing the median, the quartiles, whiskers, and points beyond 1.5 interquartile ranges as possible outliers (Chapter 6).
- Broadcasting
- NumPy's rule for combining arrays of different shapes by stretching dimensions of length 1 (Chapter 3).
- Categorical variable
- A variable whose values are categories: nominal (no order) or ordinal (ordered) (Chapter 1).
- Central limit theorem
- The result that the mean of many independent observations is approximately normally distributed, whatever the shape of the population (Chapter 7).
- Centroid
- The mean of the examples in a cluster; k-means represents each cluster by its centroid (Chapter 11).
- Chained assignment
- Selecting from a DataFrame twice and assigning to the result, as in
df[mask]["col"] = value; it never changes the original under copy-on-write (Chapter 4). - Classification
- Supervised learning in which the target is a category (Chapter 9).
- Clustering
- Grouping examples so that those in a group are more similar to each other than to those in other groups, without using labels (Chapter 11).
- Coefficient of determination (\(R^2\))
- One minus the ratio of a model's squared error to that of predicting the mean; 1 is perfect, 0 is no better than the mean (Chapter 8).
- Confidence interval
- A range of plausible values for a parameter, computed by a method that captures the true value in a stated share (such as 95%) of samples (Chapter 7).
- Confusion matrix
- A table counting true positives, false positives, false negatives and true negatives (Chapter 9).
- Copy-on-write
- pandas' rule (always on from version 3) that a selection behaves as an independent copy, so changing it never changes the original table (Chapter 4).
- Correlation
- Pearson's \(r\), a measure from \(-1\) to \(1\) of the strength of a straight-line relationship between two numerical variables (Chapter 6).
- Cosine similarity
- The cosine of the angle between two vectors; used to compare documents represented as TF-IDF vectors (Chapter 12).
- Cross-validation
- Estimating performance by training on part of the data and validating on the rest, repeatedly, so that every example is used for validation once (Chapter 10).
- CSV
- Comma-separated values: plain text with one record per line and fields separated by commas (Chapters 2 and 4).
D–H¶
- Data leakage
- The use, during training or evaluation, of information that would not be available when the model is used, making the evaluation falsely optimistic (Chapter 10).
- Data minimisation
- The principle of collecting and keeping only the personal data that a purpose needs (Chapter 13).
- DataFrame
- pandas' table: a set of Series (columns) sharing one index (Chapter 4).
- Decision boundary
- The border between the regions of feature space that a classifier assigns to different classes (Chapter 9).
- Decision tree
- A model that predicts by a sequence of yes/no questions about the features, grown by choosing splits that make groups purer (Chapter 9).
- Dendrogram
- A tree diagram showing the order and distances at which hierarchical clustering merged clusters (Chapter 11).
- Distribution
- The pattern of how often each value of a variable occurs; for a random variable, the probabilities of its values (Chapters 6 and 7).
- dtype
- The type of the values in a NumPy array or a pandas column, such as
int64,float64orstr(Chapters 3 and 4). - Expectation
- The long-run average value of a random variable, also called its mean (Chapter 7).
- Exploratory data analysis (EDA)
- Getting to know a dataset through summaries and charts before modelling it (Chapter 6).
- F1 score
- The harmonic mean of precision and recall (Chapter 9).
- Feature
- An input variable used by a model to make predictions (Chapter 8).
- Feature importance
- A measure of how much a model relies on each feature; see permutation importance (Chapters 9 and 14).
- Gini impurity
- \(1 - \sum_c p_c^2\), a measure of how mixed the classes in a group are; decision trees choose splits that reduce it (Chapter 9).
- Gradient boosting
- An ensemble method that adds decision trees one after another, each fitted to correct the errors of those before (Chapter 14).
- Grid search
- Trying every combination in a grid of hyperparameter values and keeping the one with the best cross-validated score (Chapter 10).
- Groupby
- pandas' split-apply-combine operation: split rows into groups by a key, apply a function to each, and combine the results (Chapter 5).
- Histogram
- A chart of a numerical variable's distribution, with bars counting the values in each interval (bin) (Chapter 6).
- Hyperparameter
- A setting chosen before training, such as \(k\) in k-NN or the depth of a tree, rather than learned from the data (Chapters 8 to 10).
- Hypothesis test
- A procedure for judging whether data is consistent with a null hypothesis, usually summarised by a p-value (Chapter 7).
I–O¶
- Imputation
- Filling in missing values, for example with the median of a column (Chapters 5 and 14).
- Index
- The labels of the rows of a pandas Series or DataFrame (Chapter 4).
- Inertia
- The sum of squared distances from each point to its cluster's centroid, which k-means minimises (Chapter 11).
- Interquartile range (IQR)
- The distance between the first and third quartiles; the box of a box plot (Chapters 5 and 6).
- k-anonymity
- The property that every combination of quasi-identifier values in a table is shared by at least \(k\) records (Chapter 13).
- k-fold cross-validation
- Cross-validation with the data divided into \(k\) folds, each used once for validation (Chapter 10).
- k-means
- A clustering algorithm that alternates assigning points to the nearest centroid and moving centroids to the mean of their points (Chapter 11).
- k-nearest neighbours (k-NN)
- A classifier that predicts by a majority vote of the \(k\) closest training examples (Chapter 9).
- Lag
- A copy of a time series shifted so that each time point sees an earlier value; created with
shift(Chapter 12). - Lasso
- Linear regression with a penalty on the sum of the absolute values of the coefficients, which can set some coefficients exactly to zero (Chapter 8).
- Least squares
- Fitting a model by minimising the sum of squared residuals (Chapter 8).
- Logistic regression
- A classification model that passes a linear combination of the features through the sigmoid function to give a probability (Chapter 9).
- Mean absolute error (MAE)
- The average absolute difference between predictions and true values (Chapter 8).
- Median
- The middle value of sorted data; half the values lie below it (Chapters 1 and 6).
- Missing value
- A value that was not recorded, shown by pandas as
NaN,NaTor<NA>(Chapters 3 and 5). - Normal distribution
- The symmetric bell-shaped distribution fixed by its mean and standard deviation (Chapter 7).
- Null hypothesis
- In a hypothesis test, the "no effect" or "no difference" position that the test assesses (Chapter 7).
- One-hot encoding
- Representing a categorical variable by one 0/1 column per category (Chapter 8).
- Outlier
- A value far from the rest of the data, which may be genuine or an error (Chapters 5 and 6).
- Overfitting
- Fitting the noise of the training data as well as the pattern, so that performance on new data suffers (Chapter 8).
P–S¶
- p-value
- The probability, if the null hypothesis were true, of a result at least as extreme as the one observed (Chapter 7).
- Permutation importance
- The increase in a model's error when one feature's values are randomly shuffled (Chapter 14).
- Permutation test
- A hypothesis test that compares an observed statistic with its values after many random reshufflings of the group labels (Chapter 7).
- Pipeline
- A chain of preprocessing steps and a model that is fitted and used as one estimator, so that every step learns from training data only (Chapters 8 to 10).
- Population
- The whole set of cases about which conclusions are wanted; a sample is drawn from it (Chapter 7).
- Power
- The probability that a hypothesis test detects a real effect of a given size (Chapter 7).
- Precision
- Of the examples predicted positive, the fraction that truly are positive: \(TP/(TP + FP)\) (Chapter 9).
- Principal component analysis (PCA)
- A method that finds orthogonal directions of greatest variance in the data, used to reduce the number of dimensions (Chapter 11).
- Pseudonymisation
- Replacing an identifier with a code that cannot be traced back without separately held information, such as a secret key (Chapter 13).
- Quasi-identifier
- A field, such as date of birth or postal code, that does not identify a person alone but can in combination with others (Chapter 13).
- Random forest
- An ensemble of decision trees, each grown on a bootstrap sample with random subsets of features, whose predictions are averaged or voted (Chapter 9).
- Recall
- Of the truly positive examples, the fraction predicted positive: \(TP/(TP + FN)\); also called sensitivity or the true positive rate (Chapter 9).
- Regression
- Supervised learning in which the target is a number (Chapter 8).
- Regularisation
- Adding a penalty on the size of a model's coefficients to the loss, to reduce overfitting (Chapter 8).
- Reproducibility
- The property that the same data and code give the same results when the analysis is rerun by anyone (Chapter 13).
- Resampling (time series)
- Changing the frequency of a time series, for example from daily to monthly, with
resample(Chapter 12). - Residual
- The difference between a true value and a model's prediction, \(y - \hat{y}\) (Chapter 8).
- Ridge regression
- Linear regression with a penalty on the sum of squared coefficients (Chapter 8).
- ROC curve
- A plot of the true positive rate against the false positive rate over all classification thresholds (Chapter 10).
- Sampling distribution
- The distribution of a statistic over all possible samples of a given size (Chapter 7).
- Seed
- The starting value of a pseudo-random number generator; fixing it makes random results repeatable (Chapters 2, 3 and 13).
- Series
- pandas' one-dimensional labelled array: values plus an index (Chapter 4).
- Sigmoid
- The function \(1/(1 + e^{-z})\), which maps any number to the range 0 to 1 (Chapter 9).
- Silhouette score
- The average over all points of \((b - a)/\max(a, b)\), comparing distances within a point's cluster (\(a\)) and to the nearest other cluster (\(b\)) (Chapter 11).
- Skew
- Asymmetry of a distribution; a right-skewed distribution has a long tail of large values (Chapter 6).
- Standard deviation
- The square root of the variance; a measure of spread in the units of the data (Chapters 3 and 7).
- Standard error
- The standard deviation of a statistic's sampling distribution; for a mean, \(\sigma/\sqrt{n}\) (Chapter 7).
- Standardisation
- Rescaling a variable to mean 0 and standard deviation 1 by subtracting its mean and dividing by its standard deviation (Chapters 3, 8 and 9).
- Stratified split
- A split or set of folds that keeps the same proportion of each class in every part (Chapters 9 and 10).
- Supervised learning
- Learning to predict a target from examples where the target is known (Chapter 8).
T–Z¶
- Target
- The variable a model predicts, also called the label or response (Chapter 8).
- Test set
- Data held back and used once, at the end, to estimate how a final model performs on new data (Chapters 8 and 10).
- TF-IDF
- Term frequency–inverse document frequency: word counts reweighted so that words common to many documents count less (Chapter 12).
- Threshold
- The predicted probability above which a classifier predicts the positive class; 0.5 by default (Chapter 9).
- Tidy data
- A table with one variable per column and one observation per row (Chapter 1).
- Time series
- Measurements of a quantity taken in time order (Chapters 6 and 12).
- Tokenisation
- Splitting text into units, usually words, called tokens (Chapter 12).
- Training set
- The data used to fit a model's parameters (Chapter 8).
- Type I error
- Rejecting a true null hypothesis: a false positive (Chapter 7).
- Type II error
- Failing to reject a false null hypothesis: a missed effect (Chapter 7).
- Underfitting
- Using a model too simple to capture the pattern in the data, so that it does poorly on both training and new data (Chapter 8).
- Unsupervised learning
- Finding structure in data without a target, as in clustering and dimensionality reduction (Chapter 11).
- Validation set
- Data used to make choices about a model, such as its hyperparameters, separate from the test set (Chapter 10).
- Variance (of a random variable)
- The average squared distance of a random variable from its mean (Chapter 7).
- Variance (in modelling)
- The error that comes from a model's sensitivity to the particular training sample; high in overfitted models (Chapter 8).
- Vectorisation
- Writing operations on whole arrays or columns at once instead of looping over elements (Chapter 3).
- z-score
- The number of standard deviations by which a value lies above or below the mean: \((x - \mu)/\sigma\) (Chapter 7).