Skip to content

Glossary of Terms

A

Accuracy
The share of all predictions that are correct; misleading on imbalanced data, where predicting the majority class can score highly.
Activation function
A non-linear function such as ReLU, GELU or sigmoid applied inside a neural network so it can learn complex patterns.
Adam / AdamW
Popular optimisers that adapt each parameter's learning rate using running averages of gradients; AdamW decouples weight decay from the update.
AI agent
A system in which a model plans, calls tools or APIs and acts over several steps to complete a goal, rather than giving one answer.
Attention
A mechanism that lets each element of a sequence weigh information from every other element; the core of the transformer.

B

Backpropagation
The algorithm that computes gradients of the loss with respect to every weight by applying the chain rule backwards through a network.
Bagging
Bootstrap aggregating: training many models on random resamples of the data and averaging them to reduce variance, as in random forests.
Bias-variance trade-off
The balance between error from overly simple assumptions (bias) and error from sensitivity to the training sample (variance).
Boosting
An ensemble method that adds weak models one after another, each correcting the errors of the ones before, as in XGBoost and LightGBM.

C

Clustering
Unsupervised grouping of similar data points, for example with k-means, DBSCAN or Gaussian mixture models.
Concept drift
A change over time in the relationship between inputs and the correct output, which makes a trained model less accurate.
Confusion matrix
A table counting true positives, false positives, true negatives and false negatives for a classifier.
Convolutional neural network (CNN)
A neural network that slides learned filters over grid data such as images to detect local patterns.
Cross-entropy
The standard loss for classification, measuring how far predicted probabilities are from the true labels.
Cross-validation
Repeatedly splitting data into training and validation folds to get a more reliable estimate of model performance.

D

Data drift
A change in the distribution of a model's input data compared with the data it was trained on.
Data leakage
Information in training that would not be available at prediction time, producing unrealistically good evaluation scores.
Deep learning
Machine learning with many-layered neural networks that learn their own features from raw data.
Differential privacy
A mathematical guarantee, achieved by adding calibrated noise, that an output reveals almost nothing about any single individual.
Diffusion model
A generative model that learns to turn random noise into realistic data by reversing a gradual noising process.
Dimensionality reduction
Compressing many features into fewer while keeping important structure, for example with PCA, t-SNE or UMAP.
Distillation
Training a smaller 'student' model to imitate a larger 'teacher' model, keeping much of its quality at lower cost.
DPO
Direct Preference Optimization: aligning a language model to human preferences directly from preferred-versus-rejected answer pairs, without a separate reward model.
Dropout
A regularisation technique that randomly switches off neurons during training so the network does not rely on any single one.

E

Embedding
A dense vector of numbers representing an item such as a word, image or user, placed so that similar items are close together.
EU AI Act
The European Union's risk-based AI law (Regulation (EU) 2024/1689), applying in phases from 2025 to 2028.

F

F1 score
The harmonic mean of precision and recall, useful when you need a single balance of both.
Feature
An input variable a model uses to make predictions, such as age, price or a pixel value.
Feature engineering
Creating, transforming and selecting input features to make patterns easier for a model to learn.
Feature store
A system that defines, stores and serves ML features consistently for both training and real-time prediction.
Fine-tuning
Continuing to train a pretrained model on a smaller, task-specific dataset to adapt it.
Foundation model
A large model pretrained on broad data that can be adapted to many tasks, such as an LLM or a vision-language model.

G

Generalisation
A model's ability to perform well on new, unseen data rather than only on the examples it was trained on.
Generative adversarial network (GAN)
Two networks trained against each other — a generator that creates samples and a discriminator that tries to spot fakes.
GPU
Graphics processing unit: a massively parallel processor with fast memory that accelerates neural-network training and inference.
Gradient descent
An optimisation method that repeatedly adjusts parameters in the direction that most reduces the loss.

H

Hallucination
When a generative model produces fluent but false or unsupported content.
Hyperparameter
A setting chosen before training, such as learning rate, tree depth or number of layers, rather than learned from data.

I

Inference
Using a trained model to make predictions or generate outputs on new data.

K

KV cache
Stored attention keys and values from earlier tokens that speed up LLM text generation at the cost of memory.

L

Label
The correct answer attached to a training example in supervised learning, such as 'spam' or a house price.
Large language model (LLM)
A transformer trained on vast amounts of text to predict the next token, which can then follow instructions and generate text.
Learning rate
The step size used when updating parameters during gradient descent; too high diverges, too low learns slowly.
Logistic regression
A linear model that outputs a probability for a binary outcome using the sigmoid function.
LoRA
Low-Rank Adaptation: fine-tuning large models by training small low-rank matrices added to frozen weights.
Loss function
A formula measuring how wrong a model's predictions are; training aims to minimise it.

M

MLOps
Practices and tools for reliably deploying, monitoring and improving ML models in production.
Model card
A short document describing a model's intended use, training data, evaluation results, limitations and ethical considerations.
Model registry
A catalogue of model versions with metrics, lineage and approval status, from which production systems load approved models.

N

Neural network
A model made of layers of connected units whose weighted connections are learned from data.

O

Overfitting
When a model learns noise and quirks of its training data and performs poorly on new data.

P

Parameter
A value learned during training, such as a weight or bias in a neural network.
Precision
Of all the items a model flagged as positive, the share that really were positive.
Prompt injection
An attack in which text the model reads overrides its instructions, either typed directly or hidden in documents and web pages.

Q

Quantisation
Storing and computing a model's numbers at lower precision (such as 8 or 4 bits) to save memory and speed up inference.

R

Recall
Of all the truly positive items, the share the model correctly found.
Regularisation
Techniques such as L1/L2 penalties, dropout and early stopping that discourage overly complex models and reduce overfitting.
Reinforcement learning
Learning by trial and error: an agent takes actions in an environment and improves its policy from rewards.
Retrieval-augmented generation (RAG)
Giving an LLM relevant documents retrieved at query time so its answers are grounded in up-to-date or private information.
RLHF
Reinforcement learning from human feedback: fine-tuning a model using a reward model trained on human preference judgements.
ROC-AUC
The area under the curve of true-positive rate against false-positive rate across all thresholds; 0.5 is random, 1.0 perfect.

S

Scaling laws
Empirical relationships showing how model loss improves predictably with more parameters, data and compute.
Self-supervised learning
Learning from unlabelled data by creating labels from the data itself, such as predicting the next word or a masked patch.
Supervised learning
Learning a mapping from inputs to known labels using labelled examples.

T

Test set
Data held back until the very end to give an unbiased estimate of final model performance.
Token
A chunk of text — a word, part of a word or symbol — that a language model reads and generates.
Transfer learning
Reusing knowledge from a model trained on one task or dataset as the starting point for another.
Transformer
A neural-network architecture built on self-attention that underpins modern language, vision and multimodal models.

U

Underfitting
When a model is too simple to capture the real patterns, performing poorly even on training data.
Unsupervised learning
Finding structure such as clusters or lower-dimensional representations in data without labels.