Skip to content

Chapter 22: QML for NLP and Computer Vision

Natural language processing and computer vision are the two largest classical ML application domains. This chapter explores how quantum machine learning approaches — quantum language models, quantum attention mechanisms, quantum convolutional filters, and hybrid classical-quantum architectures — can be applied to text and image processing tasks. We emphasize the DisCoCat compositional framework for quantum NLP and quantum image classification with PennyLane.

Learning Objectives

By the end of this chapter, you will be able to:

  • Explain the DisCoCat categorical compositional quantum NLP framework

  • Implement a quantum text classifier using sentence diagrams

  • Apply quantum convolutional circuits to image classification

  • Build a hybrid ViT-QNN for image feature extraction

  • Discuss the quantum advantage prospects for NLP and vision tasks

22.1 Quantum NLP: DisCoCat Framework

22.1.1 Compositional Distributional Semantics

The DisCoCat framework (Coecke, Sadrzadeh, Clark, 2010; Meichanetzidis et al., 2020) (Coecke et al. 2020) maps grammatical structure to quantum circuits:

  • Nouns/entities: encoded as quantum states \(|\psi_\text{word}\rangle\)

  • Verbs/relations: encoded as quantum gates (maps between states)

  • Sentence meaning: computed as quantum circuit expectation value

Pregroup grammar: A sentence “Alice loves Bob” has type:

\[\underbrace{\text{Alice}}_{n} \cdot \underbrace{\text{loves}}_{n^r\cdot s\cdot n^l} \cdot \underbrace{\text{Bob}}_{n}\]

The type contractions are realized as SWAP/Bell measurements between word qubits, producing the sentence meaning as a quantum state measurement.

22.1.2 Quantum Sentence Circuit

image

The sentence circuit’s output state \(|\text{sent}\rangle\) is classified: \(P(\text{true}) = |\langle 1|\text{sent}\rangle|^2\) for a binary NLP task.

lambeq: The open-source toolkit for QNLP (Kartsaklis et al., 2021) converts English sentences to quantum circuits automatically.

22.2 Quantum Text Classification

22.2.1 Implementation with lambeq

# Install: pip install lambeq
# Note: requires lambeq >= 0.4.0

try:
    from lambeq import BobcatParser, AtomicType, IQPAnsatz
    from lambeq import TketModel, QuantumTrainer
    import discopy
    HAS_LAMBEQ = True
except ImportError:
    HAS_LAMBEQ = False
    print("lambeq not installed. Showing conceptual implementation.")

if HAS_LAMBEQ:
    # Parse sentences to string diagrams
    parser = BobcatParser(verbose='text')
    train_sentences = [
        "Alice likes Bob",
        "Bob loves Alice",
        "Alice hates computers",
        "Bob dislikes algorithms",
    ]
    train_labels = [1, 1, 0, 0]  # positive/negative sentiment

    diagrams = parser.sentences2diagrams(train_sentences)
    print(f"Parsed {len(diagrams)} sentences")

    # Convert to quantum circuits (IQP ansatz)
    N = AtomicType.NOUN
    S = AtomicType.SENTENCE
    ansatz = IQPAnsatz({N: 1, S: 1}, n_layers=1)
    circuits = [ansatz(d) for d in diagrams]
    print(f"Circuit dimensions: {[c.dom for c in circuits]}")

else:
    # Manual toy QNLP implementation (no lambeq dependency)
    import pennylane as qml
    import numpy as np
    import torch

    print("Running toy QNLP classifier (without lambeq)...")

    # Toy sentence encoding: encode (subject, verb, object) triplet
    n_qubits = 4  # 2 for subject, 1 for verb, 1 for object
    dev = qml.device("default.qubit", wires=n_qubits)

    @qml.qnode(dev, interface="torch", diff_method="parameter-shift")
    def sentence_circuit(word_params, verb_weight):
        """
        Toy sentence circuit.
        word_params: [subj_theta, obj_theta] (noun encodings)
        verb_weight: scalar (verb parameter)
        """
        # Encode subject (qubits 0,1)
        qml.RY(word_params[0], wires=0)
        qml.RY(word_params[0] * 0.5, wires=1)
        # Encode object (qubit 2)
        qml.RY(word_params[1], wires=2)
        # Verb interaction (entangle subject and object)
        qml.CRY(verb_weight, wires=[0, 2])
        qml.CRY(verb_weight * 0.7, wires=[1, 2])
        # Sentence meaning
        qml.RY(verb_weight, wires=3)
        qml.CNOT(wires=[2, 3])
        return qml.expval(qml.PauliZ(3))

    # Training data: (subject_angle, object_angle, verb_param, label)
    # "like"/"love" = positive, "hate"/"dislike" = negative
    train_data = [
        (0.5, 1.2, 0.3, 1),   # Alice likes Bob
        (0.8, 0.4, 0.2, 1),   # Bob loves Alice
        (0.5, 2.1, -1.2, 0),  # Alice hates computers
        (0.8, 1.8, -0.9, 0),  # Bob dislikes algorithms
    ]

    verb_param = torch.tensor(0.5, requires_grad=True)
    optimizer = torch.optim.Adam([verb_param], lr=0.1)

    print("Training quantum NLP classifier...")
    for epoch in range(80):
        total_loss = 0.0
        optimizer.zero_grad()
        for s_angle, o_angle, _, label in train_data:
            word_p = torch.tensor([s_angle, o_angle])
            pred = (sentence_circuit(word_p, verb_param) + 1) / 2
            loss = (pred - label)**2
            total_loss += loss
        total_loss.backward()
        optimizer.step()
        if (epoch + 1)
            print(f"Epoch {epoch+1}: loss={total_loss.item():.4f}, "
                  f"verb_param={verb_param.item():.3f}")

    # Evaluate
    correct = 0
    for s_angle, o_angle, _, label in train_data:
        word_p = torch.tensor([s_angle, o_angle])
        with torch.no_grad():
            pred = (sentence_circuit(word_p, verb_param) + 1) / 2
        predicted = 1 if pred.item() > 0.5 else 0
        if predicted == label:
            correct += 1
    print(f"Training accuracy: {correct}/{len(train_data)}")

22.3 Quantum Computer Vision

Hybrid CNN-QNN architecture for image classification. A classical MiniCNN extracts a low-dimensional feature vector; a linear layer maps it to n_q rotation angles for the quantum layer; two VQC…
Figure 22.1. Hybrid CNN-QNN architecture for image classification. A classical MiniCNN extracts a low-dimensional feature vector; a linear layer maps it to \(n_q\) rotation angles for the quantum layer; two VQC layers process the state; Pauli-\(Z\) measurements feed a final classical softmax classifier.
Left: Test accuracy curves for a hybrid CNN-QNN, classical CNN, and pure VQC on the binary MNIST (0 vs 1) task. Right: Confusion matrix for the hybrid CNN-QNN on a 4-class MNIST subset (\{0,1,2,3\}).
Figure 22.2. Left: Test accuracy curves for a hybrid CNN-QNN, classical CNN, and pure VQC on the binary MNIST (0 vs 1) task. Right: Confusion matrix for the hybrid CNN-QNN on a 4-class MNIST subset (\(\{0,1,2,3\}\)). Figure drawn for this book

22.3.1 Quantum Image Encoding

Flexible Representation of Quantum Images (FRQI):

\[|I\rangle = \frac{1}{2^n}\sum_{i=0}^{2^{2n}-1}(\cos\theta_i|0\rangle + \sin\theta_i|1\rangle)|i\rangle\tag{22.1}\]

Encodes a \(2^n\times 2^n\) image: \(2n\) qubits for position, 1 qubit for pixel intensity. Total: \(2n+1\) qubits for \(N=2^{2n}\) pixels.

Angle encoding (practical NISQ): Flatten and normalize image to \(n\) pixel values \(\in[0,\pi]\), encode directly as rotation angles.

22.3.2 Quantum CNN for Image Classification

Apply QCNN (Chapter 11) to image features:

  1. PCA/downsampling: reduce image to \(n\) features

  2. Encode features as qubit rotations

  3. QCNN layers: shared 2-qubit convolution + pooling

  4. Readout + classical softmax

import pennylane as qml
import numpy as np
import torch
import torch.nn as nn
from torchvision import datasets, transforms
from torch.utils.data import DataLoader, Subset

# -- Setup -----------------------------------------------------
n_qubits = 8
n_layers = 2
dev = qml.device("default.qubit", wires=n_qubits)

# Load MNIST (digits 0 and 1 only for binary classification)
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,))
])
full_train = datasets.MNIST('./data', train=True,
                             download=True, transform=transform)
# Select digits 0 and 1
idx = [i for i in range(len(full_train))
       if full_train.targets[i] in [0, 1]][:200]
subset = Subset(full_train, idx)
loader = DataLoader(subset, batch_size=16, shuffle=True)

# -- Classical feature extractor (CNN) ------------------------
class MiniCNN(nn.Module):
    def __init__(self, out_dim=n_qubits):
        super().__init__()
        self.cnn = nn.Sequential(
            nn.Conv2d(1, 8, 3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(2),       # 14x14
            nn.Conv2d(8, 16, 3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(2),       # 7x7
            nn.Flatten(),
            nn.Linear(16*7*7, 32),
            nn.ReLU(),
            nn.Linear(32, out_dim),
            nn.Tanh()              # scale to [-1, 1]
        )
    def forward(self, x):
        return self.cnn(x) * np.pi  # scale to [-pi, pi]

# -- Quantum classifier layer ----------------------------------
@qml.qnode(dev, interface="torch", diff_method="parameter-shift")
def quantum_layer(inputs, weights):
    qml.AngleEmbedding(inputs, wires=range(n_qubits), rotation='Y')
    qml.StronglyEntanglingLayers(weights, wires=range(n_qubits))
    return qml.expval(qml.PauliZ(0))

weight_shape = (n_layers, n_qubits, 3)

class HybridImageClassifier(nn.Module):
    def __init__(self):
        super().__init__()
        self.cnn = MiniCNN(out_dim=n_qubits)
        self.q_layer = qml.qnn.TorchLayer(
            quantum_layer,
            {"weights": weight_shape}
        )
        self.fc = nn.Linear(1, 2)  # 2-class output

    def forward(self, x):
        features = self.cnn(x)          # (batch, n_qubits)
        q_out = self.q_layer(features)  # (batch,) expval
        return self.fc(q_out.unsqueeze(-1))

# -- Training -------------------------------------------------
model = HybridImageClassifier()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)
criterion = nn.CrossEntropyLoss()

print("Training hybrid CNN-QNN image classifier...")
for epoch in range(10):
    total_loss, correct, total = 0.0, 0, 0
    for images, labels in loader:
        optimizer.zero_grad()
        outputs = model(images)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()
        total_loss += loss.item()
        preds = outputs.argmax(dim=1)
        correct += (preds == labels).sum().item()
        total += len(labels)
    print(f"Epoch {epoch+1:2d}: Loss={total_loss:.3f}, "
          f"Acc={correct/total:.3f}")

22.4 Quantum Attention Mechanisms

Classical attention (Transformer): \(\text{Attention}(Q,K,V) = \text{softmax}(QK^\top/\sqrt{d_k})V\).

Quantum attention: Replace the dot product with quantum inner products (SWAP test fidelity):

\[a_{ij} = \frac{|\langle\phi(q_i)|\phi(k_j)\rangle|^2}{\sum_{j'}|\langle\phi(q_i)|\phi(k_{j'})\rangle|^2}\tag{22.2}\]

This is computed by running \(O(T^2)\) SWAP tests for a sequence of length \(T\).

Note

Quantum attention gives an \(O(d)\) speedup in the inner product computation when query/key dimensions \(d\) are exponentially large relative to qubit count \(n = \log_2 d\). For NISQ where \(n=10\)–\(30\), this requires \(d=1024\)–\(10^9\) dimensional embeddings, potentially useful for very large language models.

22.5 Chapter Summary

Chapter Summary

  • QNLP (DisCoCat): Compositional quantum semantics maps sentence grammar to quantum circuits. Words = quantum states, relations = quantum gates. Implemented in the lambeq toolkit.

  • Quantum text classification: Sentence circuits with shared word parameters, trained via gradient descent on NLP tasks.

  • Quantum vision: FRQI for image encoding; hybrid CNN+QNN for MNIST classification; QCNN with parameter sharing for translation invariance.

  • Quantum attention: SWAP-test inner products replace scaled dot-product attention. Potential exponential speedup for very high-dimensional embeddings.

  • Practical status: Hybrid CNN+QNN and QNLP prototypes work on NISQ; competitive with classical on small datasets. Large-scale advantage requires fault-tolerant hardware.

Exercises

Exercise . In the DisCoCat framework, explain why verbs are represented as maps (linear operators) rather than states. How does this connect to category theory?

Exercise . For a hybrid CNN-QNN classifier with 8 qubits and 3 strongly entangling layers: count the total parameters. Compare with a classical 2-layer MLP with 8 hidden units. Which has more parameters?

Exercise . Extend the hybrid image classifier to 4-class MNIST (digits 0-3). Use 4 PauliZ measurements (\(\langle Z_0\rangle, \langle Z_1\rangle, \langle Z_2\rangle, \langle Z_3\rangle\)) as the quantum layer output and train with cross-entropy loss.

Further Reading

  • Coecke et al. (2020) (Coecke et al. 2020) — DisCoCat for quantum NLP

  • Kartsaklis et al. (2021) — lambeq: a QNLP toolkit

  • Henderson et al. (2020) — quantum graph transformer

  • Cong et al. (2019) — quantum convolutional neural networks

  • Mari et al. (2020) (Mari et al. 2020) — quantum transfer learning