Skip to content

Lesson 3: Linear Algebra II: Inner Products, Norms and Hilbert Spaces

3.1 Learning Outcomes

By the end of this hour a student will be able to:

  1. Compute the inner product \(\langle\phi|\psi\rangle\) of two states and interpret it physically.
  2. State the axioms of an inner product space and the definition of a Hilbert space.
  3. Test vectors for orthogonality and sets for orthonormality.
  4. Apply the Gram–Schmidt procedure to build an orthonormal basis.
  5. Use the completeness (resolution of identity) relation and compute projections.
  6. State and apply the Cauchy–Schwarz inequality and understand fidelity/overlap.

3.2 The Inner Product

Definition (Inner product on \(\mathbb{C}^n\)). For \(|\psi\rangle = (\psi_1,\dots,\psi_n)^T\) and \(|\phi\rangle = (\phi_1,\dots,\phi_n)^T\),

\[ \langle\phi|\psi\rangle \;=\; \sum_{j=1}^{n} \phi_j^{*}\,\psi_j \;=\; \begin{pmatrix}\phi_1^* & \cdots & \phi_n^*\end{pmatrix} \begin{pmatrix}\psi_1\\ \vdots \\ \psi_n\end{pmatrix}. \]

In Dirac notation this is literally the bra \(\langle\phi|\) multiplied by the ket \(|\psi\rangle\) — a \(1\times n\) row times an \(n \times 1\) column, giving a \(1\times1\) matrix, i.e. a scalar. The notation is designed so that the "bra-c-ket" closes up.

Convention. Physicists conjugate the first (left) argument; many mathematics texts conjugate the second. This course uses the physics convention throughout, which is also what Qiskit, Cirq and PennyLane implement.

Axioms of an inner product space

For all \(|\psi\rangle,|\phi\rangle,|\chi\rangle \in V\) and \(a,b \in \mathbb{C}\):

Axiom Statement
I1 Conjugate symmetry \(\langle\phi\rvert\psi\rangle = \langle\psi\rvert\phi\rangle^{*}\)
I2 Linearity in the second slot \(\langle\phi\rvert(a\psi + b\chi)\rangle = a\langle\phi\rvert\psi\rangle + b\langle\phi\rvert\chi\rangle\)
I3 Conjugate-linearity in the first slot \(\langle (a\phi + b\chi)\rvert\psi\rangle = a^{*}\langle\phi\rvert\psi\rangle + b^{*}\langle\chi\rvert\psi\rangle\)
I4 Positive definiteness \(\langle\psi\rvert\psi\rangle \ge 0\), with equality iff \(\lvert\psi\rangle = 0\)

Note that I3 follows from I1 and I2. The mixed behaviour (linear on the right, conjugate-linear on the left) is called sesquilinearity — "one-and-a-half linear".

Why \(\langle\psi|\psi\rangle\) is real and non-negative:

\[ \langle\psi|\psi\rangle = \sum_j \psi_j^*\psi_j = \sum_j |\psi_j|^2 \ge 0 . \]

Norm

Definition (Norm). \(\;\big\||\psi\rangle\big\| = \sqrt{\langle\psi|\psi\rangle}\).

A state is normalised (a legal physical state) when \(\langle\psi|\psi\rangle = 1\). Given any non-zero \(|\psi\rangle\), the normalised version is \(|\psi\rangle / \big\||\psi\rangle\big\|\).

3.3 Hilbert Spaces

Definition (Hilbert space). A Hilbert space \(\mathcal{H}\) is a complex vector space equipped with an inner product that is complete with respect to the induced norm (every Cauchy sequence converges in \(\mathcal{H}\)).

In this course all spaces are finite-dimensional, and every finite-dimensional inner product space over \(\mathbb{C}\) is automatically complete. So for us:

\[ \text{Hilbert space} \;=\; \mathbb{C}^{N} \text{ with the standard inner product}, \qquad N = 2^n. \]

Completeness matters only in infinite dimensions (e.g. a particle's position wavefunction in \(L^2(\mathbb{R})\)). Students should know the word and why it is harmless here.

3.4 Orthogonality and Orthonormality

Definition (Orthogonal). \(|\phi\rangle \perp |\psi\rangle\) iff \(\langle\phi|\psi\rangle = 0\).

Definition (Orthonormal set). \(\{|e_1\rangle,\dots,|e_k\rangle\}\) is orthonormal iff

\[ \langle e_i | e_j \rangle = \delta_{ij} = \begin{cases} 1 & i = j \\ 0 & i \ne j \end{cases} \]

where \(\delta_{ij}\) is the Kronecker delta.

Theorem. An orthonormal set is linearly independent.

Proof. Suppose \(\sum_j c_j |e_j\rangle = 0\). Apply \(\langle e_k|\) to both sides:

\[ \sum_j c_j \langle e_k|e_j\rangle = \sum_j c_j \delta_{kj} = c_k = 0 . \]

This holds for every \(k\), so all coefficients vanish. \(\blacksquare\)

Physical meaning of orthogonality. Two states are perfectly distinguishable by a single measurement if and only if they are orthogonal. If \(\langle\phi|\psi\rangle \ne 0\) then no measurement can tell them apart with certainty.

Expansion coefficients come from inner products

If \(\{|e_j\rangle\}\) is an orthonormal basis and \(|\psi\rangle = \sum_j c_j |e_j\rangle\), then

\[ \boxed{\,c_j = \langle e_j | \psi \rangle\,} \]

Proof. \(\langle e_k|\psi\rangle = \sum_j c_j\langle e_k|e_j\rangle = \sum_j c_j\delta_{kj} = c_k\). \(\blacksquare\)

This is the workhorse identity of the entire subject. For a qubit, \(\alpha = \langle 0|\psi\rangle\) and \(\beta = \langle 1|\psi\rangle\).

Outer products, projectors and completeness

Definition (Outer product). \(|\phi\rangle\langle\psi|\) is the \(n\times n\) matrix obtained by multiplying a column by a row:

\[ |\phi\rangle\langle\psi| = \begin{pmatrix}\phi_1\\ \vdots\\ \phi_n\end{pmatrix} \begin{pmatrix}\psi_1^* & \cdots & \psi_n^*\end{pmatrix} = \begin{pmatrix} \phi_1\psi_1^* & \cdots & \phi_1\psi_n^* \\ \vdots & \ddots & \vdots \\ \phi_n\psi_1^* & \cdots & \phi_n\psi_n^*\end{pmatrix}. \]

Definition (Projector). For a normalised \(|\psi\rangle\), the operator \(P_\psi = |\psi\rangle\langle\psi|\) projects any vector onto the line spanned by \(|\psi\rangle\):

\[ P_\psi |\chi\rangle = |\psi\rangle\langle\psi|\chi\rangle = \big(\langle\psi|\chi\rangle\big)\,|\psi\rangle. \]

Projectors satisfy \(P^2 = P\) (idempotent) and \(P^\dagger = P\) (Hermitian).

Completeness relation (resolution of the identity). For any orthonormal basis \(\{|e_j\rangle\}_{j=1}^{n}\) of \(\mathcal{H}\):

\[ \boxed{\ \sum_{j=1}^{n} |e_j\rangle\langle e_j| = I\ } \]

Proof. Apply the left side to an arbitrary \(|\psi\rangle\): \(\sum_j |e_j\rangle\langle e_j|\psi\rangle = \sum_j c_j|e_j\rangle = |\psi\rangle\). Since it acts as the identity on every vector, it is the identity. \(\blacksquare\)

For a single qubit: \(|0\rangle\langle 0| + |1\rangle\langle 1| = I\). Explicitly,

\[ \begin{pmatrix}1&0\\0&0\end{pmatrix} + \begin{pmatrix}0&0\\0&1\end{pmatrix} = \begin{pmatrix}1&0\\0&1\end{pmatrix}. \]

This identity is used constantly to "insert a complete set of states" in the middle of an expression.

3.5 Cauchy–Schwarz, Overlap and Fidelity

Theorem (Cauchy–Schwarz inequality).

\[ |\langle\phi|\psi\rangle|^2 \le \langle\phi|\phi\rangle\,\langle\psi|\psi\rangle, \]

with equality iff \(|\phi\rangle\) and \(|\psi\rangle\) are proportional.

Proof sketch. For any \(\lambda \in \mathbb{C}\), positivity gives \(\langle \psi - \lambda\phi | \psi - \lambda\phi\rangle \ge 0\). Expanding and choosing \(\lambda = \langle\phi|\psi\rangle / \langle\phi|\phi\rangle\) yields the result. \(\blacksquare\)

Consequence. For normalised states, \(0 \le |\langle\phi|\psi\rangle| \le 1\).

Definition (Fidelity of pure states). \(F(\phi,\psi) = |\langle\phi|\psi\rangle|^2\). It is the probability that a measurement designed to test "is the state \(|\phi\rangle\)?" answers yes when the true state is \(|\psi\rangle\).

\(\lvert\langle\phi\rvert\psi\rangle\rvert\) Interpretation
\(1\) Same state (up to a global phase) — indistinguishable
\(0\) Orthogonal — perfectly distinguishable in one shot
between Partially distinguishable; distinguishing requires many copies

3.6 Gram–Schmidt Orthonormalisation

Input: a linearly independent set \(\{|v_1\rangle,\dots,|v_k\rangle\}\). Output: an orthonormal set \(\{|e_1\rangle,\dots,|e_k\rangle\}\) with the same span.

\[ |u_1\rangle = |v_1\rangle, \qquad |e_1\rangle = \frac{|u_1\rangle}{\||u_1\rangle\|} \]
\[ |u_m\rangle = |v_m\rangle - \sum_{j=1}^{m-1} \langle e_j | v_m\rangle\,|e_j\rangle, \qquad |e_m\rangle = \frac{|u_m\rangle}{\||u_m\rangle\|} \]

The subtraction removes from \(|v_m\rangle\) every component already covered by the earlier \(|e_j\rangle\).

3.7 Solved Problems

Problem 1 — Computing an inner product

Let \(|\psi\rangle = \begin{pmatrix}1+i\\ 2\end{pmatrix}\) and \(|\phi\rangle = \begin{pmatrix}3\\ 1-i\end{pmatrix}\). Compute \(\langle\phi|\psi\rangle\) and \(\langle\psi|\phi\rangle\), and verify conjugate symmetry.

Solution.

\[ \langle\phi| = \begin{pmatrix} 3^* & (1-i)^*\end{pmatrix} = \begin{pmatrix}3 & 1+i\end{pmatrix} \]
\[ \langle\phi|\psi\rangle = 3(1+i) + (1+i)(2) = (3+3i) + (2+2i) = \mathbf{5+5i} \]

Now the other order:

\[ \langle\psi| = \begin{pmatrix}(1+i)^* & 2^*\end{pmatrix} = \begin{pmatrix}1-i & 2\end{pmatrix} \]
\[ \langle\psi|\phi\rangle = (1-i)(3) + 2(1-i) = (3-3i) + (2-2i) = \mathbf{5-5i} \]

Indeed \(\langle\psi|\phi\rangle = \overline{\langle\phi|\psi\rangle}\) ✓ (axiom I1).

Problem 2 — Orthogonality and distinguishability

Show that \(|+\rangle\) and \(|-\rangle\) are orthonormal, and compute \(|\langle 0 | + \rangle|^2\). Interpret both results.

Solution. With \(|\pm\rangle = \frac{1}{\sqrt2}(|0\rangle \pm |1\rangle)\):

\[ \langle +|-\rangle = \tfrac{1}{\sqrt2}\tfrac{1}{\sqrt2} \big(\langle 0| + \langle 1|\big)\big(|0\rangle - |1\rangle\big) \]
\[ = \tfrac12\big(\langle0|0\rangle - \langle0|1\rangle + \langle1|0\rangle - \langle1|1\rangle\big) = \tfrac12(1 - 0 + 0 - 1) = 0 \ \checkmark \]
\[ \langle +|+\rangle = \tfrac12(1 + 0 + 0 + 1) = 1 \ \checkmark \]

Now the overlap with \(|0\rangle\):

\[ \langle 0|+\rangle = \tfrac{1}{\sqrt2}\big(\langle0|0\rangle + \langle 0|1\rangle\big) = \tfrac{1}{\sqrt2}, \qquad |\langle 0|+\rangle|^2 = \tfrac12 . \]

Interpretation. \(|+\rangle\) and \(|-\rangle\) are perfectly distinguishable from each other (a measurement in the \(X\) basis separates them with certainty). But \(|+\rangle\) and \(|0\rangle\) overlap with probability \(1/2\): measuring \(|+\rangle\) in the \(Z\) basis gives \(0\) half the time, so a single \(Z\) measurement cannot distinguish them.

Problem 3 — Expansion via inner products

Given \(|\psi\rangle = \frac{1}{\sqrt{5}}\big(|0\rangle + 2i|1\rangle\big)\), find its coefficients in the \(\{|+\rangle,|-\rangle\}\) basis using \(c_j = \langle e_j|\psi\rangle\), and verify normalisation.

Solution.

\[ \langle +| = \tfrac{1}{\sqrt2}\big(\langle 0| + \langle 1|\big), \qquad \langle -| = \tfrac{1}{\sqrt2}\big(\langle 0| - \langle 1|\big) \]
\[ c_+ = \langle +|\psi\rangle = \tfrac{1}{\sqrt2}\cdot\tfrac{1}{\sqrt5}\big(1 + 2i\big) = \frac{1+2i}{\sqrt{10}} \]
\[ c_- = \langle -|\psi\rangle = \tfrac{1}{\sqrt2}\cdot\tfrac{1}{\sqrt5}\big(1 - 2i\big) = \frac{1-2i}{\sqrt{10}} \]

Check: \(|c_+|^2 = \frac{1^2+2^2}{10} = \frac{5}{10} = \frac12\) and likewise \(|c_-|^2 = \frac12\); sum \(=1\) ✓.

So \(|\psi\rangle = \frac{1+2i}{\sqrt{10}}|+\rangle + \frac{1-2i}{\sqrt{10}}|-\rangle\), and an \(X\)-basis measurement gives each outcome with probability exactly \(1/2\).

Problem 4 — Gram–Schmidt in C²

Orthonormalise \(|v_1\rangle = \begin{pmatrix}1\\1\end{pmatrix}\), \(|v_2\rangle = \begin{pmatrix}1\\ i\end{pmatrix}\).

Solution.

Step 1. \(\||v_1\rangle\|^2 = 1 + 1 = 2\), so

\[ |e_1\rangle = \frac{1}{\sqrt2}\begin{pmatrix}1\\1\end{pmatrix}. \]

Step 2. Compute the overlap:

\[ \langle e_1|v_2\rangle = \tfrac{1}{\sqrt2}\begin{pmatrix}1 & 1\end{pmatrix} \begin{pmatrix}1\\ i\end{pmatrix} = \frac{1+i}{\sqrt2}. \]

Subtract the parallel component:

\[ |u_2\rangle = |v_2\rangle - \frac{1+i}{\sqrt2}\cdot\frac{1}{\sqrt2}\begin{pmatrix}1\\1\end{pmatrix} = \begin{pmatrix}1\\ i\end{pmatrix} - \frac{1+i}{2}\begin{pmatrix}1\\1\end{pmatrix} \]
\[ = \begin{pmatrix} 1 - \frac{1+i}{2} \\[2pt] i - \frac{1+i}{2}\end{pmatrix} = \begin{pmatrix} \frac{1-i}{2} \\[2pt] \frac{-1+i}{2}\end{pmatrix} = \frac{1-i}{2}\begin{pmatrix}1\\ -1\end{pmatrix}. \]

Step 3. Norm: \(\left|\frac{1-i}{2}\right|^2 = \frac{2}{4} = \frac12\), and \(\left\|\begin{pmatrix}1\\-1\end{pmatrix}\right\|^2 = 2\), so \(\||u_2\rangle\|^2 = \frac12\cdot 2 = 1\).

\[ |e_2\rangle = \frac{1-i}{2}\begin{pmatrix}1\\-1\end{pmatrix}. \]

Verification. \(\langle e_1|e_2\rangle = \frac{1}{\sqrt2}\cdot\frac{1-i}{2}(1 - 1) = 0\) ✓.

Note. Up to the global phase \(\frac{1-i}{2}\cdot\sqrt2 = e^{-i\pi/4}\), \(|e_2\rangle\) is just \(|-\rangle\) — as expected, since \(\{|+\rangle,|-\rangle\}\) is the natural orthonormal basis containing \(|v_1\rangle\)'s direction.

Problem 5 — Using the completeness relation

Verify that \(|+\rangle\langle+| + |-\rangle\langle-| = I\) by explicit matrix computation.

Solution.

\[ |+\rangle\langle+| = \frac{1}{\sqrt2}\begin{pmatrix}1\\1\end{pmatrix} \cdot \frac{1}{\sqrt2}\begin{pmatrix}1&1\end{pmatrix} = \frac12\begin{pmatrix}1&1\\1&1\end{pmatrix} \]
\[ |-\rangle\langle-| = \frac{1}{\sqrt2}\begin{pmatrix}1\\-1\end{pmatrix} \cdot \frac{1}{\sqrt2}\begin{pmatrix}1&-1\end{pmatrix} = \frac12\begin{pmatrix}1&-1\\-1&1\end{pmatrix} \]

Adding:

\[ \frac12\begin{pmatrix}1+1 & 1-1 \\ 1-1 & 1+1\end{pmatrix} = \frac12\begin{pmatrix}2&0\\0&2\end{pmatrix} = \begin{pmatrix}1&0\\0&1\end{pmatrix} = I \ \checkmark \]

Significance. Every orthonormal basis resolves the identity. This is precisely why the total measurement probability is always 1:

\[ \sum_j \Pr[j] = \sum_j |\langle e_j|\psi\rangle|^2 = \sum_j \langle\psi|e_j\rangle\langle e_j|\psi\rangle = \langle\psi| \Big(\sum_j |e_j\rangle\langle e_j|\Big)|\psi\rangle = \langle\psi|\psi\rangle = 1 . \]

3.8 Practice Problems

  1. Compute \(\langle\phi|\psi\rangle\) for \(|\psi\rangle = (2, -i)^T\), \(|\phi\rangle = (1+i, 3)^T\). [Ans: \(2-2i-3i = 2-5i\)]
  2. Are \((1, i)^T\) and \((1, -i)^T\) orthogonal? Normalise them. [Ans: yes; divide each by \(\sqrt2\) — these are \(\sqrt2\,|{\pm}i\rangle\) up to phase]
  3. For \(|\psi\rangle = \cos\theta|0\rangle + \sin\theta|1\rangle\), find \(|\langle+|\psi\rangle|^2\). [Ans: \(\frac{1}{2}(1+\sin2\theta)\)]
  4. Prove \(P = |\psi\rangle\langle\psi|\) satisfies \(P^2 = P\) for normalised \(|\psi\rangle\).
  5. Apply Gram–Schmidt to \((1,0,1)^T, (1,1,0)^T, (0,1,1)^T\) in \(\mathbb{C}^3\).
  6. Show that if \(\langle\phi|\psi\rangle = 1\) for normalised states, then \(|\phi\rangle = |\psi\rangle\). (Use Cauchy–Schwarz equality condition.)

3.9 Summary

  • \(\langle\phi|\psi\rangle = \sum_j \phi_j^*\psi_j\) — conjugate the left vector.
  • Sesquilinear: linear in the right slot, conjugate-linear in the left; \(\langle\phi|\psi\rangle = \langle\psi|\phi\rangle^*\).
  • A Hilbert space is a complete inner-product space; in finite dimensions this is just \(\mathbb{C}^N\).
  • Orthogonal states are perfectly distinguishable; \(|\langle\phi|\psi\rangle|^2\) is the fidelity/overlap.
  • Expansion coefficients: \(c_j = \langle e_j|\psi\rangle\). Completeness: \(\sum_j |e_j\rangle\langle e_j| = I\).
  • Gram–Schmidt converts any independent set into an orthonormal basis of the same span.

Next lesson: L04 — Operators, eigenvalues, unitarity and the tensor product.

Practise now: unit1-qubit-lab.html → Inner Product tab computes \(\langle\phi|\psi\rangle\) and the overlap for any two states you enter.