18  Postscript

The concept of a Hilbert space was introduced in an attempt to formalise the mathematics which physicists were inventing in order to explain the bizarre behaviours on subatomic scales. After all you’ve seen in this course, does it do so? Not really!

This was known as soon as the theory was formulated. Dirac already wrote in his 1930 textbook:

The bra and ket vectors that we now use form a more general space than a Hilbert space.

Von Neumann, similarly, rejected the concept of a Hilbert space as a description of QM by 1935.

The failure is severe. Essentially every operator you are likely to write down in quantum mechanics is a function of position and momentum (and possibly spin). On the other hand, the first postulate of quantum mechanics says that states live in a Hilbert space. Yet the eigenstates of position do not live in a Hilbert space!

The issues are already perfectly evident from consideration of the two simplest problems in quantum mechanics.

18.1 Plane waves

Let us return briefly to the distinction between bounded Hermitian operators and (potentially unbounded) self-adjoint operators in Section 4.4. The absolute simplest quantum problem has \(V\left(x\right)=0\), \(x\in\left(-\infty,\infty\right)\) The solutions to the time independent Schroedinger equation

\[\hat{H}|p\rangle=\frac{\hat{p}^{2}}{2m}|p\rangle=E_{n}|p\rangle\] are eigenstates of the momentum operator:

\[\hat{p}|p\rangle=p|p\rangle\]

where

\[\langle x|p\rangle\propto\exp\left(ipx/\hbar\right).\]

I.e. plane waves. But these are not normalisable under the \(\left\langle \cdot|\cdot\right\rangle _{2}\) inner product:

\[\int^{\infty}_{-\infty}\left|\exp\left(ipx/\hbar\right)\right|^{2}\text{d}x=\infty\] hence they do not live in a Hilbert space.

Clearly, though, they are objects of physical interest in quantum mechanics. Furthermore, they can even be made mathematically rigorous! To see this, note that, on any interval, we can actually decompose any function containing a finite number of discontinuities into the basis of plane waves, using the Fourier transform. So plane waves are mathematically and physically well defined, and it seems odd to exclude them by fiat¹.

18.2 Infinite Well

The second simplest case in QM is the infinite potential well, with

\[V\left(x\right)=\begin{cases} \begin{array}{c} 0,\\ \infty, \end{array} & \begin{array}{c} 0\le x\le L\\ \text{otherwise} \end{array}\end{cases}\]

and

\[\hat{H}|\phi_{n}\rangle=E_{n}|\phi_{n}\rangle\]

with

\[\langle x|\phi_{n}\rangle=\frac{1}{\sqrt{L}}\sin\left(\frac{n\pi x}{L}\right)=\langle x|s_{n}\rangle.\]

Let us revisit the issues raised in Section 8.4.3. Since the eigenenergies are non-degenerate, \(|\phi_{n}\rangle\) must form a complete orthonormal basis for a Hilbert space. Evidently the dimension of this space is infinity. But which infinity?

Well, \(n\in\mathbb{Z}\), so \(\dim\mathcal{H}=\aleph_{0}\): it is countable. Any state \(|\psi\rangle\in\mathcal{H}\) can be decomposed into the \(|s_{n}\rangle\) basis using

\[\mathbb{I}=\sum^{\aleph_{0}}_{n=1}|s_{n}\rangle\langle s_{n}|\]

and

\[\langle s_{n}|s_{m}\rangle=\delta_{nm}.\]

But on the other hand, any state can also be written in the position basis, using

\[\mathbb{I}=\int^{L}_{0}|x\rangle\langle x|\text{d}x\]

and

\[\langle x|y\rangle=\delta\left(x-y\right).\]

This assigns a separate state to each point on the real interval \(x\in\left[0,L\right]\). But the number of points on an interval of the real line is the definition of the cardinality of the continuum, suggesting \(\dim\mathcal{H}=\mathfrak{c}\). And Cantor showed that \(\mathfrak{c}>\aleph_{0}\).

So which is it? The resolution to both of these paradoxes is that states in quantum mechanics do not actually live in Hilbert spaces after all...

18.3 Concluding remarks

Von Neumann and Dirac proposed alternative approaches to axiomatising quantum mechanics. Von Neumann’s approach rejects plane waves and delta functions as non-rigorous. In his 1932 textbook he states that Hilbert spaces are the correct objects of study for quantum mechanics. But already by 1935 he had realised that this was not the case. In the subsequent century, Von Neumann’s approach has been formalised into the theory of \(C^{*}\)-algebras. Coarsely, we might say that this is the mathematicians’ approach to quantum mechanics².

Dirac includes plane waves and delta functions, and so rejects the notion that Hilbert spaces fully describe quantum systems. Dirac’s approach was later formalised using the idea of rigged Hilbert spaces (which, in being rigged to work formally, cease to be Hilbert spaces). Coarsely, this is the physicists’ approach to quantum mechanics.

More generally, incorporating special relativity into quantum mechanics, and allowing for particle creation and annihilation, one arrives at quantum field theory (QFT). Von Neumann’s approach led to Algebraic QFT, while Dirac’s led to Axiomatic QFT (Wightman’s axioms).

A rigorous formal axiomatisation of quantum field theory, which simultaneously allows useful calculations, is yet to be found. It is one of the Clay Mathematics Institute’s million-dollar Millennium Prize Problems (Existence & the Mass Gap). You are encouraged to give it a go.

---

1. Prof. Robbins comments: “I would say that \(|x\rangle\) and \(|p\rangle\) are well defined mathematically - they don’t need to be banished from mathematically rigorous discourse. But they are not elements of \(L^{2}\).

   Are they well defined physically? I guess I would say they are unphysical idealizations/limits of physically meaningful states. Why unphysical? Well, for one thing, for most Hamiltonians, \(|x\rangle\) and \(|p\rangle\) would have infinite energy.

   In the same vein, it might be worthwhile pointing out that there is a mathematical distinction between spectrum and eigenvalue. Self-adjoint operators have real spectra, but a point in the spectrum needn’t be an eigenvalue (as with the position and momentum operators). This brings us back to \(|x\rangle\) and \(|p\rangle\) not being in the Hilbert space. One can say that for points in the spectrum, there are approximate eigenstates (eg, functions that approximate delta functions in the case of \(x\)).”

2. Prof. Robbins comments: “I don’t know how von Neumann’s views evolved, but I don’t think all mathematicians have given up on the Hilbert space formulation. That is, you can find mathematical treatements of nonrelativistic QM formulated in terms of Hilbert space. As I understand, the move to \(C^{*}\) algebras is motivated by regarding density operators rather than wavefunctions as describing the state of a physical system. This has the advantage of removing the phase ambiguity as well as allowing for mixed states alongside pure states.”