Hermiticity and vanishing wavefunction

A hermitian operator can have complex eigenvalues if the wave function is not normalizable. This is for example seen in Gamow's theory of alpha decay, treated for example in Griffith (Example 8.2). Full details are presented in Understanding alpha decay, the point is specifically discussed at the bottom of page 1063. Alternatively, a totally analogous exposition is available in the open-source article Subtleties in Quantum Mechanical Instability, Section II.

If on the other hand the wave function is normalisable, the hermitian operator will have real eigenvalues only. The proof is presented in Section 3.3.1 of Griffith, and I summarise it below, based on the text.

** <...>The normalisable eigenfunctions of a hermitian operator have two important properties<...>**

Theorem 1: Their eigenvalues are real

Suppose

$$hat{Q}f = q f$$ $f$ is an eigenfunction of $hat{Q}$, and in addition $$ left langle f lvert hat{Q} fright rangle = left langle hat{Q} f lvert fright rangle $$ as $hat{Q}$ is Hermitian. Then $$q left langle f lvert fright rangle = q^{*} left langle f lvert fright rangle$$ As $ left langle f lvert fright rangle$ cannot be $0$, $q$ equals it s complex conjugate, $$q = q^{*} $$ and it is hence real.

The proof breaks down if the eigenfunction is not normalisable.

Strictly speaking, in the space of functions $L^2(mathbb{R})$ operator $hat P = -ipartial_x$ doesn't really have any eigenvalues or eigenfunctions. That is because none of the functions that solve the equation $$ -ipartial_xpsi = p psi$$ are normalizable (even if they are bounded), and thus they do not belong to the space $L^2(mathbb{R})$.

However, even though $u_p(x)=e^{ipx}$ isn't a vector in $L^2(mathbb{R})$, and there's no vector $|prangle$ we can still define functional $langle p|$, as follows:

$$ langle p|psirangle = int_{-infty}^infty u_p(x)^* psi(x) dx = int_{-infty}^infty e^{-ipx} psi(x) dx $$

These functionals are not defined on every vector of $L^2(mathbb{R})$, they all have a domain that excludes some veectors. Still, it can be proven that if $p$ is real, then the domain of the functional $langle p|$ is dense in $L^2(mathbb{R})$, which is a very important quality. And to prove that it is important that the function $e^{ipx}$ is bounded.

Moreover, if the function $psi$ belongs to the domains of all functionals $langle p|$ for $pinmathbb{R}$ then it can be proven that $$ psi(x) = int_{-infty}^infty langle p|psirangle u_p(x) frac{dp}{2pi} = int_{-infty}^infty langle p|psirangle e^{ipx}frac{dp}{2pi}$$

which looks very much like a decomposition of a function in a basis, even though it's not really that (because functions $e^{ipx}$ are not vectors, even for real $p$). And as you can see, this decomposition uses only real $p$.