What is up with normalizable wavefunctions?

Why the heck do we even allow non-normalizable wave functions?

We don't.

Why should I have to put my particle in a box?

You don't.

If we have a Hilbert space of states, why are some of the states physical, but others not physical? Isn’t the whole point of a state space to contain actual physical states?

We can debate back and forth about what constitutes a physical state, but normalizability is a prerequisite for all of them. Plane waves are not elements of $L^2(mathbb R)$, and are therefore not physical states.

---

The reason that we use plane waves and other non-normalizable functions is because they (i) are exceedingly useful, and (ii) often give us a sense for how physical states behave with far less work.

Consider plane waves of the form $phi_k(x) = e^{ikx}$ as an example. These objects do not belong to the Hilbert space, which means that that a particle cannot have one of them as a wavefunction. However, we see that $phi_k(x)$ is an eigenfunction of the energy operator with eigenvalue $E_k = hbar omega_k = frac{hbar^2 k^2}{2m}$, and of the momentum operator with eigenvalue $p_k = hbar k$.

The time evolution of such a "state" is given by

$$phi_k(x,t) = e^{i(kx - omega_k t)} ,omega_k equiv frac{hbar k^2}{2m}$$ which can be quickly verified with the Schrodinger equation. Real states can be built from these plane waves via integration:

$$psi(x) = int_{-infty}^infty A(k) e^{ikx} dk$$

where $A(k)$ must be square-integrable. You can think of this as a "continuous sum" of plane waves, where $A(k)$ tells you how much of $e^{ikx}$ to add. We can find the evolution of this state by simply evolving each of the plane waves:

$$psi(x,t) = int_{-infty}^infty A(k) e^{i(kx-omega_k t)} dk$$

In this sense, if we understand how the individual plane waves evolve (which is very easy to ascertain), we can figure out how any arbitrary physical state will evolve by writing it as a continuous sum of plane waves and then evolving the plane waves individually.

---

This may or may not appear trivial or pointless, but if you try to do the same thing with e.g. the step potential, you will find yourself in a world of difficulty if you don't use the non-normalizable eigenstates as computational tools.

In such a case, one solves for the non-normalizable energy eigenstates $phi_k(x)$, which are now piecewisely-defined plane waves with different solutions for $x<0$ and $x>0$. Again from the Schrodinger equation, we can quickly find how these solutions evolve with time. By writing a real state (e.g. a wave packet which approaches the step from the left) as a continuous sum of these non-normalizable states (which is not a trivial task), we can formally write down its evolution as

$$psi(x,t) = int_{-infty}^infty A(k) phi_k(x) e^{-iomega_k t} dk, qquad omega_k equiv E_k/hbar$$

As a specific example, imagine that $A(k) = e^{-(k-k_0)^2/2sigma^2}$ - a Gaussian peaked at some momentum $k_0$ with standard deviation $sigma$. If $sigma ll |k_0|$, then the $A(k)$ is sharply peaked at $k_0$ and so the particle has an approximate momentum of $p=hbar k_0$ (as already stated, a particle with definite momentum is not allowed).

If you're interested in the probability that the particle is reflected or transmitted, then you can get a very good idea by examining the reflection and transmission coefficients for $k=k_0$; the particle is not composed of a single momentum state, but the continuous sum of which it is composed is restricted to a small neighborhood of $k_0$, so restricting our attention to the plane wave state is a good approximation (and far easier).

---

To recap, non-normalizable states such as plane waves are not valid states for a particle to occupy, but they are indispensable computational tools. They are the machinery through which the time evolution of an arbitrary unbound state can be expressed, and their behavior gives us a good indication of the behavior of real particles in many cases of interest.