Why energy restrictions in the infinite square well are general?

It is perhaps a question of semantics, to some extent, but you can't talk about the energy of a quantum particle unless it is in a state of definite energy i.e. an eigenstate of the Hamiltonian. If you had a superposition of two energy eigenstates, say $$|psirangle = c_1 |E_1rangle + c_2 |E_2rangle,$$ any measurement of the energy will result in either the state collapsing to the state $|E_1rangle$ or $|E_2rangle$, with the resulting energy being $E_1$ or $E_2$ respectively. The probability that an energy $E_1$ will be measured is thus $|c_1|^2$, and that an energy $E_2$ will be measured is $|c_2|^2$. So speaking of the "energy" of the state $|psirangle$ doesn't make sense.

One can, of course, speak of the expectation value of the energy in the state $|psirangle$: $$langle Erangle = langle psi | hat{H}|psi rangle.$$

This quantity is well defined, and can take a range of values, but isn't the same thing as the "energy" of the state.

Note: Nothing about this is unique to the Hamiltonian. It is not sensible to speak of the "value" of momentum of a quantum state if it isn't in a state of definite momentum (i.e. an eigenstate of $hat{p}$) and so on for any quantum observable. However, you could always speak of the expectation values of these observables, though its interpretation is different.

---

Edit: As mentioned in the comments, I interpreted the OP's question as follows: why is it that the only energies that a particle in a box is allowed to be in are the eigenvalues of the time-independent Schrodinger equation. I feel that the OP clearly understands why the energy restrictions apply to the time-independent Schrodinger Equation, just not why are arbitrary state cannot be said to have some fixed energy.

That being said, @BioPhysicist raises a fair point: my argument does not explain why these energies have to be discrete. Indeed, in general, they do not. There are two classes of solutions to the Schrodinger Equation, solutions that have a discrete spectrum ("Bound states") and solutions that have a continuous spectrum ("Scattering states"). If a particular potential allows for scattering states (as, for example, the finite potential well, these solutions will not have discretely spaced energies. Nevertheless, the only states in which the "energy" of the particle can be spoken of in any meaningful way still remain the eigenstates of the Hamiltonian.

In general, a good guess as to whether or not a potential admits scattering states can be made by looking at the behaviour of the potential at infinity. Scattering states are -- crudely -- states where at the particle "starts off" as a free particle at $x = -infty$, and ends up as a free particle at $x = +infty$. For certain potentials (like the infinite potential well and the harmonic oscillator) the "strength" of the potential keeps increasing as we go further away, and therefore the particle never has the possibility of being "free". Such potentials only admit bound (and therefore discrete) state solutions.

I would not blame you if you didn't find the above analysis convincing, but I cannot think of an "intuitive" reason for the existence of discrete energies for the infinite potential well.

Further reading: What exactly is a bound state and why does it have negative energy? and Bound states, scattering states and infinite potentials.

I believe your question boils down to "when can we use separation of variables?"

This is not an easy question to answer in general, but there are some discussions here, here, here, here, and here. Rigorous quantum mechanics is often hard!

The summary of the above posts: Not every solution to the Schrödinger equation is separable, and it depends on both the shape of the potential and on the initial/boundary conditions. However, for simple potentials and initial/boundary conditions, one can prove that it is possible. This is luckily the case for the infinite well potential.

I don't know exactly what you mean with 'general'. For your specific example, we find that the eigenvalues of the Hamiltonian $E_n$ are given by the equation in your post. Hence, the corresponding eigenfunctions fulfill $hat{H} psi_n(x) = E_n, psi_n(x)$. As you pointed out, in the example a solution of the time-dependent Schrödinger equation is given by $Psi_n(x,t) = psi_n(x), varphi_n(t)$ with $varphi_n(t) equiv e^{-i E_n t/ hbar }$. Note that $hat{H}Psi_n(x,t) = E_n Psi_n(x,t)$ and thus the expectation value of the energy in the state $Psi_n(x,t)$ reads:

$$ langle E rangle_{Psi_n} = E_n quad .$$

Was that your question?

Edit: Note that although a superposition of different $Psi_n(x,t)$ still solves the time-dependent Schrödinger equation, the expectation value of the energy will, in general, not be an eigenvalue of the Hamiltonian.