How can one derive Schrödinger equation?

According to Richard Feynman in his lectures on Physics, volume 3, and paraphrased "The Schrodinger Equation Cannot be Derived". According to Feynman it was imagined by Schrodinger, and it just happens to provide the predictions of quantum behavior.

Fundamental laws of physics cannot be derived (turtles all the way down and all that).

However, they can be motivated in various ways. Direct experimental evidence aside, you can argue by analogy - in case of the Schrödinger equation, comparisons to Hamiltonian mechanics and the Hamilton-Jacobi equation, fluid dynamics, Brownian motion and optics have been made.

Another approach is arguing by mathematical 'beauty' or necessity: You can look at various ways to model the system and go with the most elegant approach consistent with constraints you imposed (ie reasoning in the vein of 'quantum mechanics is the only way to do X' for 'natural' or experimentally necessary values of X).

Small addition to ACuriousMind's great answer, in reply to some of the comments asking for a derivation of Schrödinger wave equation, using the results of Feynman's path integral formalism:

(Note: not all steps can be included here, it would be too long to remain in the context of a forum-discussion-answer.)

In the path integral formalism, each path is attributed a wavefunction $Phi[x(t)]$, that contributes to the total amplitude, of let's say, to go from $a$ to $b.$ The $Phi$'s have the same magnitude but have differing phases, which is just given by the classical action $S$ as was defined in the Lagrangian formalism of classical mechanics. So far we have: $$ S[x(t)]= int_{t_a}^{t_b} L(dot{x},x,t) dt $$ and $$Phi[x(t)]=e^{(i/hbar) S[x(t)]}$$

Denoting the total amplitude $K(a,b)$, given by: $$K(a,b) = sum_{paths-a-to-b}Phi[x(t)]$$

The idea to approach the wave equation, describing the wavefunctions as a function of time, we should start by dividing the time interval between $a$-$b$ into $N$ small intervals of length $epsilon$, and for a better notation, let's use $x_k$ for a given path between $a$-$b$, and denote the full amplitude, including its time dependance as $psi(x_k,t)$ ($x_k$ taken over a region $R$):

$$psi(x_k,t)=lim_{epsilon to 0} int_{R} expleft[frac{i}{hbar}sum_{i=-infty}^{+infty}S(x_{i+1},x_i)right]frac{dx_{k-1}}{A} frac{dx_{k-2}}{A}... frac{dx_{k+1}}{A} frac{dx_{k+2}}{A}... $$

Now consider the above equation if we want to know the amplitude at the next instant in time $t+epsilon$:

$$psi(x_{k+1},t+epsilon)=int_{R} expleft[frac{i}{hbar}sum_{i=-infty}^{k}S(x_{i+1},x_i)right]frac{dx_{k}}{A} frac{dx_{k-1}}{A}... $$

The above is similar to the equation preceding it, the difference relying on the hint that, the added factor with $exp(i/hbar)S(x_{k+1},x_k)$ does not involve any of the terms $x_i$ before $i<k$, so the integration can be preformed with all such terms factored out. All this reduces the last equation to:

$$psi(x_{k+1},t+epsilon)=int_{R} expleft[frac{i}{hbar}sum_{i=-infty}^{k}S(x_{i+1},x_i)right]psi(x_k,t)frac{dx_{k}}{A}$$

Now a quote from Feynman's original paper, regarding the above result:

This relation giving the development of $psi$ with time will be shown, for simple examples, with suitable choice of $A$, to be equivalent to Schroedinger's equation. Actually, the above equation is not exact, but is only true in the limit $epsilon to 0$ and we shall derive the Schroedinger equation by assuming this equation is valid to first order in $epsilon$. The above need only be true for small $epsilon$ to the first order in $epsilon.$

In his original paper, following up the calculations for 2 more pages, from where we left things, he then shows that:

Canceling $psi(x,t)$ from both sides, and comparing terms to first order in $epsilon$ and multiplying by $-hbar/i$ one obtains

$$-frac{hbar}{i}frac{partial psi}{partial t}=frac{1}{2m}left(frac{hbar}{i}frac{partial}{partial x}right)^2 psi + V(x) psi$$ which is Schroedinger's equation.

I would strongly encourage you to read his original paper, don't worry it is really well written and readable.

---

References: Space-Time Approach to Non-Relativistic Quantum Mechanics by R. P. Feynman, April 1948.

Feynman Path Integrals in Quantum Mechanics, by Christian Egli

In Mathematics you derive theorems from axioms and the existing theorems.

In Physics you derive laws and models from existing laws, models and observations.

In this case we can start from the observations in photoelectric effect to get the relation between photon energy and frequency. Then continue with the special relativity where we observed the speed of the light is constant in all reference frames. From this when generalizing the kinetic energy we can get the mass energy equivalence. Combining the two we can assign mass to the photon, consequently we can get the momentum of a photon as function of the wavenumber.

Generalizing the energy-frequency and the momentum-wavenumber relation when have the De-Broglie relations. Which is applicable to any particles.

Assuming that a particle have 0 energy when it stands still (you can do it), although it doesn't cause too much trouble if you leave the constant term there, in the later phases you can simply put it into the left side of the equation. We can deal with the kinetic energy. Substituting the non-relativistic kinetic into the relation and reordering we can have the following dispersion relation:

$$omega = frac{hbar k^2}{2m}$$

The wave equation can be derived from the dispersion relation of the matter waves using the way I mentioned in that answer.

In this case we will need the laplacian and first time derivative:

$$nabla^2 Psi + partial_t Psi = -k^2Psi - frac{i hbar k^2}{2m}Psi$$

Multiplying the time derivative with $-frac{2m}{ihbar}$, we can zero the right side:

$$nabla^2 Psi - frac{2m}{ihbar} partial_t Psi = -k^2Psi + k^2Psi = 0$$

We can reorder it to obtain the time dependent schrödinger equation of a free particle:

$$ partial_t Psi = frac{ihbar}{2m} nabla^2 Psi$$

While it is in general impossible to derive the laws of physics in the mathematical sense of the word, a strong motivation or rationale can be given most of the time. Such impossibility arises from the very nature of physical sciences which attempt to stretch the all-to-imperfect logic of the human mind onto the natural phenomena around us. In doing so, we often make connections or intuitive hunches which happen to be successful at explaining phenomena in question. However, if one had to point out which logical sequence was used in producing the hunch, he would be at a loss - more often than not such logical sequence simply does not exist.

"Derivation" of the Schroedinger equation and its successful performance at explaining various quantum phenomena is one of the best (read audacious, mind-boggling and successful) examples of the intuitive thinking and hypothesizing which lead to great success. What many people miss is that Schroedinger simply took the ideas of Luis de Broglie further to their bold conclusion.

In 1924 de Broglie suggested that every moving particle could have a wave phenomenon associated with it. Note that he didn't say that every particle was a wave or vice versa. Instead, he was simply trying to wrap his mind around the weird experimental results which were produced at the time. In many of these experiments, things which were typically expected to behave like particles also exhibited a wave behavior. It is this conundrum which lead de Broglie to produce his famous hypothesis of $lambda = frac{h}{p}$. In turn, Schroedinger used this hypothesis as well as the result from Planck and Einstein ($E = hnu$) to produce his eponymous equation.

It is my understanding that Schroedinger originally worked using Hamilton-Jacobi formalism of classical mechanics to get his equation. In this, he followed de Broglie himself who also used this formalism to produce some of his results. If one knows this formalism, he can truly follow the steps of the original thinking. However, there is a simpler, more direct way to produce the equation.

Namely, consider a basic harmonic phenomenon:

$ y = A sin (wt - delta)$

for a particle moving along the $x$-axis,

$ y = A sin frac{2pi v}{lambda} (t - frac{x}{v}) $

Suppose we have a particle moving along the $x$-axis. Let's call the wave function (similar to the electric field of a photon) associated with it $psi (x,t)$. We know nothing about this function at the moment. We simply gave a name to the phenomenon which experimentalists were observing and are following de Broglie's hypothesis.

The most basic wave function has the following form: $psi = A e^{-iomega(t - frac{x}{v})}$, where $v$ is the velocity of the particle associated with this wave phenomenon.

This function can be re-written as

$psi = A e^{-i 2 pi nu (t - frac{x}{nulambda})} = A e^{-i 2 pi (nu t - frac{x}{lambda})}$, where $nu$ - the frequency of oscillations and $E = h nu$. We see that $nu = frac{E}{2 pi hbar}$ The latter is, of course, the result from Einstein and Planck.

Let's bring the de Broglie's result into this thought explicitly:

$lambda = frac{h}{p} = frac{2pi hbar}{p}$

Let's substitute the values from de Broglie's and Einstein's results into the wave function formula.

$psi = A e^{-i 2 pi (frac{E t}{2 pi hbar} - frac{x p}{2 pi hbar})} = A e^{- frac{i}{hbar}(Et - xp)} (*)$

this is a wave function associated with the motion of an unrestricted particle of total energy $E$, momentum $p$ and moving along the positive $x$-direction.

We know from classical mechanics that the energy is the sum of kinetic and potential energies.

$E = K.E. + P.E. = frac{m v^2}{2} + V = frac{p^2}{2 m} + V$

Multiply the energy by the wave function to obtain the following:

$Epsi = frac{p^2}{2m} psi + Vpsi$

Next, rationale is to obtain something resembling the wave equation from electrodynamics. Namely we need a combination of space and time derivatives which can be tied back into the expression for the energy.

Let's now differentiate $(*)$ with respect to $x$.

$frac{partial psi}{partial x} = A (frac{ip}{hbar}) e^{frac{-i}{hbar}(Et - xp)}$

$frac{partial^2 psi}{partial x^2} = -A (frac{p^2}{hbar^2}) e^{frac{-i}{hbar}(Et - xp)} = frac{p^2}{hbar^2} psi$

Hence, $p^2 psi = -hbar^2 frac{partial^2 psi}{partial x^2}$

The time derivative is as follows:

$frac{partial psi}{partial t} = - A frac{iE}{hbar} e^{frac{-i}{hbar}(Et - xp)} = frac{-iE}{hbar}psi$

Hence, $E psi = frac{-hbar}{i} frac{partial psi}{partial t}$

The expression for energy we obtained above was $Epsi = frac{p^2}{2m} psi + Vpsi$

Substituting the results involving time and space derivatives into the energy expression, we obtain

$frac{-i}{hbar} frac{partial psi}{partial t} = frac{- hbar ^2}{2m} frac{partial ^2 psi}{partial x^2} + Vpsi$

This, of course, became better known as the Schroedinger equation.

There are several interesting things in this "derivation." One is that both the Einstein's quantization and de Broglie's wave-matter hypothesis were used explicitly. Without them, it would be very tough to come to this equation intuitively in the manner of Schroedinger. What's more, the resulting equation differs in form from the standard wave equation so well-known from classical electrodynamics. It does because the orders of partial differentiation with respect to space and time variables are reversed. Had Schrodinger been trying to match the form of the classical wave equation, he would have probably gotten nowhere.

However, since he looked for something containing $p^2psi$ and $Epsi$, the correct order of derivatives was essentially pre-determined for him.

Note: I am not claiming that this derivation follows Schroedinger's work. However, the spirit, thinking and the intuition of the times are more or less preserved.

To my mind there are two senses in which we can "derive" a result in physics. New theories try to address the shortcomings of older ones by upgrading what we already have, giving new results. They also recover old results. I suppose we can call both derivations.

For example, the TISE and TDSE were first obtained because quantum mechanics said that, where classical mechanics would imply $f=0$, we should have $hat{f}left|psirightrangle = 0$, with $hat{f}$ the operator promotion of $f$, which in this case is $f=E-frac{p^2}{2m}-V$ with operators $E=ihbarpartial_t,,mathbf{p}=-ihbarboldsymbol{nabla}$. (Some results become the weaker $leftlanglepsiright|hat{f}left|psirightrangle = 0$, e.g. with $f=frac{dmathbf{p}}{dt}+boldsymbol{nabla}V$, so I'm not being entirely honest here. But we expect $hat{E}$-eigenstates are important because the probability distribution of $E$ is conserved.)

Note that the above paragraph summarises how Schrödinger was derived in the first sense, and its ending parenthesis hints at how Newton's second law was "derived" in my second sense. And everyone talking about path integrals is hinting at a type-2 derivation for both results (path integrals obtain a transition amplitude in terms of $e^{iS}$ with $S$ the classical action now miraculously coming out of a hat, so technically our direct recovery is of Lagrangian mechanics rather than the equivalent Newtonian formulation).

I'll leave people to fight over which, if either, type of derivation is "valid" or "better", but physical insight requires frequent doses of both. I think it's worth distinguishing them in a discussion like this.