Why are $L_4$ and $L_5$ lagrangian points stable?

I think the fact that you're forgetting (which is easy, I just read the article to find it!) is that in an orbit, angular momentum is conserved. So if an object at $L_4$ or $L_5$ is perturbed inwards, say, then it speeds up slightly and later crosses the orbit. But then it slows down again. In the rotating frame of reference, there's some movement about the Lagrange point but it represents motion about the stable point.

In contrast to the collinear Lagrangian points, the triangular points (L4 and L5) are stable equilibria (cf. attractor), provided that the ratio of $M_1/M_2$ is greater than $24.96$. This is the case for the Sun-Earth system, the Sun-Jupiter system, and, by a smaller margin, the Earth-Moon system. When a body at these points is perturbed, it moves away from the point, but the factor opposite of that which is increased or decreased by the perturbation (either gravity or angular momentum-induced speed) will also increase or decrease, bending the object's path into a stable, kidney-bean-shaped orbit around the point (as seen in the rotating frame of reference).

Here's another way of looking at it. Let $M_1$, $M_2$, $M_3$ be our three masses. In the three body problem we're considering, the whole frame containing $M_1$, $M_2$ and $M_3$ is rotating.

You're right to think that if that frame was fixed then the points L4 and L5 would not be stable. After all if you perturb $M_3$ from L4 or L5 then it should just roll down the potential hill.

But there is another force at work here. Because the frame is rotating there is a fictitious force called the Coriolis force which you would feel. This is the same force that makes hurricanes rotate into spirals when seen from space.

When you take the Coriolis force into account, L4 and L5 become stable fixed points. That is, if you perturb $M_3$ from L4 a little bit it will just stay at it's perturbed distance and orbit the point L4.

If anyone wants to see the maths for this then check out this useful article.

I think what is confusing you is the rotating reference frame... The L4 and L5 points are rotating around the center of mass (COM) of the Earth-Sun system at the same speed as the Earth and Sun orbit around each other. So when you do the thought experiment of what happens to a point mass at L5 or L4 if infinitesimally displaced, it is easier to think of an inertial reference frame. If in the image above the Earth is rotating counterclockwise, this is a qualitative description of what happens to a point mass moved slightly from L5:

• if displaced slightly away from/closer to the COM, it would not have enough/have too much tangential velocity for its new position, and so would start falling behind/getting ahead of L5, i.e. moving clockwise/counterclockwise.
• if displaced slightly ahead/behind of L5, i.e counterclockwise/clockwise around the COM, its velocity will no longer be tangential to its orbit, so it would start drifting away from/closer to the COM.

Combine this together, and you get that an object displaced from its position at L5 will go into a clockwise orbit around L5. This orbit is stable if the ratio of the masses is larger than ~25.

As I know, in L4,L5 , the potential of the Gravitational power is at it's maximum, although it is unusual for kinematics, that considers stable points to be when U->Min , but in Dynamic systems, stable points can be even when U->Max , but then we call it "Dynamical Equilibrium" in sense that the object will actually move around the stable point (but will never become at the stable point itself, even after passing infinite time) (there is also other differences of the classical stable points). Mathematically, this can be proved by using Lyapunov definition of stability for Dynamical Systems.

P.S I never did that by my self but our professor spoke about this issue during our course of Astrophysics.

Three topics:

1. There's an electromagnetic analogy for the non-inertial forces in a rotating reference frame.
2. Before looking at the restricted three-body problem of the question, analyze the familiar two-body problem in a rotating reference frame. I think that interpreting these results in terms of the known solutions for this problem is instructive.
3. Finally, look at the Lagrange points.

1. Electromagnetic analogy. It turns out that the non-inertial forces on a mass $m$ in a rotating reference frame can be manipulated into exactly the form of the electromagnetic Lorentz force on a charge $q$ in an inertial frame.

Reference by Moreno and Barrachina (pdf) . There are some unfortunate typos, but I think the results I state are accurate.

Specifically, for a rotating reference frame with angular velocity $boldsymbol{omega}$, if the electromagnetic scalar potential $phi$ and vector potential $boldsymbol{A}$ are taken to be: $$ phi = -left(frac{m}{q}right) frac{boldsymbol{(omega times r)^2}}{2}$$ $$ boldsymbol{A} = left(frac{m}{q}right) boldsymbol{omega times r} $$ then the electromagnetic force on the charge $q$ in an inertial frame is the same as the non-inertial forces on the mass $m$ in the rotating frame.

In particular, for a constant angular velocity $boldsymbol{omega}$ that is perpendicular to the plane of motion (like in this problem), in cylindrical coordinates $(r,theta,z)$, the analogous electric field $boldsymbol{E}$ and magnetic field $boldsymbol{B}$ are: $$E=left(frac{m}{q}right) omega^2 r , , boldsymbol{hat{r}}$$ $$B=left(frac{m}{q}right) 2 omega boldsymbol{hat{z}}$$

The electric field "simulates" the centrifugal force, and the magnetic field the Coriolis force. All intuition from the Lorentz force then carries over: the Coriolis force diverts a body perpendicular to its velocity, forming circles in the absence of other forces.

2. Two-body problem in a rotating frame. The two-body problem ($M_s$ (sun), $M_e$ (earth)) can be formulated in a rotating frame with angular velocity $omega$ (making the angular velocity the independent variable). Performing the usual conversion to a single-body potential $U$, one gets: $$U=-frac{GM_t M_r}{r} - M_r frac{omega^2 r^2}{2} -M_r omega r^2 dot{theta}$$ $$text{ where } qquad M_t=M_s+M_e , qquad frac{1}{M_r}=frac{1}{M_s} + frac{1}{M_e} , qquad alpha=frac{M_e}{M_t}, beta=frac{M_s}{M_t} $$

Solving for a stationary point $r_o$ (with $dot{theta}=0$), one gets the usual result: $$r_o = left(frac{G M_t}{omega^2} right)^frac{1}{3} , text{sun x-y coordinates }(x_s,y_s)=(-alpha r_o, 0) , text{earth }(x_e,y_e)=(beta r_o, 0)$$

But observe that the potential at $r_o$ is a maximum, not a minimum!

To see what's going on, find the linearized equations of motion around the stationary point. $$(x,y)=(x_o+delta x,y_o+delta y) qquad , qquad (v_x,v_y) = (delta v_x, delta v_y) $$ $$ frac{d}{dt} left( begin{array}{c} delta x delta y delta v_x delta v_y end{array} right) = left( begin{array}{cccc} 0 & 0 & 1 & 0 0 & 0 & 0 & 1 -q_{xx} & -q_{xy} & 0 & 2 omega -q_{xy} & -q_{yy} & -2 omega & 0 end{array} right) left( begin{array}{c} delta x delta y delta v_x delta v_y end{array} right) $$ $$ text{ with } q_{xx}=frac{1}{m} frac{d^2U}{dx^2}, q_{yy}=frac{1}{m} frac{d^2U}{dy^2}, text{ and } q_{xy}=frac{1}{m} frac{d^2U}{dx , dy} $$ The natural frequencies $s_i$ are the roots of the characteristic polynomial, which turns out to be: $$ s^4 - Gamma s^2 + Delta = 0$$ $$ text{ where } Gamma = -( q_{xx} + q_{yy} +4 omega^2) text{ and } Delta = q_{xx} q_{yy} - q_{xy}^2$$

For this case, only $q_{xx}=-3 omega^2$ is non-zero (negative curvature at a maximum), so one finds $Gamma =- omega^2$ and $Delta=0$, and the natural frequencies are: $$ s_{1,2} = 0 qquad , qquad s_{3,4}=pm i omega $$ corresponding to neutral response or oscillatory motion at the rotation frequency. The Coriolis force bends the trajectories and thereby "converts" an unstable equilibrium into a center of oscillatory motion.

Interpretation: A circular orbit (corresponding to the stationary point) is only one possible solution of the two-body problem; in general orbits are ellipses, so one should expect oscillatory motion around the stationary point, periodic at the rotation frequency. (I think the 0 natural frequencies correspond to an azimuthal shift in position.) The point is that the motion is not unstable, despite it being a local maximum, and the lack of damping (negative real part of the natural frequencies) should be expected.

(By the way, if $q_{xx}$ were positive, corresponding to a local minimum, the resulting oscillatory natural frequencies would be at a higher frequency that $omega$, and the resulting motion would not be elliptical.)

3. Lagrange points The restricted three-body problem can be analyzed in the same fashion as above, and the stability of each stationary point determined. A reference is Cornish (pdf). It turns out that L1, L2, and L3 all have positive real natural frequencies and therefore are unstable. L4 and L5 have all pure imaginary natural frequencies (as long as $M_e<25 M_s$), and therefore have motions similar to that of the two-body problem.

Update: Here are some notes on the character of the dynamics at the various Lagrange points:

L4 & L5: These points are most like the two-body problem, with a relatively small, negative (de-stabilizing) potential curvature that is stabilized by the Coriolis force.

L3: This point (planet "x" on the other side of the sun from the earth) is the most interesting to me. Since the curvatures are again small (like for the earth two body problem), and one of them is in fact positive (stabilizing), it seems odd that L3 is unstable. The resolution is that the opposite-sign curvatures form a saddle point. A body can escape from such a point by traveling slowly near the line of zero curvature (and hence zero force); with the proper angle and velocity it accelerates slowly and the Coriolis force is ineffective at stopping the escape.

L1 & L2: These points are unstable on two counts: a) the large negative (de-stabilizing) potential curvature from the nearby earth overwhelms the stabilizing effect of the Coriolis force and b) the principal curvatures differ in sign (like L3), forming a saddle point.

I think there are some good explanations here, but I'll try to add a very simplified, hopefully intuitive explanation.

First, let's agree on some conventions. Suppose the Earth is going counterclockwise around the Sun. It's going in a circular orbit at a distance $R$ (from the barycenter) with angular velocity $Omega$.

We're going to focus on L4. First suppose you started exactly on L4, moving at just the right speed to stay there. This is certainly an equilibrium that doesn't require any force to maintain - we'll leave the issue of stability alone for a bit. What speed are you going while staying on L4? Well, your angular velocity must simply be the pattern speed $Omega$, meaning in the corotating frame (the one in which the Sun and Earth are fixed just as pictured) you don't move at all. In the inertial frame with the barycenter of the system at rest, you must have a speed $Omega R$, and it's directed entirely in the azimuthal direction - that is, tangent to the circle of your orbit.

Now suppose you're displaced from that cozy equilibrium position. For specificity, we'll assume you now find yourself a little bit closer to the barycenter; you were moved along the line connecting it to L4. Also, you didn't lose too much kinetic energy (to be made precise in the next paragraph). This is important. If something, say a viscous drag, keeps draining your total energy, you will have to fall into a lower part of the potential surface, and you won't be able to stay near L4.

Another simplifying assumption: your momentum is still in the same direction as it was when you were blissfully sitting at L4. That is, it is entirely azimuthal. So what's your situation now. Well, your new distance from the barycenter is $R - Delta R$. To keep up with the pattern speed (i.e., to not be moving with respect to the Earth and L4 in that diagram), you need an azimuthal speed of $Omega (R - Delta R) < Omega R$. You don't need as much speed, and hence you don't need as much kinetic energy, to keep up with the rotation. You originally had a specific kinetic energy (kinetic energy per unit mass) of $(1/2) Omega^2 R^2$, and now as long as you have more than $(1/2) Omega^2 (R - Delta R)^2$, you'll advance ahead of the pattern.

So referring to the figure, you were displaced to somewhere below and to the left of L4, and now you'll be moving up and to the left, at least initially. Just like any test mass with a tangential velocity too great for a circular orbit, you'll be moving outward. In the rotating frame, this could be construed as a manifestation of the Coriolis force. [Aside: The Coriolis force acts in the direction $-vec{Omega} times vec{v}$, and with $vec{Omega}$ pointing up out of the page and $vec{v}$ (your velocity in the rotating frame) pointing azimuthally counterclockwise, you can see this will cause an outward-directed acceleration.] But this is more obvious in the inertial frame, where you're now going too fast for the Sun's gravity to bend your path into a circle.

So what happens? As you advance ahead of the pattern (your angular velocity is greater than $Omega$), you also are pushed outward. Eventually you find yourself at a distance $R$ again, but now you're ahead of (counterclockwise of, up and to the left of) L4. The radial "force" will vanish somewhere around here, but remember you have inertia, whether your reference frame is inertial or not! So you'll coast outward, to distances from the barycenter greater than $R$. On this side of $R$, however, everything is reversed. You don't have enough speed to keep up with an angular velocity $Omega$, so you fall behind. You are now moving counterclockwise in that rotating frame, looping around the outside of L4. Of course, because you're going too slow to keep a circular orbit (described in the inertial frame), or because there's an inward-directed Coriolis force (described in the rotating frame), you're accelerated inward. Eventually you'll move around behind L4 and return somewhere near where you started this journey just after getting displaced.

The orbits can be complicated to describe analytically, but hopefully this shows where the "forces" come into play, and where they're directed. You end up tracing a path that goes clockwise around L4 (note all my other uses of "clockwise"/"counterclockwise" have been with respect to the barycenter).

One final note: In the rotating frame this is truly a dynamics problem, where you can't get the motion just by differentiating a simple potential. Note that we started to do that, and that's what lead to the confusion. In addition to the gravitational potentials induced by the two masses, we added a term to the effective potential (the function whose contours are plotted in the diagram) to take care of centrifugal force: $$ Phi_text{eff}(vec{x}) = -G left(frac{M_odot}{lvert vec{x}-vec{x}_odotrvert} + frac{M_oplus}{lvert vec{x}-vec{x}_oplusrvert} + frac{M_odot+M_oplus}{2lvert vec{x}_oplus-vec{x}_odot rvert^3} lvert vec{x} rvert^2right), $$ where $vec{x}$ is the two-dimensional vector in the plane of the system with origin at the barycenter. (Binney & Tremaine's Galactic Dynamics derives this in a roundabout aside; there are probably more direct treatments.) We can only stop here, however, if we only want to consider forces on particles at rest in the rotating frame. As pointed out in another post, the centrifugal force ($-vec{Omega} times vec{Omega} times vec{x}$) isn't the only fictitious force we have to worry about. If the test object has a velocity with respect to the rotating frame, as in the case at hand, we have the Coriolis force ($-2vec{Omega} times dot{vec{x}}$), which we did not account for in setting up the effective potential. The reason is it's not just a function of 2-D position space, but of the whole 4-D phase space. There's one other common fictitious force that fortunately we didn't have to consider: the Euler force ($-dot{vec{Omega}} times vec{x}$). This would only come into play if the pattern speed were changing (say if the Earth had some additional acceleration, perhaps from being in an eccentric orbit).

I'm not sure that there is a simple intuitive explanation, but let's try this:

Before we can clarify the reason for why these point are stable, note the following: if you have an object orbiting around another (to make it simple) much bigger one. This system is isolated an contains a certain total energy. Now if we would like to find other configurations of this two body system with the same energy, we'd find them by decreasing the potential energy (so decreasing the distance between the two objects) the same amount as we increase the kinetic energy (by increasing the tangential speed), therefore leaving the total energy of the system unchanged. Note, that we could do the inverse as well. So, bottom line: For a given total energy, the orbiting body can ether be close to the central object and have a high tangential velocity, or be far away with a lower tangential velocity (see Keppler's 3rd law if you want to know more about how distance and velocity are related).

So, back to the three body problem. To understand why the small mass is in a stable equilibrium state, you might draw an equilateral triangle or look at the satellite in this picture: Since the centre of mass (around which each of the object orbits) is NOT in the centre of the very big mass, the orbit of the 3rd mass (the moon) is slightly inside the orbit of the small mass (in the picture, the orbit of the moon is slightly inside the orbit of the satellite). If now we move the satellite along its trajectory towards the moon, we do not modify its radial distance and thus not change its tangential velocity (according to what was mentioned above). What we do is we bring him closer to the moon AND further away from the earth. Since the satellite does rotate around the moon (also clockwise, to get an idea of that, imaging yourself on the moon and try to figure out how you would see the satellite.) by bringing him closer, we decrease the potential energy between moon-satellite, which leads (since energy conservation) to an increase in tangential speed (clockwise around the moon), thus the satellite tends to move away from the earth (and the centre of mass). By moving away from the earth we increase its potential energy with respect to the earth, which makes him rotate slower around the earth, thus he tends to move away from the moon (since the rotation he describes around the earth is also clockwise. By moving away from the moon, we increase the potential energy between satellite-moon, thus decrease the angular velocity. The satellite tends to move towards the earth, decreasing the potential energy in the system satellite-earth, thus increase of angular velocity bringing him closer to the moon,... and we are back at the beginning.

So following this line of thought it might get clearer why these points are stable (by the way, it does not depend upon the direction of the rotation, try to think it through with counter-clockwise rotation). Unfortunately this is not the end of the story: You could make the same Gedankenexperiment independent on the actual masses of moon and earth and you would always find a stable point. What actually is important, is the mass ration between the two masses. It is only in a rather small range, where the point is stable. For many systems the change in distance between satellite-moon could not be counterbalanced by the resulting change in tangential velocity (leading to a compensating change in distance between satellite earth) and the satellite would leave the point and ether start to orbit around the moon or the sun. Well, if you want to know exactly how it works, check out the math behind. This is a nice description of the phenomenon.

Hope that helped.

By the way: Note that the Coriolis force does NOT exist. It has to be introduced when one uses an accelerated reference system, to compensate the acceleration of the reference system. Typically, we'd need to introduce this fictional force when choosing the earth or the moon as reference system (they both are accelerated since they rotate around each other.) I'd recommend not to start to include such non-existing forces, since you might risk to confuse them with real ones. Use the centre of mass as a reference system, and describe the system with the use of spherical coordinates. When deriving the unit vector, you'll see 'strange' terms that have to be included for the acceleration. When then using Newton 2nd law, these therms stay on the side of the acceleration and there is no need to invent some new forces.

As others have commented, including OP, the effective potential (consisting of gravity & the centrifugal potential) $$V~=~ -frac{Gm_1}{|z_1|} - frac{Gm_2}{|z_2|} -frac{Omega^2 |z|^2}{2} tag{1} $$ in the orbital plane $mathbb{R}^2congmathbb{C}$ has a global maximum at the Lagrange points $L_4$ and $L_5$, $$begin{align}z_1~=~&hspace{5mm} Rexpleft{pm frac{ipi}{3} right} ~=~hspace{5mm}frac{R}{2}pmfrac{sqrt{3}iR}{2}, cr z_2~=~&-Rexpleft{mp frac{ipi}{3} right}~=~-frac{R}{2}pmfrac{sqrt{3}iR}{2} .end{align} tag{2}$$ I'm here using the same notation as my Phys.SE answer here: $$begin{align}z_1~:=&~ z-r_1~neq~0,cr z_2~:=&~ z-r_2~neq~0,cr Omega^2~:=&~frac{G(m_1+m_2)}{R^3}.end{align} tag{3}$$ Usually a test particle doesn't want to be at a global maximum! However, we should not forget the Coriolis force.

Main statement. When the test particle tries to leave $L_4$ or $L_5$, the Coriolis force will prevent this via deflection iff one of the mass ratios $m_1/m_2$ or $m_2/m_1$ exceeds $$frac{25}{2}+frac{3}{2}sqrt{69}~approx~25.tag{4}$$

In this answer, I would like to calculate this mass ratio condition (4), which is also mentioned on Wikipedia.

The Hessian of the effective potential (1) is $$ begin{align}H^{ik}~:=~&frac{partial^2 V}{partial z^i partial z^k},qquad i,k~in~{1,2},qquad {bf z}~in~mathbb{R}^2,cr {bf H}~=~& -Gm_1 left(frac{3{bf z}_1{bf z}_1^T}{|{bf z}_1|^5}-frac{{bf 1}_{2times 2}}{|{bf z}_1|^3}right)cr &-Gm_2 left(frac{3{bf z}_2{bf z}_2^T}{|{bf z}_2|^5}-frac{{bf 1}_{2times 2}}{|{bf z}_2|^3}right) -Omega^2{bf 1}_{2times 2} . end{align} tag{5}$$ At $L_4$ or $L_5$, the Hessian becomes $$begin{align} {bf H}~=~&-frac{3Omega^2}{4} begin{pmatrix} 1 & pm sqrt{3}epsilon_{12} cr pm sqrt{3}epsilon_{12} & 3 end{pmatrix}, cr epsilon_{12}~:=~&epsilon_1-epsilon_2~=~frac{m_1-m_2}{m_1+m_2}~=~1-2epsilon_2~in~]-!1,1[. end{align} tag{6}$$ The trace & determinant are $$ {rm tr}({bf H})~=~-3Omega^2 qquadtext{and}qquad det({bf H})~=~frac{27Omega^4}{16}(1-epsilon^2_{12})~>~0. tag{7}$$

We now use the following theorem$^1$ mentioned in Ref. 1:

Theorem. Let there be given a Hessian ${bf H}$ for an effective potential $V:mathbb{R}^2tomathbb{R}$ at a Lagrange point. Let $Omega$ be the angular velocity. The following 3 conditions are necessary & sufficient conditions for the Lagrange point to be stable:

1. $C~:=~det({bf H})~geq~0. $

2. $B~:=~{rm tr}({bf H})+4Omega^2~geq~0. $

3. $D~:=~B^2-4C~geq~0. $

NB: The above theorem ignores higher-order terms in $V$, which could become important if $det({bf H})=0$.

The first two conditions are satisfied: $$C~=~frac{27Omega^4}{16}(1-epsilon^2_{12})~>~0 qquadtext{and}qquad B~=~Omega^2~>~0.tag{8} $$ The third condition reads $$0~<~D~=~frac{27epsilon^2_{12}-23}{4}~=~27left(epsilon^2_2-epsilon_2 +frac{1}{27} right) tag{9}. $$ The roots of the quadratic equation $D=0$ are $$ frac{m_2}{m_1+m_2}~equiv~epsilon_2~=~frac{1}{2} pm frac{sqrt{69}}{18}~approx~left{begin{array}{c}25/26,cr 1/26.end{array}right.tag{10}$$ This leads to the condition (4). $Box$

References:

1. J. Binney & S. Tremaine, Galactic Dynamics, 2nd edition (2008); p. 181-182.

--

$^1$ Proof of theorem: The linearized EOM at a Lagrange point ${bf z}={bf 0}$ in the orbital plane $mathbb{R}^2$ reads

$$ddot{bf z} ~approx~underbrace{2 Omega ~{bf J} dot{bf z}}_{text{Coriolis}} - {bf H} {bf z}, qquad {bf J}~:=~begin{pmatrix} 0 & 1 cr -1 & 0 end{pmatrix},tag{11}$$

where the first term on the rhs. is the Coriolis force. The Hessian is a real symmetric matrix, and therefore diagonalizable with 2 principal axes. After a possible coordinate rotation ${bf z}mapsto e^{Jphi}{bf z}$, we may assume that the Hessian

$${bf H}~=~begin{pmatrix} H_1 & 0 cr 0 & H_2 end{pmatrix}tag{12}$$

is diagonal. (The rotation commutes with the Coriolis term (11) and leaves the 3 conditions in the theorem invariant!) The EOMs (11) are 2 coupled 2nd-order homogeneous ODEs with constant coefficients. Their characteristic equation is a 4th order equation $$0~=~left| begin{array}{cc} lambda^2 +H_1& -2Omega lambda cr 2Omega lambda & lambda^2 +H_2 end{array} right| ~=~(lambda^2)^2 +Blambda^2 +C tag{13},$$ which has 4 roots $$ lambda^2~=~frac{-Bpm sqrt{D}}{2}tag{14} .$$ Eq. (13) is a 2nd order eq. in $lambda^2$. The solution to the 2 coordinates $z^1$ and $z^2$ are linear combinations of exponentials $e^{lambda t}$, where $lambda$ is a root. The condition of stability is that ${rm Re}(lambda)leq 0$ for all 4 roots.

However due to symmetry in eq. (13), if $lambda$ is a root, so is $-lambda$. So the condition of stability is that ${rm Re}(lambda)=0$ for all 4 roots, i.e. that $lambda$ is imaginary. Or equivalently, that $lambda^2 leq 0$ is non-positive for all 4 roots.

This is only possible if the discriminant $Dgeq 0$ in eq. (14) is non-negative, i.e. the 3rd condition in the theorem.

The 1st & 2nd conditions $$ C,B~geq~ 0qquadLeftrightarrowqquadlambda^2_{pm}~leq~0 tag{15}$$ follow from the well-known fact that $$ C~=~ lambda^2_+lambda^2_-qquadtext{and}qquad-B~=~lambda^2_++lambda^2_- tag{16} $$ in the 2nd order eq. (13) is the product and sum of its roots, respectively. $Box$