Where do we have it from that the Moon is migrating away from Earth?

The orbit of an astronomical body around another astronomical body is an ellipse, with the primary in one of the two focal points of the ellipse. Thus the orbiting body gets closer to the primary until it reaches its closest point, and then gets farther away from the primary until it reaches its farthest point, and then gets closer again.

When an astronomical body orbits around another astronomical body, it will gain speed as it gets closer to its primary, but gaining speed will make it move farther from its primary, and as it moves farther from its primary it will lose speed, until losing speed causes it to move closer to the primary, in an eternal cycle.

According to Wikipedia, the Moon's perigee, its closest distance to Earth, is about 362,600 kilometers, but varies from 356,400 to 370,400 kilometers as the Moon's orbit slowly becomes more elliptical and then becomes less elliptical.

According to Wikipedia, the Moon's apogee, its farthest distance from Earth, is about 405,400 kilometers, but varies from 404,000 to 406,700 kilometers as the Moon's orbit slowly becomes more elliptical and then becomes less elliptical.

So that means that the apogee of the Moon is about 42,800 kilometers farther from Earth than the perigee of the Moon is. Because the Moon's orbit slowly becomes more or less elliptical, the difference between the apogee and perigee varies between 34,000 and 50,300 kilometers.

Meanwhile, the tidal interactions between the Earth and the Moon cause the Earth's rotation to slowly get slower, so that the length of an Earth day gets longer, and the Moon slowly moves farther from the Earth.

How slowly does the Moon gradually get farther from Earth?

Measurements from laser reflectors left during the Apollo missions (lunar ranging experiments) have found that the Moon's distance increases by 38 mm (1.5 in) per year (roughly the rate at which human fingernails grow).

Wikipedia: Moon#Tidal effects

So at that rate, it should take the Moon about 26,315.789 years for The Moon's average distance to get 1 kilometer farther from Earth, and about 42,240 years for the Moon's average distance from Earth to get 1 mile farther from Earth.

And in a single month the Moon's distance from Earth varies by about 42,800 kilometers or 26,594.687 miles.

So the very slow and gradual constant movement of the average distance between the Earth and Moon away from the Earth is real, but much smaller in scale than the monthly movement of the Moon toward the Earth and then away from the Earth during a single orbit around the Earth.

There are, I think, at least four parts to this argument: the first being the theoretical argument that ties it all together and the remainder being observational evidence for the Moon's orbit increasing in size.

1. The underlying theoretical argument. This, of course, is the idea that tidal braking causes the Earth to slow down in its rotation and the Moon to move further out in its orbit. (Once upon a time this also caused the Moon to slow its rotation, until it became tidally locked, with its rotation period = its orbital period.) This predicts two things: the Moon should be gradually getting further away from the Earth over time; and 2) The Earth should be slowing down in its rotation (days getting longer) over time.

There are at least three sets of evidence supporting this.

1. The increasing distance of the Moon, as derived from lunar laser ranging. The key point here is that it's the average distance that's increasing. Since the Moon's orbit is elliptical, the distance varies over the course of one lunar orbit, and there are additional variations introduced by the gravitational influence of the Sun, other planets, the not-perfectly-spherical shapes of the Earth and Moon, etc. What this means is that if you trace the Moon's distance over time, you will see it increasing and decreasing, but you will also see that the average distance is gradually getting larger. (And this is what has been measured with the laser ranging experiments; e.g., this 1994 article by Dickey et al. which found that semi-major axis of the Moon's orbit was increasing by about 3.8 cm/year.)

2. The slowing down of the Earth's rotation. As required by basic physics, if the Moon is getting further out in its orbit, and thus gaining angular momentum, there has to be a compensating loss of angular momentum, and this happens via the Earth gradually spinning more slowly. This has been measured in at least two ways:

A. Timing of historical lunar and solar eclipses. There are Chinese records of eclipses going back to roughly 400 BC, and Babylonian records back to almost 800 BC, as well as more recent Greek, Arab, and European records. These can be used to estimate changes in the length of the day, in two ways. First, some records include an approximate time of day, so we can run our calculations backwards and figure when an eclipse visible at, say, Babylon in 200 BC should have occurred. And even when we only know the day (and location) of an eclipse, we can predict where it should have been visible. As explained by this article describing a 2016 study by Stephenson et al.: "The oldest event in the catalog, a total solar eclipse that occurred in 720 B.C.E., was observed by astronomers at a site in Babylon (now modern-day Iraq). But, working backward, today’s astronomers would have predicted that the eclipse should have been seen a quarter of a world away, somewhere in the western Atlantic Ocean. The discrepancy means Earth’s rotation has gradually slowed since the 8th century B.C.E."

B. Geological and paleontological measurements of the length of the day.* It turns out there are some geological records which can be used to derive the number of days in a lunar cycle, or the number of days in a year, millions or hundreds of millions of years in the past.

One way is via the analysis of "tidal rhythmites", which record alternating effects of ocean tides. Since tides have cycles of twice per day (high and low tides) due to the Earth's rotation and twice per lunar orbit (spring and neap tides), you can use the combination to work out how many days there were in a lunar orbit. There can also be yearly variations, which allows you to work out how many days there were in a year when the tidal rhythmites were formed.

Another way is by studying certain fossils, where the growth of part of the organism is recorded in both daily and annual variations: for example, the growth of a shell can vary over the course of a day, but also over the course of a year (e.g., more growth in summer, less in winter). Putting this together allows you to figure out how many days there were in a year when the organism was alive. A fascinating recent example is this careful study of a 70-million-year-old fossil of a rudist (an extinct type of mollusc), which reveals that a year had 372 days back then, so that each day was 23.5 hours long instead of 24 hours.