Why is there an air gap between lenses and digital sensors, as opposed to continuous glass-like higher refractive index?

Since lenses work based on the difference of index of refraction across the interface as well as the curvature of the lens surface (the more the difference in I.R., the more the light rays are bent) putting a higher I.R material between the rear element and the sensor would reduce the effectiveness of the lens.

You could incorporate that into the design of the lens, but you'd need to use a substance with a very low coefficient of expansion across the temperature ranges the system would be used in. Can't have the fluid expanding and forcing its way into the lens proper, or contracting and either creating a vacuum bubble or suction on the lens mount/barrel. And a solid that tried to expand could conceivably crack the lens' rear element or the sensor.

As you know, focal length defines the angle of view and the degree of magnification realized. Some of the lenses we use are “prime” having a fixed focal length. Some lenses are “zoom”, we choose and pre-set the focal length pre-exposure. Additionally, subject distance varies scene-to-scene. We or the camera’s automation must adjust focus based on conditions. In most cases, focusing is a mechanical action that changes the distance lens-to-sensor. However, in a few designs, focusing can be accomplished by repositioning the various lenses in the array keeping the back-focus distance constant. The latter places severe restraints as to subject distance. This is because the travel distance to focus on an object at infinity vs. magnification 1 (life-size) is equal to the focal length. As an example, suppose a 50mm focal length is in use. When focused at infinity, the rear nodal (measuring point) will fall 50mm forward of the sensor. When imaging at “unity” (magnification 1 sometimes states as 1:1) the rear nodal is positioned 100mm forward of the imaging sensor.

What I am trying to say, in normal use the distance, rear nodal to sensor alters considerably. Seems to me, any optical gain realized (if any) would over complicate camera/lens design.

Mostly because the vast majority of lenses are not designed to do extremely high magnification as is the case with microscopy. Why increase complexity and cost to exorbitant levels when there is no added benefit for most applications?

It happens that until very recently there are two things that go between the lens and the sensor in high-end cameras: a mirror in DSLRs, and a curtain shutter. So until came designers figure out how they can completely avoid the two, the question is moot. And after that, it becomes a matter of practicality & cost v.s. added image quality.