Impossible to properly adjust rear derailleur indexing

The number one cause of poor shifting is excess cable friction somewhere along the line. You describe "hesitant" shifting coming down from largest cog to the next. This is a symptom of either too much cable tension or excess friction. With the symptom of "nothing happens" when shifting between 3 and 4, which involves not enough cable tension, I would bet that there is something causing the inner cable to get hung up. What you need to do is first do an inspection of the cable from shifter to derailleur. Note whether the cabling takes a smooth path between shifter and the first housing stop. There should be no sharp bends of the cable and it's housing. At the cable stops, make sure that the cable has a ferrule and is fully seated in the stop. The ferrule itself should have a nice round hole and be free of worn edges or burrs which can develop over time from the cable moving through the opening. The inner cable can even wear a groove in the ferrule so that it no longer moves through the actual opening in the ferrule. Check these thoroughly. The inner cable should be round and the strands tightly woven together. Look for any kinks or fraying especially near where it slides in and out of a ferrule at a cable stop. If the inner cable passes underneath the bottom bracket shell on its way back to the derailleur, make sure the cable guide under there is clean and also free of excess wear marks. Make sure the inner cable is routed correctly and cleanly through the cable guide. At the derailleur, the housing should be long enough to create a smooth loop back to the derailleur's housing stop at the barrell adjuster so that there is about 1-2 inches of straight cable coming into the derailleur. Check that the cable is routed through the pinch bolt area correctly.

If the cabling passes inspection there still could be dirt or kinks within the housing that is unseen. Now to check for excess friction you can release the inner cable from the pinch bolt. With your hand, take hold of the inner cable a few inches past where it exits the first housing stop and is without the outer casing. Using mild tension with your fingers, make some shifts at the lever noting the quality of the cable movement. It should slide very smooth through the housing. Keep going back this way past the next run of outer casing to determine if there is a section that is rough or take more effort to move the cable through. You can grasp the inner cable with your fingers on either side of an outer housing run and move the cable back and forth to determine if there is a hiccup along that stretch. If you isolate an area that you suspect might be causing excess friction, at least that stretch of housing should be changed. Any deformations of the inner cable and that should be changed.

If all seems ok, it's always beneficial to clean and lube the inner cable. Clean the exposed sections of inner cable first. Then take the outers out of their stops so you can expose and lube the inner cable that is covered by housing. I use a spray lube called TriFlow for this type of work. It's liquid and a good penetrant and leaves a nice slick coat of Teflon infused lubrication when the the carrier solvent dries off. Lubricate the shifter innards and the derailleur pivot points.

Make sure your derailleur moves smoothly by pushing the body to the inside mimicking it's movement caused by cable tension and it's spring return. Do this with the inner cable still disconnected. Poor shifting after inspection, cleaning and lubrication could be related to incorrect set up. If your confident the setup and indexing are correct, it still could be related to the cable or housing and they should be replaced.