Imprecise shifting on a Shimano 9 speed set

There is no compatibility problem.

If the direction you have to shift twice in is downshifting (pulling cable, going to bigger cogs), that can indicate not enough cable tension. If it's upshifting, it can indicate friction in the cable/housing.

It sounds a lot like one or both (could be both) of the above are true. Misalignment in the derailleur hanger can also add to the finickiness of the adjustment, i.e. making it harder to get the chain to shift right and then stay in place without the ghost shifting. Excess pulley and pivot slop in the RD can also contribute to sluggish shifting. So can a B-tension screw that's too far out.

The most common cause of faulty shifting is excess cable friction. Whether it be from dirt, a worn ferrule with a sharp edge that nips at the inner cable or even a cable that makes too sharp a turn along it's route...anything that adds resistance to the inner cable movement can cause deterioration in shift performance. One study I've read suggests that the total number of cable direction changes (measured in degrees, as you might a circle or angle) is directly correlated to friction within the system. Thus, even though you may route a derailleur cable with nice, flowing sweeps, it's more relevant to pay attention to the number and tightness of the curves.

At any rate, Shimano 7,8 & 9 speed mountain and road plus road 10 speed all share a common derailleur "actuation ratio." Essentially this means that derailleur's from any of these "speed" systems will move the same distance for any given amount of cable pulled. Thus, specific to your question, despite mixing a 9 speed road STI shifter with a mountain 9 speed derailleur, the shifting should work, no problem. Since compatability isn't the issue, we look elsewhere for the hiccup in the system.

In addition to excess friction, another area to examine is the alignment of the derailleur. The derailleur cage should be parallel to the cassette cogs. From a position behind the bike, observe the rear drivetrain. The derailleur cage should be straight up and down and the upper jockey wheel should be directly underneath the cog of the gear it is running the chain on (one exception is at the small, outside cog, where the proper setting would have the teeth of the jockey wheel running directly under the outside plane of the small cog). Common causes of misalignment of the derailleur cage is a bent derailleur hangar, a cross threaded derailleur fixing bolt, a loose or otherwise messed up hangar fixing bolt that causes the hangar to be misaligned. While we're making all these observations from the back of the bike, the very first thing to check is that the rear wheel is fully engaged and centered in the dropouts.

The RD-M592 is a shadow type derailleur and if it is connected to the bike conventionally, there is a little tab on the derailleur's bracket axle unit (the piece of the derailleur that has the fixing bolt that attaches to the bikes hangar) which must rest on the B-tab of the hangar. Subtle issues with shifting result when there is a gap here.

Lotsa things to check. I'd advise doing yourself a favor and change that outer housing. It's inexpensive peace of mind.