On PS4 Pro, enabling HDR in game (MK11 or SOTTR) seems to actually "disable" HDR

I have bad news for you. It appears your TV has very poor support for HDR.

According to Sony's spec sheet for the KD-85X8500G:

DISPLAY TYPE: LCD
BACKLIGHT DIMMING TYPE: Frame Dimming

As you already know, HDR is supposed to produce very bright whites, and very dark blacks. It's the perfect feature to showcase the abilities of an OLED display.

Fortunately, LCDs can support HDR too, though not nearly at the same quality as OLEDs. In order to truly support HDR, an LCD needs a special type of backlight dimming technology, called FALD (Full Array Local Dimming).

With FALD, your TV is divided into several so-called "backlight zones", each lit by a single LED. The more zones there are, the better the quality of the HDR. In OLEDs, each pixel is its own zone, which is the ideal case. TVs that don't support HDR have only one zone, lit by several LEDs (to ensure sufficient brightness).

The problem with having only one zone, is that the image can either have bright whites (and gray "blacks"), or dark blacks (and gray "whites"). They can't have bright whites and dark blacks at the same time. This problem can still appear with FALD, but is then limited to one of the many smaller zones (and therefore not as noticeable). If you watch HDR content on a FALD LCD screen, you might notice how the "black" around bright pixels appear to be somewhat bright, as if the light was bleeding through. This is called "haloing".

^{Haloing on an Asus ROG Swift PG27UQ, picture by Anandtech}

What you are seeing is the same "haloing" that happens on a FALD LCD screen, except the halo is not limited to a local zone; it extends to the entire screen.

I've noticed that all Sony TVs with "Full Array LED" have a backlight dimming type of "Local Dimming". This means that your TV does indeed have only one backlight zone, and therefore can only dim the entire frame, rather than a local zone of a frame.

So why does content look more vivid with HDR off?

Because the purpose of HDR is to increase the difference between dark and bright. With conventional (SDR) screens, there are only 256 different shades of gray, so only 256 different intensities of brightness. From what I understand, HDR 10 increases that to 1024 shades (possibly even more).

In HDR mode, your TV likely has to display images that contain pixels that are far brighter than what your TV displays in SDR mode. In other words, the backlight LEDs are shining much more brightly, and the LCD fails to completely block out all that extra light, resulting in gray blacks and a pale image in general.

With your current TV, the only benefit to HDR mode, is that dark scenes will be much darker than in SDR mode, as long as your TV doesn't have to display bright pixels.