Why can I recover more highlight detail from a predecessor model?

The cameras produce different results because you're using the same raw processing settings for cameras that have different sensors and processing pipelines. You need to tweak the settings to match the camera.

To improve highlight detail:

• Don't increase the exposure setting so much. Increase shadow and highlight recovery. This should work if the detail you need is contained in the raw file.

• Reduce exposure when capturing the image. Then use shadow and highlight recovery when processing the raw. This is basically how dynamic range expansion works in some cameras.

Probably the sensor in the D850 is more sensitive, so it looks like it blows out sooner. But shadow detail is probably slightly better. (Sensitivity ≠ Dynamic Range.)

Color correction profiles for different cameras can also have subtle effects that change how settings affect the appearance of images. However, it would not alone explain the large difference in highlight detail.

If the difference between the cameras is consistent across multiple exposure tests, external causes, such as aperture variability and lighting changes are unlikely to cause the differences. (External variability would not consistently affect one camera over the other.) Flickering from AC power cycles is ruled out by the 30 second exposure.

There are probably several things going on here all at once that can each contribute to the variability you have noticed.

• Aperture positions are not exact from one frame to the next, particularly with cameras that use mechanical linkages between the camera body and the lens to set the position of the aperture diaphragm, such as the vast majority of Nikon F-mount lenses.¹ Variability in the exact position of the aperture diaphragm from one frame to the next will result in variability in brightness from one frame to the next if the scene brightness is identical. This is a well known issue among those who use digital cameras for time lapse photography.

• Different sensor designs can be slightly less or slightly more sensitive than another. Each sensor has full well capacity based on the design and materials used to make that sensor. All else being equal (which it never is) a sensor design with higher full well capacity will be more capable of absorbing more photons before it is fully saturated.

• Raw processing conversion applications are sensor dependent. That is, the same application, such as Lightroom, must take into account a specific camera's sensor design and the amount of analog amplification used at each ISO setting when processing raw images from that camera. In your examples, the color profile used with each camera is obviously different.²

• Simply changing color temperature and tint settings can sometimes affect the brightness of objects in a scene based on their color. A very blue object or light source, for example, will be less bright when the color temperature selected amplifies the amber end of the color temperature range more than the blue end than when the color temperature selected amplifies the bluer end of the CT range more than the amber end. The color profiles appear to change between the first and second sets of examples in the question as the exposure is boosted and the highlights are pulled back. This seems to indicate you may be letting LR automatically set WB based on the scene contents. As the contents change based on exposure and highlight adjustments, the WB also appears to be changing.

• Light sources can be variable over time. Artificial light sources usually operate on alternating current that oscillates at either 50Hz or 60Hz. Depending on the type of lighting technology used, lights can flicker rapidly at the frequency of the alternating current or at harmonics of that frequency. Even over longer time periods, such as your thirty seconds exposures, the exact voltage provided by an electrical utility service can fluctuate with the varying loads on their grids. Lights powered by direct current via a battery can also be variable from one shot to the next, particularly if there is a significant time interval between the two, as the energy provided by the battery ebbs and flows due to the chemistry involved. Even illumination from the sun or, possibly in your case, the moon can vary based on atmospheric conditions and cloud cover.

For more about aperture variability, particularly with Nikon F-mount lenses which continued to use mechanical connections long after many other camera systems had transitioned to electronic communication between body and lens, please see: What is an electromagnetic diaphragm? and Inconsistent exposure with same settings--why?

For more about flickering light sources, please see: What causes these inconsistent dark bands in some of my photos from an indoor event? Though the effect is different when the light source in question is a small element within the frame rather than the primary source illuminating the entire frame, the principal is the same. How bright a light appears to be will depend on exactly where in the alternating current cycle it was when the camera captured that spot in the frame.

_{¹ Since I can find no reference to any F-mount 16-35mm f/4E lens having ever been offered by Nikon, I'm assuming your lens has a mechanical connection to the body, which can introduce significant variability from one shot to the next.}

_{² Either that or all of the lighting in the photo was being fed by the same alternating current in the same phase. In that case, the brighter image could have been caught when the lights were at their peak (and also fuller spectrum) and the dimmer image was caught when the lights were in the low point of the AC cycle and less full spectrum and more heavily orange-brown in color. Or maybe the line voltage varied between the two exposures. It's not that uncommon, even in developed countries. The UPS I use for my computer typically measures line voltages anywhere from around 117V to 128V on a regular basis. Though the line voltage provided by the public utility is rated at 120V, it normally runs 122V to 123V most of the time, and I live in a fairly densely populated area of the United States.}

The distribution of dynamic range appears to be different between the pictures. One picture, provides more stops of dynamic range above the midpoint. The other picture more stops below the midpoint.

For example if each picture expresses twelve stops of dynamic range, one picture might distribute four stops below the midpoint and seven stops above the midpoint resulting in more highlight detail at the expense of shadow detail. The other picture might express seven stops of dynamic range below the midpoint and four stops above it...this would provide more shadow detail at the expense of highlight detail.

Usually with night shots, unsophisticated users will prefer shadow detail over highlight detail and more recent cameras might use internal software and their more powerful processors to bias the dynamic range in the corresponding direction.

Or not - I am speculating. But modern cameras have substantial processing power and use it to try to give people usable images whenever possible based on sensible defaults.