How to remove this noise (color patches)?

I’m having images with a visible color noise taking form of color blocks:

One can reveal the patches with modular multiplication (x -> x*d%255, x a 8 bits RGB value of a pixel, d some value, eg., 1.2):

I tried to remove the noise with the state of the art denoiser block-matching 3D BM3D (exactly BM3D with collaborative hard-thresholding and the seconds step is a regularized Wiener inversion using BM3D with collaborative Wiener filtering, as described in Image restoration by sparse 3D transform-domain
collaborative filtering, 2009, Dabov et al):

The BM3D filtered with the modular reveal:

I lost a significant amount of details (shoe, pant folds, grass).

Can someone recommend a denoiser algorithm that would do a better job correcting this noise while preserving the details ?

The image is from a raw file, developped then into a 10bits tiff file, and never went through any jpeg compression, even though, it looks like it did; a jpeg algorithm seems to be applied to the raw image, at least the blocks make me think about it.

Funnily enough, the denoising with BM3D is revealing those blocks, which are less visible in the original.