Where do incompressible fluids arise in nature?

The word "incompressible" is a pet-peeve of mine, so I apologize in advance!

As you note, what we call "incompressible" is just a mathematical convenience -- it never actually happens in nature! Everything in nature is actually compressible. But sometimes the impact of compression is small, so we can choose to model the material as if it doesn't exist.

So why don't I like that word? Well, it's not precise enough! There are two flavors of incompressibility that show up, depending on which community you work with:

1. "Constant density" incompressible flow -- here, $rho = text{constant}$ and by implication, all derivatives of density with other state variables are zero: $partial rho / partial p = 0$, $partial rho/partial T = 0$, and $partial rho/partial C = 0$ (where $C$ is the composition of the material, expressed in concentrations or moles or mass fractions or whatever you prefer).

2. "Density varies with temperature" incompressible flow -- here, the density is allowed to vary with temperature and composition, but not with pressure. Some folks still call this incompressible for reasons I'll show in a second, but in reality it is just "low Mach number" and should be called that way. Anyway, the result here is that $partial rho / partial p = 0$, $partial rho/partial T neq 0$, and $partial rho/partial C neq 0$.

So there's a lot of interesting things to unpack there, but the most interesting one is the part that is the same between both flavors of "incompressible" (which gives rise to the sloppy nomenclature):

$$ frac{partial rho}{partial p} = 0 $$

This is important because if you work through your equation of state, you'll find:

$$ frac{partial p}{partial rho} propto text{speed of sound}$$

This means the mathematical approximation that density does not change with pressure leads to the effect that the speed of sound is approximately infinite, at least with respect to the other rates of change in the problem.

The impact of this is to change the equations from hyperbolic PDEs in time to elliptic PDEs in time, in general, and it also reduces numerical stiffness issues that arise in low Mach numbers. That's an interesting area on its own and one I would be happy to expand upon in other answers.

---

So how can you tell if a material can be modeled as incompressible?

A material can be modeled as incompressible when the speed of sound is significantly larger than the other time scales of interest, and it can therefore be treated as infinite

But tpg2114's corollary to that is:

A material model should not be called incompressible. When density is constant, it should be called constant density; if density varies due to state variables other than pressure, it should be called low Mach number.