Why don't we see more lightsaber colors?

Because of the Jedi who created them

The color of a lightsaber comes from an interaction between the Jedi who created it, and the crystal itself. According to the Databank:

A lightsaber crystal is colorless until first attuned and connected to a Jedi – at which times it glows either blue or green, or in some rare instances, another shade. From that point on, it retains that hue.

Indeed, the nature of Sith lightsabers is actually relevant to the question. The color of Sith lightsabers is created by the application of the dark side.

Dark crystals were made, too, but not in that holy place. They were plundered from their rightful bearers and corrupted by the hands that stole them. Even rock could be changed by the power of the Force, bleeding alterations until their color was the deepest red. The balance was finely staged between the two, light and dark, and it took very little to upset it.

_Ahsoka

If the interaction between a dark side user and a kyber crystal can produce a crystal of invariably red color, it’s not surprising that the interaction between a light side user and a kyber crystal can produce a crystal that’s generally blue or green. Lightsabers of a different shade, such as Mace Windu’s purple blade, presumably indicate something special about their wielders, some element of their personality or presence in the Force that differs from most Jedi.

You won't find the answer to this question in universe.

Initially, George wanted the good guys to have blue lightsabers and bad guys red. The colours represented them well.

Luke's lightsaber is green in ROTJ because he waves it against the blue sky a lot on the sail barge and skiffs. Blue would not have provided enough contrast.

Samuel L. Jackson wanted a purple lightsaber so he would stand out of the crowd of Jedi. He got his way. When he asked George about the colours, George said "that's just the way it is."

The animated TV shows and video games are less restrictive.

Legends

Well, in Legends, the Kyber Crystals were not colourless, they already had a natural colour which could vary depending of where said crystal was found.

So why 2 colours only when there is certainly thousands of planets full of crystals within them ?

Because of the several war between Jedi and Sith, the planets were either destroyed or their location lost. You can read the following quote from the book Jedi vs. Sith: The Essential Guide to the Force :

"Toward the end of the Old Republic era, Ilum was the primary source of most crystals used for lightsabers; this dramatically limited the color range, as the crystals harvested from Ilum produce only blue and green blades."

Canon

In Canon there is, as far as I know, no explanations of why these 2 colours are the most common.

But if we follow the logic of "Your stance in the force decide of the colour", maybe the black and white vision of the Force by the Jedi has brainwashed them from their infancies.

Thus their personalities shaped by the doctrines of old are the main reasons of why the last Jedi could only produce 2 colours save for a few open-minded ones (Windu, Ashoka,..) .