Why didn't Anakin just leave the Jedi Order?

He doesn't have anywhere else to go.

Anakin's mother dies in Ep. 2, and he is essentially left with no one on his home planet of Tatooine that he knows well and likes. He also has the memory of being a slave there, and the embarrassing incident with the Sand People that he later confides to Palpatine. Perhaps he could go try to live on the Lars' farm, but that is possibly the one place he would never want to return to.

He doesn't have any real job skills.

Anakin went from slavery straight into Jedi Training (TM). Other than moving stuff around with the Force, he doesn't seem to have many skills that would translate well to a civilian career. The Republic (unlike Star Trek's Federation) is not a post-scarcity society - they have social problems galore, serious poverty, and major inequalities. One can't just "float around" and depend on a social safety net. People who try that get killed, thrown into a Rancor pit, or starve.

We see this sort of job skills problem in our own world - the stereotypical "homeless veteran" who gets out of the military with plenty of combat skills, but without the skills necessary to "make it" in the civilian world. Anakin has literally been with the Jedi since he was nine years old, way younger than the age that most people enter the military in our own world, meaning that his skills, contacts, etc. are even lower than one might expect for someone who joined a military force as an adult.

It's true that he's a good pilot, but he's impulsive and insubordinate - perhaps OK for the kinds of missions that the Jedi go on, but exactly the kind of pilot that would not be wanted for regular cargo missions or even smuggling.

Anakin wants to save Padme

The Force is not Jedi/Sith exclusive. Anakin would still be Force sensitive, even if he somehow resigned from the Jedi Order. And he had lot of skills (piloting, engineering ) even if he had to put his lightsaber away. Also, Padme was not exactly penniless :)

But, there was another problem - his visions about Padme's death. His fear that he will lose her (just like he lost his mother) made him search for more power and knowledge about the Force. The primary reason he wanted to become a Jedi Master (and major disappointment when he was refused) was to have access to Jedi Archives with forbidden knowledge only Masters were allowed to study. He hoped he would find information there about the ways to cheat death.

Therefore, he was an easy prey for Sidious when he offered exactly the knowledge that the Jedi Order denied. Sidious purposely put him close to Padme in Episode 2, sensing his attraction to her and counting on her to become his greatest weakness. Throughout the Clone Wars she was his unwilling pawn to twist Anakin to Dark Side.

While the other answers are legitimate, I believe there is another answer. The fact that he thought Padme would die kept him in the order, where he was 100% sure he could speak to Palpatine as a Jedi. Therefore, for fear of losing Padme, he stayed in the order.