Is it possible to have your Pokemon get out of its Pokeball and follow you?

This is only possible in Pokémon Yellow (Pikachu only), and Heart Gold/Soul Silver (The first Pokémon in your party), where your Pokémon follow you automatically.

There's also a special park on Platinum and a side-quest on Black/White 2, but these don't count as "walking Pokémon".

Also in the side games Pokémon Ranger and Pokémon Mystery Dungeon games, Pokémon can be outside with you in most places.

Edit: Pokémon are also allowed to be out on games such as Let's Go Pikachu and Eevee, or Sword and Shield on the DLC content's Islands.

Sources