How can there be a baby Yoda in "The Mandalorian" when the Empire hasn't been formed yet?

Just to add on to iandotkelly's answer,

While it appears that The Child is a separate character to Yoda, as The Mandalorian season 1 takes place 5 years after Return of the Jedi and Master Yoda's death, Star Wars: Rebels introduced time travel with The World Between Worlds, in which Ezra Bridger was able to enter through a portal on Lothal and where he finds another portal to the moment in time: just before Darth Vader kills Ahsoka Tano.

He was able to reach through space & time, pulling her into the World Between Worlds, where she can also now reside in present, although she does not leave with Ezra, but does show up at the end of *Star Wars Rebels in a scene that jumps ahead to a time shortly after Return of the Jedi. So it IS possible that The Child *could be Yoda (who would then theoretically have to eventually time travel hundreds of years into the past), but there is no indication that this is the case yet.

In addition to answering your questions pertaining to the Empire, yes, the Empire has fallen and the outer rim is left with "remnants" of the Empire vying for power and becoming crime lords. The reason that is significant is because The Rise of Skywalker purposes that somehow

The Mandalorian also purposes that some members of the former Empire (The Client, Moff Gideon) want The Child for it's genetic materials, which may play a key role in "such and such" character's ability to exist in The Rise of Skywalker. In other words there have to be people that were helping that character and The Mandalorian may be telling part of the missing backstory.

As an aside: The World Between Worlds is also relevant to The Rise of Skywalker, as Luke's/Rey's ancient Jedi text include or explain it's theorem in some of it's passages (Rise of Skywalker Visual Dictionary) and because Rey experiences the World Between Worlds when she looks up at all of the stars and the "voices" of mostly dead light-sided/Jedi characters can be heard talking to her, including Ahsoka Tano, who may or may not be dead also.

Star Wars: Rebels also shows us that "such and such character" was trying to gain access to the WBW, as they also mention that they wanted a way to cheat death. So things feel like they all go together...

---

"Luminous beings are we, not this crude matter." - Yoda

_{There is massive multi-platform project titled, Project Luminous that is to be fully announced sometime this year.}

_{It is highly speculated that it will relate to the Force and the next film or film Saga staring in 2022. It's believed Disney is building a MCU within Star Wars with deeper mythology where everything is going to be more interconnected, including perhaps where the Child is going to fit into this "bigger picture" beyond The Mandalorian.}