Yes: locative inversion is normal

As Professor Tolkien so famously opens The Hobbit:

In a hole in the ground there lived a hobbit. Not a nasty, dirty, wet hole, filled with the ends of worms and an oozy smell, nor yet a dry, bare, sandy hole with nothing in it to sit down on or to eat: it was a hobbit-hole, and that means comfort.

And as the great Danny Kaye once sang to us:

I’m Hans Christian Andersen,
     I bring you a fable rare
There once was a table,
     who said “Oh how I’d love a chair”
And then and there came a sweet young chair
     all dressed in a bridal gown
He said to her in a voice so true
“Now I did not say I would marry you
     But I would like to sit down”
I’m Hans Christian Andersen,
     Andersen’s in town.

What you have here in all these cases is a simple case of inversion — locative inversion, to be precise. This is a perfectly normal syntactic variation in English and many other languages. The linked Wikipedia article observes:

An adjunct phrase is switched from its default postverbal position to a position preceding the verb, which causes the subject and the finite verb to invert.

So instead of SV (subject–verb) or SVO (subject–verb–object), under inversion you can also have VS (verb–subject), triggered by the locative prepositional phrase.

And that’s not all!

That isn’t the only time this happens, either. Although the normal ordering of clausal elements in English is SVO, never should you think that the only one possibility, for others too we have.

(Although that latter one is not inversion sensu stricto, because S still precedes V even when the O comes foremost in the clause as I did there. Inversion requires V before S. OSV is just another syntactic possibility.)

Older versions of English once even SOV had, as in:

With this ring I thee wed
Till death do us part.

Despite those SOV examples sounding archaic today, syntactic variations from SVO such as OSV and VSO continue to spice up our writing even in present day English.