Fastest way to get to Hyrule castle town from Laynaru tower?

To better illustrate what I said in my comment to the question, I decided to write an answer. Ideally, your route would have you avoid Hyrule Fields altogether because that's where a lot of guardians like to roam. So thinking about that, what I would propose is a route similar to this.

Basically I would head out of the tower and get myself to be on the mountain range directly east of Castle Town. The yellow line means that I would then attempt to fly in to the north-east corner of the town. I don't remember if there are guardians there or not so you would have to watch and see. I also don't know what kind of stamina you have but I imagine with enough food/potions it could be done. If the flying is not possible, an alternative suggestion would be to get into the water north of the north-eastern part of town and try to climb up the cliff to it. Neither of these ideas have been tested or optimized and there is a probably a better solution, but one thing I loved about this game is you don't have to take the obvious route to get somewhere. James Bond that ish and get creative with it.