Physical location of Philosopher's Stone before Harry's pocket?

The stone was probably in the room and probably under a spell that will cause you not to find it (kind of like the Fidelius Charm). Dumbledore then enchanted the stone to appear only in the pocket of a person whose deepest desire was to find the stone, but not use it. The Mirror of Erised was there to determine what your desire was.

"How did I get the Stone out of the mirror?"
"Ah, now, I'm glad you asked me that. It was one of my more brilliant ideas, and between you and me, that's saying something. You see, only one who wanted to find the Stone — find it, but not use it — would be able to get it, otherwise they'd just see themselves making gold or drinking Elixir of Life. My brain surprises even me sometimes..."

Dumbledore says the stone was there, but only if you wanted to find it and not use it. Quirrell focused on the mirror, but didn't realise it was only the path to get to the stone. Breaking the mirror wouldn't help — the stone would only reveal itself under very specific conditions.

In short: Dumbledore put the stone in a condition that will cause it to appear only in the pocket of the seeker who's deepest desire was to retrieve the stone but not use it.