What does "follow him even in thought" mean in "The Vampire of the Village"?

In "The Vampire of the Village" by G. K. Chesterton, the author was describing Father Brown while visiting some persons, saying:

The conversation of Miss Carstairs-Carew, on whom he called next, was certainly calculated to paint the parson’s son in the darkest colours. But as it was devoted to blasting him with all the special vices which Father Brown was quite certain the young man did not exhibit, he put it down to a common combination of Puritanism and gossip. The lady, though lofty, was quite gracious, however, and offered the visitor a small glass of port-wine and a slice of seed-cake, in the manner of everybody’s most ancient great-aunts, before he managed to escape from a sermon on the general decay of morals and manners.

His next port of call was very much of a contrast; for he disappeared down a dark and dirty alley, where Miss Carstairs-Carew would have refused to follow him even in thought; and then into a narrow tenement made noisier by a high and declamatory voice in an attic

Does it mean "not at all"?

Or not even in imagination?