What does "at all" mean in "The Vampire of the Village"?

In "The Vampire of the Village" by G. K. Chesterton, Father Brown was talking about an old parson and his son, saying:

‘I mean,’ said Father Brown, ‘that the son still speaks of his father in a hard unforgiving way; but he seems after all to have done more than his duty by him. I had a talk with the bank manager, and as we were inquiring in confidence into a serious crime, under authority from the police, he told me the facts. The old clergyman has retired from parish work; indeed, this was never actually his parish. Such of the populace, which is pretty pagan, as goes to church at all, goes to Dutton-Abbot, not a mile away. The old man has no private means, but his son is earning good money; and the old man is well looked after. He gave me some port of absolutely first-class vintage; I saw rows of dusty old bottles of it; and I left him sitting down to a little lunch quite recherche in an old-fashioned style. It must be done on the young man’s money.’

Can "at all" be used in a positive statement, and what does it mean here?