I hardly dared breathe

According to Oxford's "A Practical English Grammar", "dare" is, as you mentioned, a "semi-modal" as well as an auxiliary verb.

Then in the section related to "Adverbs of frequency", such as "hardly", there are two guidelines that can help us understand why "hardly" in this sentence is placed before the auxiliary.

The first guideline is, of course, the one that makes us a little bit confused about your example, it reads:

With compound tenses, they [adverbs of frequency] are placed after the first auxiliary, or, with interrogative verbs, after auxiliary + subject

But the second one may help resolve the confusion by saying:

Frequency adverbs are often placed before auxiliaries when these are used alone, in additions to remarks or in answers to questions:

Can you park your car near the shops? - Yes, I usually can.

and when, in a compound verb, the auxiliary is stressed:

I never can remember.
She hardly ever has met him.

That is probably why in the example you provided, the adverb "hardly" before the auxiliary "dare" sounds natural and is grammatically correct, since the auxiliary "dare" has kind of an emphatic meaning and is naturally stressed in the sentence.