What’s the rule for the sound of the letter A in the middle of three-letter words?

How do you actually pronounce A when it's in the middle of a 3 letter word like mac or rap?

So what is the correct pronunciation of those words and what is the rule?

There's no 'rule', but in standard varieties of English, the letter a is usually pronounced /æ/ (the vowel in the word trap) when it's in the middle of two consonants in a monosyllabic word.

Examples: Trap, rap, mac, bat, mat, hat etc., have /æ/.

Gay and guy have /eɪ/ and /aɪ/ respectively. They have long vowels/diphthongs probably because they're open syllables. Open syllables often have long vowels/diphthongs.

There are some accents where /æ/ and /ʌ/ (as in the word strut) are merged.

There are also some mergers that merge some vowels:

• Salary–celery merger merges /æ/ and/ ɛ/ when they occur before /l/.
• Mary–marry–merry merger merges /æ/, /ɛ/ and /eɪ/.

In summary, I'd say the pronunciation varies from accent to accent or region to region. But the pronunciation of the letter a between two consonants in (monosyllabic words) standard varieties of English is /æ/.

There are more exceptions to rules than there are rules.

See tchrist's comment:

If it ends in ‹‑ar› it’s not /æɹ/ but /ɑɹ/ as in bar, car, far, jar, mar, par, scar, spar, star, tar, tsar — except for war, which has /ɔ/ not /æ/ in /wɔɹ/. It’s also /ɔ/ not /æ/ if it ends in ‹‑aw› as in claw, craw, flaw, gnaw, jaw, law, maw, paw, raw, saw, slaw, straw, thaw. And it’s also /ɑ/ not /æ/ in blah, khan, swab, swan, thwap, twat, wan — as well as wand /wɑnd/ which is quite unlike sand /sænd/ which is normal. Lastly was has /ʌ/, so it’s /wʌz/ not /wæz/ or /wɑz/.

It seems to me that the letter a is /æ/ before plosives in monosyllabic words as in bad, hat, bag, rat, lad etc. (Twat also has /æ/ for me.)

Before r, it's /ɑ/ (as tchrist points out) as in bar, tar, jar, far, scar etc.
But in war, it's /ɔ/ probably because of the /w/ at the beginning.

Sand, land, band have /æ/ but wand has /ɒ/ (in BrE), probably because of the /w/.