Why are my red potatoes so bitter

Have these potatoes started to sprout? According to this article from Caroline Wright, a lecturer in horticulture, "there can still be a bitter flavour to potatoes that have begun to grow." Exposure to light produces solanine. Solanine is a bitter toxin and even when pared from the potato, can have imparted bitterness throughout the spud.

As I have aged I’ve stopped being able to eat red bliss potatoes. The rest of my family loves them but for me, the skins are extremely bitter. I recall my grandmother saying she couldn’t eat them. I think it’s something genetic.

The question is a bit underspecified - you are talking about "red potatoes", but out of the 400 cultivars of potatoes, there are several dozens of red ones, so we don't know which ones you mean.

Anyway, you have to consider that 1) potatoes are a naturally bitter plant (they evoloved so to defend themselves from being eaten!), it is humans who bred the bitterness out of them, and 2) human flavor perception is not absolutely uniform. It is much more varied for smells (with different humans having or not having receptors for different molecules) than for taste, but I wouldn't see it as impossible that it happens for taste too in individual cases.

So, my best guess here is as you said in the question: there is some substance which you can taste that almost nobody else can. The potato variety you are talking about seems to be producing it, and breeders didn't breed it out, because they, and their customers, didn't taste it and so they found that potato variety perfectly palatable.

To test the theory, you can cook a batch of potatoes which is bitter to you, and have a handful of friends (not genetically related family members) taste it. If they notice no bitterness, it is a highly likely explanation.

If that's the case, you cannot get rid of it. Just buy other varieties of potatoes.