'80s or '90s book about a world where society is beginning to break down due to mass infertility

I’ve read a great book in the late ’80s or possibly early ’90s, and cannot remember either the title or author. It was about the beginnings of a breakdown of society due to mass infertility … might have been something in the water.

Society has turned into a ugly patriarchal system, with the few women who are still fertile tightly controlled, but not by governments. It’s more like cult-like groups.

The main protagonist is some type of ex-soldier and gets in with a group of sensible people. He has the skills to protect them and falls for a woman there. At one point they decide to have a last stand in a supermax prison. It was built but never used due to the breakdown, but turns out to be a great fortress.

During the last part of the book there is an attack by some group. The protagonist leads the defence and prevails. His girlfriend who is somewhere else during the attack later tells him how she baked a cake to keep her mind off things.

It sounds very chauvinist but is actually not and a great read. It’s not "Children of Men", "The Handmaiden’s Tale" or other common ones.

Any ideas?

EDIT

Here’s the book cover, thanks to the answer below.

Rear cover synopsis:
"Gavin Knox was bodyguard to the President of the United States and witness to a crime which could shake civilization to its foundations.

Judith Grenfell was a neurologist who discovered a side effect of the most common pharmaceutical on the market which could cause the greatest biological disaster in human history.

Both were prisoners in the most advanced maximum-security prison ever devised.

Without their information the few survivors of biological catastrophe could dissolve in bloody civil war. They had to escape, and fast, to safeguard the survival of the human race, or leave the world barren for eternity."