WW2-era bureaucrats escaping nuclear war into Lovecraftian dimension

I think this might be Charles Stross's "A Colder War", available to read linked from Wikipedia and the author's website.

The main viewpoint character, Roger Jourgensen, is a CIA analyst who writes up a report on the state of both the U.S. and Soviet governments' occult research for incoming President Ronald Reagan. This report attracts the attention of "the Colonel" (implied to be Oliver North), who arranges for Jourgensen's transfer and for him to work on a variant of the Iran–Contra affair: secret dealings between the U.S. and Iran to counter Saddam Hussein's Iraq, frustrate the Soviet intervention in Afghanistan, as well as arrange the freeing of hostages in Lebanon.

In the Cold War, the U.S. and NATO lag behind the Soviets in mastery of the dark arts, and rely on nuclear weapons as their main countermeasure. The Soviets gained their knowledge from Nazi Germany, which had moved a sleeping entity from an underwater city in the Baltic Sea; the entity is now contained at Chernobyl. They have also deployed "servitors", unstoppable robot-like beings found in the original Pabodie expedition. Satellite reconnaissance by the U.S. shows that the servitors may have been deployed in Afghanistan, which would violate a secret multinational treaty prohibiting the use of these alien entities in war to which even Adolf Hitler adhered. U.S. countermeasures include 300 megatons of nuclear weapons and a continuity of government base hundreds of light years from Earth, connected via a gate in Washington. The CIA also uses these gates to other planets as roundabout ways to transport drugs and arms to and from the Afghan mujahideen as part of Operation Cyclone.

Stephen Jay Gould briefs the CIA on the evolutionary implications of the alien lifeforms, confirming they come from no Earthly source. Other nations emulate the superpowers; Iran and Israel covertly plan a joint nuclear defence against Iraq's attempts to open a gate to the stars. Eventually, the Colonel's dealings are leaked, and Jourgensen has to testify before a congressional committee. One congressman, horrified by the accounts of the Colonel's dabbling, inquires about the Great Filter: why no aliens have openly stopped by to visit humanity, and only relics and servants remain. He points out that meddling with relics of the Elder Ones would be a good explanation for why other intelligent life has been exterminated before it could visit.

Saddam stabilises the gate of Yog-Sothoth, destroying opposing tribes in Iraq, causing Iran to retaliate with a nuclear attack. The timing unfortunately lines up with a joke by President Reagan; the Soviets and their leader Yegor Ligachev retaliate, with a nuclear war destroying the Middle East and much of the U.S. and Soviet Union. More worryingly, the entity behind the Soviet program, Cthulhu, has somehow been loosed; the U.S. nuclear strike does not appear to slow it down as it heads west across the Atlantic Ocean. Jourgensen and other U.S. personnel retreat to a hidden constructed colony on a distant dying planet, codenamed XK Masada. There, riven by phantom voices, Jourgensen contemplates suicide. He decides against it, as death would be no escape if – as he suspects – he has been devoured by Yog-Sothoth already.

It's not set during World War II, but the eldritch knowledge was acquired then, and otherwise, the plotline seems to fit with them being transported to another dimension to escape nuclear launches, and with the protagonist realizing that the solution may be worse than the problem.

Roger looks at his cigarette disbelievingly: throws it far out into the night sky above the plain. He watches it fall until its ember is no longer visible. Then he gets up. For a long moment he stands poised on the edge of the cliff nerving himself, and thinking. Then he takes a step back, turns, and slowly makes his way back up the trail towards the redoubt on the plateau. If his analysis of the situation is wrong, at least he is still alive. And if he is right, dying would be no escape.

He wonders why hell is so cold at this time of year.