Gas fireplace venting issue

I'm only a novice on these things, but it sounds like you may need to clean your Blower Wheel of dust, hair and lint. Also, your Air Shutter may need a cleaning. A schematic can be found here fireplaceparts.com/

I couldn't see any mention of it, but you may need a double-wall piping. Or, need to go all of the way to triple-wall for temperatures below 40...if you already have double-wall. As, double-wall can get heated enough to cause outward draft and kill the inward drawing.

Gas , oil, and wood need air( oxygen) to burn , then the flue gas can be vented. So , let in some air, A K A -open a window. When I lived in a colder climate , I always provided an air inlet , generally a duct pipe from outside to near the furnace, or fireplace in this case. Newer furnaces are generally required to have an ejector fan on the exhaust vent because newer houses are well sealed and it is difficult for a natural draft to work. The lower the outside temperature , the more the problem.