Are all carcinogens mutagens?

Alcohol itself is non-mutagenic because it does not directly alter DNA. (Additionally ethanol enhances carcinogenesis and is itself not a carcinogen - updated) There are similar non-mutagenic carcinogens such as estrogen - which is a carcinogen. Another important thing to realize is that a non-mutagenic carcinogen may not alter DNA, but instead alter receptors. This can lead to changes in expression of the DNA without actually changing the DNA. Research, such as in this article (https://www.ncbi.nlm.nih.gov/m/pubmed/2372870/) show that these compounds may alter membranous receptor sites or base their effects on long-term residence in the cells proper. Also, a somewhat older research article may also help your discovery: https://www.ncbi.nlm.nih.gov/m/pubmed/8476535/?i=3&from=/2372870/related. There are plenty of non-genotoxic carcinogens but also some that do alter DNA via intermediates, so it is important to check which one of these is the case.

How can a non-mutagenic agent be carcinogenic?

An agent that causes overexpression of oncogenes or inhibition of tumor supressors, would be carcinogenic but not mutagenic.

HPV, for instance, produces proteins that cause inactivation and degradation of tumor suppressors, p53 and pRb^[1].

Regarding alcohol. As you guessed, alcohol is metabolized to form acetaldehyde which is mutagenic as well as carcinogenic. Moreover, alcohol metabolism can lead to production of ROS which can also be mutagenic. Overall, alcohol is considered a potential carcinogen by IARC due to the observed effects. These classifications are not based on molecular mechanisms.

More recently, a study by Riva et al (2020) has found that only three out of the twenty carcinogens they tested had a mutagenic effect.

Thus, to conclude, not all carcinogens are mutagens.

---

Reference:

^{[1] IARC Working Group on the Evaluation of Carcinogenic Risk to Humans. Human Papillomaviruses. Lyon (FR): International Agency for Research on Cancer; 2007. (IARC Monographs on the Evaluation of Carcinogenic Risks to Humans, No. 90.) 4, Molecular Mechanisms of HPV-induced Carcinogenesis.}

"Some retroviruses contain cancer-causing genes (oncogenes), and cells infected by such retroviruses are oncogenically transformed into tumor cells"

Molecular Cell Biology Lodish et al. 8th ed

DNA is integrated into the genome and cancer is not technically caused by "mutation". That is the case in some of the types of cervical cancers(HPV)

I believe that "not all carcinogens are mutagens" and "not all mutagens are carcinogens".

You may also consider the situation in which a chemical/protein effects the DNA in a way that causes a change in epigenetic code of the genome .Consequently that would also result in cancer by changing the gene expression patterns/amounts of products etc.