BERT: it is possible to use it for topic modeling?

From Quora

I do not think you can use BERT to do topic modeling out of the box. You could use BERT embeddings to cluster texts by cosine distance and do a topic modeling using Gensim or other packages on each cluster.

No. Most folks use https://radimrehurek.com/gensim/models/ldamodel.html for topic modeling and https://github.com/bmabey/pyLDAvis for visualization.

So, overall the question is about understanding the BERT architecture and whether it can be used in topic modelling. I think to answer this question, I will give a brief overview of the BERT architecture and how its trained.

Overall, BERT is essentially a deep neural network consisting of multiple transformer layers. The BERT model is pre-trained which a large corpus to effectively develop a language model over the corpus. A language model is exactly what it says on the tin, it models a language given a corpus. So the language model essentially can tell you (or another model, for example) how likely is a given sentence to be in a particular language (i.e. essentially measuring fluency of a given sentence). (Good video series which talks about BERT: https://www.youtube.com/watch?v=FKlPCK1uFrc)

Topic modelling on the other hand focuses on categorising texts into particular topics. For this task, it is arguably arbitrary to use a language model since topic modelling focuses more on categorisation of texts, rather than the fluency of those texts.
Thinking about it though, as well as the suggest given above, you could also develop separate language models, where for example, one is trained on texts within topic A, another in topic B, etc. Then you could categorise texts by outputting a probability distribution over topics.
So, in this case, you might be able to do to transfer learning, whereby you take the pre-trained BERT model, add any additional layers, including a final output softmax layer, which produces the probability distribution over topics. To re-train the model, you essentially freeze the parameters within the BERT model itself and only train the additional layers you added.

One year later I just found out about this project which is exactly about your question https://towardsdatascience.com/topic-modeling-with-bert-779f7db187e6