Does Gym defending pokemons automatically get kicked out for a specific condition without getting invaded?

I live in a small town where exists 6 Gyms and 18 Pokestops. Since a week I have been noticing that 2 of those 6 gyms are malfunctioning, or maybe I am wrong…

Now let’s come to the main point…

So story of 5 days ago, I captured all of the 6 gyms. So today 4 of the 6 Pokemons got kicked out.
Out of the 4 gyms from which my pokemons got kicked, 2 of the gyms are being controlled by another team and the rest of the 2 gyms are empty. For the rest of the 2 empty gyms, I doubt that had someone invaded it, he would have left his pokemon in it, but the rest 2 gyms remained empty .

Today is not the first time when those 2 out of 6 gyms malfunctioned it happened to me a couple of times

(Both of the malfunctioning gyms are newly transformed).

So, is it a gym glitch or someone is secretly invading the gym without leaving a pokemon to defend?

Or

Is it some unknown condition being met, the Pokemons are getting kicked out ?

Thank You!