Applying GradCam to video classification models

In the original paper, it says that GradCam visualization can be applied to any convolution based model. The problem is stated for convolutions that process images. In my case, I am classifying videos so therefore I should apply GradCam to every frame individually by calculating the gradients with the loss of the entire video (At least that’s how I think).

The problem is that I am using different models to experiment, such as a ConvLSTM. These use convolutions in each LSTM gate, and though I return intermediate results of each frame, these are maxpooled when passed to the next layer so I cannot get activations corresponding to each frame.
But I also work with a model that uses MobileNet to feature extract each frame, and pass that to a GRU network. In this case mi approach should work?

I am nothing attaching example code because I believe this to be a theoretical question, but if need be I will.