What is Monte Carlo dropout?

Let's start with normal dropout, i.e. dropout only at training time. Here dropout serves as a regularization to avoid overfitting. During test time, dropout is not applied; instead, all nodes/connections are present, but the weights are adjusted accordingly (e.g. multiplied by the keep ratio, which is 1 - dropout_ratio). Such a model during test time can be understood as a average of an ensemble of neural networks.

Notice that for normal dropout, at test time the prediction is deterministic. Without other source of randomness, given one test data point, the model will always predict the same label or value.

For Monte Carlo dropout, the dropout is applied at both training and test time. At test time, the prediction is no longer deterministic, but depending on which nodes/links you randomly choose to keep. Therefore, given a same datapoint, your model could predict different values each time.

So the primary goal of Monte Carlo dropout is to generate random predictions and interpret them as samples from a probabilistic distribution. In the authors' words, they call it Bayesian interpretation.

Example: suppose you trained an dog/cat image classifier with Monte Carlo dropout. If you feed a same image to the classifier again and again, the classifier may be predicting dog 70% of the times while predicting cat 30% of the time. Therefore you can interpret the result in a probabilistic way: with 70% probability, this image shows a dog.