How to choose an optimizer for your Neural Network?

My question boils down to this : Which optimizer should I use to train my neural network.

I understand this question depends on the problem.

However, for instance it seems that ADAM optimizer generally outperforms SGD. So maybe we can find some rationality in the choice of optimizers (and their learning rate) ?

Maybe it won’t be mathematically correct, but maybe some of you came accross a standard method for choosing an optimizer ?