Deep learning and label noise. Best practices for the real world

Unlike MNIST or other benchmark datasets collected data often come with subpar, inaccurate labels.

What are the best practices to help the neural networks to don’t overfit the noise? Things that comes to my mind:

• gradient clipping

• regularization

• adjustment to the loss function (e.g. Patrini Making Deep Neural CVPR 2017 paper), but they typically assume the noise to be independent from the features. Have you used them successfully?

• dataset distillation. How did you do that? Do you have references you can share?

I found few papers on the topic, but I found them quite academic (e.g. they simulate the noise by randomly flipping the labels, they assume very large datasets and computational resources). Has anyone applied them successfully?