How does skewed data affect deep neural networks?

Strictly theoretically it makes no difference on DNN, I answered it today here and I said:

Here is why: We already know mathematically that NN can approximate any function. So lets say that we have Input X. X is highly correlated, than we can apply a decorrelation technique out there. Main Thing is, you get X` that has different numerical representation. Most likely more difficult for NN to learn to map to Outputs y. But still in Theory you can Change the architecure, Train for longer and you can still get the same Approximation, i.e. Accuracy.

Now, Theory and Praxis are same in Theory but different in Praxis, and I suspect that this Adjustments of Architecture etc will be much more costly in reality depending on the dataset.

BUT I want to add another point of view: Convergence speed. Strickly theoretically you dont even need [batch normalization] for performance (you can just adjust weights and bias and you should get same results) but we know that making this transformation has big benefits for NN

To conclude for you: Yeah, I had experience where it made difference, and where it didnt. You cant expect theoretical results that say skewed is bad