Data Science Asked on January 22, 2021
Recently I got very interested in NLP applications of deep learning. Diving into literature (on arXiv for instance) I noticed that is very unpopular to quote and estimate uncertainties on scores of ML tasks. In the era of pretrained language model (i.e. bert, gpt etc.) all further improvements quoted in papers seems to be compatible among each other within 1 or less standard deviations, making all the results statistically compatible with a fluctuation due to stochastic optimization in neural network training procedure (at fixed data-set). I am a physicist, and this looks really confusing to me when compared to the statistical treatment of experimental data performed by routine in laboratories.
I am sure this question has already been discussed in the past in ML/Data Science community, could you point me some review or paper addressing this issue?
Also, could you please share with me your thoughts about?
Thank you very much
In Physics, the uncertainty of a value can be calculated or estimated from some well established rules. For various reasons, these rules are not as well established in Machine Learning/AI. The actual uncertainty value in a machine learning model can suffer from several issues, which are difficult to detect or track down. Here are just a few:
Usually having a highly accurate uncertainty is not necessary, as any change or inaccuracy will usually have little effect in the actual predictions.
Reference:
https://ai.googleblog.com/2020/01/can-you-trust-your-models-uncertainty.html
Answered by Donald S on January 22, 2021
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP