Data Science Asked by Marzi Heidari on December 14, 2020
I’m studying a paper about Named Entity Recognition. The following is a part of the abstract:
To assess the robustness of NER systems, we propose an evaluation method that focuses on subsets of tokens that represent specific
sources of errors: unknown words and label shift or ambiguity.
I don’t know what "label shift" definition is. The paper doesn’t explain it and I can not find anything that I could understand by Googling it.
Label shift is the opposite of a covariate shift.
In this case, the assumption is that even though the feature distribution remains the same, the Label distribution might changes.
e.g. Symptoms --> Diseases
It can be different for different country (based on medical education of the Country/Doctor)
It can change with time also based on advancement in Medical knowledge
Similar logic can be built for "Words --> Slang". It can change with time due to the acceptance of these words.
Read the references for a formal explanation.
References-
Dive into Deep Learning
Detecting and Correcting for Label Shift with Black Box Predictors
Correct answer by 10xAI on December 14, 2020
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP