Data Science Asked on July 18, 2021
Regarding Tensorflow/Keras SparseCategoricalcrossEntropy.
SparseCategoricalcrossEntropy(from_logits=True)
expects the logits that has not been normalized by softmax. Then does SparseCategoricalcrossEntropy applies softmax internally?
I suppose applying softmax would not be required internally as taking argmax
would give the index expected.
Please help clarify.
Yes, Softmax function is called when logit=True
Infact, if we check the keras code [Link], the softmax output is ignored in every condition and tf.nn.sparse_softmax_cross_entropy_with_logits
is called. This function calculate softmax prior to cross_entropy as explained [Here]
I suppose applying softmax would not be required internally as taking argmax would give the index expected.
Predicting the class is not the only task here. It has to calculate the appropriate loss value to provide feedback to the network
If the target is in probability values[OHE] i.e. sum=1, not an open-ended logit i.e. [-inf, inf]. So the output should be similarly normalized prior to the loss calculation.
Correct answer by 10xAI on July 18, 2021
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP