TransWikia.com

Comparing AUC, logloss and accuracy scores between models

Data Science Asked on October 2, 2020

I have the following evaluation metrics on the test set, after running 6 models for a binary classification problem:

  accuracy logloss   AUC
1   19%      0.45   0.54
2   67%      0.62   0.67
3   66%      0.63   0.68
4   67%      0.62   0.66
5   63%      0.61   0.66
6   65%      0.68   0.42

I have the following questions:

  • How can model 1 be the best in terms of logloss (the logloss is the closest to 0) since it performs the worst (in terms of accuracy). What does that mean ?
  • How come does model 6 have lower AUC score than e.g. model 5, when model 6 has better accuracy. What does that mean ?
  • Is there a way to say which of these 6 models is the best ?

One Answer

I will start with your last question as the answer explains the other questions. In general, this is a hard question. That is exactly why there are different metrics available. Depending on your problem and what you want to achieve, you should choose the metric that measures this best.

The differences in metrics can be found all over the internet. Others have explained it much better than I can.

Answered by henk.henkert on October 2, 2020

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP