TransWikia.com

Statistical test using Gmean to compare multiple algorithms on multiple datasets

Data Science Asked by Nafisa Anjum Samia on July 14, 2021

I am new in this area. I am facing some issues while comparing the algorithms using statistical test. I have following result of Gmean of some classification algorithm. Abalone, Balance-scale, Car, Chess are the datasets here and ROS, RUS, RFS, NoS are the algorithms.

If I want to compare which dataset is better than others using some statistical test like, t-test, Friedmen test, Wilcoxon test etc then is it possible that I can compare the algorithm using the following table?

                  ROS       RUS     RFS     NoS
Abalone           0.003     0.0036  0.0039  0
Balance-scale     0.8858    0.8065  0.8966  0.9417
Car               0.9191    0.7216  0.9056  0.9094
Chess             0.4912    0.1973  0.5084  0.1438

If anyone has any idea about it please help me. Or you can share any references from where I can find the solutions. I studied these statistical test, null hypothesis, p-value etc but couldn’t understand whether it is possible or not to compare these algorithms using Gmean.

***G-mean = Geometric Mean, used to evaluate the performance of multiclass classifiers

Thanks in advance.

One Answer

The performance of machine learning algorithms is not commonly evaluated with null hypothesis significance testing (NHST).

Machine learning performance is evaluated with performance on a hold-out data (e.g., validation or test), regardless of the evaluation metric.

Answered by Brian Spiering on July 14, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP