Lower Variance vs. Higher Validation Scores

Question

So I'm trying to compare between two models, say model(1) has training accuracy of 90% and validation accuracy of 86%, while model(2) has training accuracy of 87% and validation accuracy of 85%.
Now, model(1) has a better validation score, but with high variance, and model(2) has a lower variance, but a slightly worse validation score.
Which one should I pick? assuming these are the best results we'll ever get.
I'm new in this, but my intuition is pushing me towards picking the more stable model with lower variance, but I would like to get feedback from more experienced professionals.

Jayaram Iyer · Answer

It seems like both models are on par in terms of performance. It will be interesting to see what would happen if you combine predictions from both models via a simple average. In many cases an ensemble of models has shown to yield better performance than any single model.
So maybe it turns out you want to keep the best of both worlds by averaging.

Lower Variance vs. Higher Validation Scores

One Answer

Add your own answers!

Ask a Question