Why training and validation similar loss curves lead to poor performance

Question

I am training a binary classification neural network model using matlab the graph that I got using 20 neurons in hidden layer is given below.
the confusion matrix and graph between cross entropy vs epochs.

to prevent overfitting in a model the training curve in a loss graph should be similar to the validation curve.

but in the current situation the third graph shows curve where validation curve is similar to training although the overall accuracy is low as compared to the curve where the two curve diverges in the above plot.

WHy this is happening and what I am doing wrong in understanding these curves?

Fadi Bakoura · Answer

to prevent overfitting in a model the training curve in a loss graph
  should be similar to the validation curve.

That's not always the case, maybe the train set has slightly a different distribution from the validation set, therefore, our focus is just when training loss decreases while validation loss slightly or never decreases.

In the former graph, you notice there was a slight increase in the validation error while the training error was decreasing, while the latter graph has the same behavior due to a less complex model, the model was not able to fit the training data well.

Alex · Answer

to prevent overfitting in a model the training curve in a loss graph
should be similar to the validation curve

Similar in what way? If your training error keeps decreasing and the validation error started growing, you are overfitting. If you have a binary problem, why is the confusion matrix 3x3? Validation set accuracy after 33 epochs looks better than after 8. I don't see a problem.

Why training and validation similar loss curves lead to poor performance

2 Answers

Add your own answers!

Ask a Question