TransWikia.com

What is the impact of adding a layer in neural networks?

Data Science Asked by Ajinkya Dandvate on October 19, 2020

I was playing with hyper-parameters on https://playground.tensorflow.org/ using spiral dataset (classification). So , first I trained a network with 2 hidden layers and the final test and train loss were around 0.02. After that , I introduced an extra layer (keeping everything other than that same) and trained it . To my surprise , the test loss was around 0.43 and train loss was around 0.37. I expected the test loss to increase , as the model was overfitting , however the significant increase in train loss was kind of counter-intuitive to me.

I am a newbie in this field and from what I know , adding a layer increases the model complexity so increase in test loss was expected but I also expected the train loss to be approximately same (or even decrease) as the model would overfit on train data. But the results show that model not only failed to generalise but even failed to fit on the training data.

I know that there can be many factors behind this , but can anyone please explain what could be the possible reason ?

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP