GRU and LSTM does not "take risk" predicting

Question

I tested LSTM and GRU models to predict the exchange rate between currencies. I do not take the raw price but a the delta with the previous day, so the data is stationnary around zero.
My problem is that my model always predict really close-to-zero values, like if it minimize the risk and does not want to guess wrong. It may be because it underfit but I wanted to be sure that it is not a common issue that I just totally don't know about.
I have really simple architecture, 1 layer of GRU or LSTM (tried both), data from the past 20 years, and using the 20 previous days to predict the next one. Tried to play with LR, dropout and number of epochs yet but this behaviour seems weird to me. Maybe I miss something?

Thanks!

Nicholas James Bailey · Answer

This is probably because the deltas you are trying to predict are less than 1, so your loss function (I’m assuming MSE) isn’t working as expected.
Squaring an error less than 1 will make it even smaller, so your model is not currently motivated to leave the cosy local minimum of the naïve strategy of always predicting delta as zero.
I recommend rescaling your deltas by multiplying them by 100 and then using MSE as a loss function.

GRU and LSTM does not "take risk" predicting

One Answer

Add your own answers!

Ask a Question