MAE and MSE are Nan for regression with Neural Networks?

Question

I doing a simple neural network for Reggression, I didn't get any error but the MSE & MAE are Nan.  The code is:

dataset = pd.read_excel('data_Z.xlsx')
#df = pd.DataFrame(stock, columns= ['X1', 'X2', 'X3', 'X4', 'X5', 'Y1', 
'Y2'])

#Variables
x=dataset.iloc[:,0:5]
y=dataset.iloc[:,5].values
y=y.reshape (-1,1)
scaler = MinMaxScaler()
print(scaler.fit(x))
print(scaler.fit(y))
xscale=scaler.transform(x)
yscale=scaler.transform(y)
X_train, X_test, y_train, y_test = train_test_split(xscale, yscale)
model = Sequential()
model.add(Dense(12, input_dim=5, kernel_initializer='normal', 
activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(1, activation='linear'))
model.summary()

model.compile(loss='mse', optimizer='adam', metrics=['mse','mae'])
history = model.fit(X_train, y_train, epochs=150, batch_size=50,  verbose=1, 
 validation_split=0.2)

I cannot understand why?

drops · Answer

I had the same problem once when my normalization was "off". I got Nans for all loss functions. Here is what I would do:

either drop the scaler.fit(y) and only do the yscale=scaler.transform(y) OR
have two different scalers for x and y.

Especially if your y values are in a very different number range from your x values. Then the normalization is "off" for x.

Ben Reiniger · Answer

Possibilities:

There are missing values in your dataset.
You are introducing missing values with your scaling.  (In particular, were any of the features constant?  And you seem to be misusing the scaler: the fit method sets the max and min from the data, so running fit(x) then immediately after fit(y) probably isn't what you meant to do.)
If neither of the above is the problem, then presumably Keras is introducing the problem.  This is trickier to diagnose, but often seems to be from vanishing/exploding gradients?  Do you get real scores initially, or NaN's from the very start of training?  See also https://stackoverflow.com/questions/37232782/nan-loss-when-training-regression-network
and its Related questions.

MAE and MSE are Nan for regression with Neural Networks?

2 Answers

Add your own answers!

Ask a Question