TransWikia.com

Number of parameters in Simple RNNs

Data Science Asked by Abir ELTAIEF on March 19, 2021

Please, I am stuck, I can not understand the number of parameters of a simple RNN, here the example and the model summary.
the example is simple:

x = np.linspace(0,50,501)
y= np.sin(x)
df= pd.DataFrame(data=y, index=x, columns=['Sinus'])

Then I would to build a simple RNNs to predict this sine wave,

test_percent = 0.1
test_point= np.round(len(df)*test_percent)
test_ind = int(len(df)-test_point)
train = df.iloc[:test_ind]
test = df.iloc[test_ind:]

from sklearn.preprocessing import MinMaxScaler
scaled_train = scaler.transform(train)
scaled_test = scaler.transform(test)

from tensorflow.keras.preprocessing.sequence import TimeseriesGenerator
generator = TimeseriesGenerator(scaled_train, scaled_train, length=50, batch_size=1)

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, SimpleRNN

n_features=1
model= Sequential()
model.add(SimpleRNN(units=50, input_shape=(50,1)))
model.add(Dense(1))
model.compile(loss='mse', optimizer='adam')
model.summary()

The model Summary

I do not understand the number 2600 in the model summary!
because according to this intuition of RNNS
RNNs Model

in my case the dimension of U is (50,1), so 50 weights, then for V, also 50 weights, and for W, also 50 weights, for me, only 150 parameters, why this number in the summary: 2600 parameters?

One Answer

Keras SimpleRNN is a fully connected RNN so indeed each unit connected with all other units. So the equation becomes:

  • (input_feature +1) x units + units x units
  • 2x50 +2500
  • +1 comes from the bias

enter image description here

Answered by benan.akca on March 19, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP