Data Science Asked by Ramón Wilhelm on December 14, 2020
A fully-connected DNN has layer sizes of 3-3-4-2, where the first layer size represents the input layer. We assume that all layers are affine ones (no ReLU). Give the dimensions of all weight matrices and all bias vectors in the network and compute the total number of free parameters in this DNN.
According to this task the first layer size represents the input layer, so it must be 3.
a(0) = x = 3
If I know the input layer, so 3-4-2 are also the sizes of the bias vectors.
I know now the dimension of the input layer and the bias vector. The columns of W must be also 3, because of the size of the input layer. The rows of W must be equal to the size of the bias vector.
So I computed the following dimensions:
W(1) = W33, b3, a(1) = 3
W(2) = W43, b4, a(2) = 4
W(3) = W24, b2, a(3) = 2
But how to compute the total number of free parameters in this DNN?
Weights at first layer will depend on the Dimension of the input too.
Let's assume, This is your Network. Input has 5 Features
$hspace{6cm}$ Image credit - ML Visuals by dair.ai
Per layer weights and biases
begin{array} {|r|r|}
hline
&Weights &Biases\
hline
Layer-1 &5*3 = 15 &3\
hline
Layer-2 &3*3 = 9 &3\
hline
Layer-3 &3*4 = 12 &4\
hline
Layer-4 &4*2 = 8 &2\
hline
end{array}
Check with Code
import tensorflow as tf
from tensorflow import keras
model = keras.Sequential()
model.add(keras.layers.Dense( 3, activation="linear",input_shape=(5,)))
model.add(keras.layers.Dense( 3, activation="linear"))
model.add(keras.layers.Dense( 4, activation="linear"))
model.add(keras.layers.Dense( 2, activation="linear"))
model.summary()
#model.get_weights() #Can check weights matrix with this
Output
Correct answer by 10xAI on December 14, 2020
Get help from others!
Recent Answers
Recent Questions
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP