TransWikia.com

Fully-Connected DNN: Compute the numbers of free parameter in a DNN

Data Science Asked by Ramón Wilhelm on December 14, 2020

A fully-connected DNN has layer sizes of 3-3-4-2, where the first layer size represents the input layer. We assume that all layers are affine ones (no ReLU). Give the dimensions of all weight matrices and all bias vectors in the network and compute the total number of free parameters in this DNN.

According to this task the first layer size represents the input layer, so it must be 3.

a(0) = x = 3

If I know the input layer, so 3-4-2 are also the sizes of the bias vectors.

I know now the dimension of the input layer and the bias vector. The columns of W must be also 3, because of the size of the input layer. The rows of W must be equal to the size of the bias vector.

So I computed the following dimensions:

W(1) = W33, b3, a(1) = 3

W(2) = W43, b4, a(2) = 4

W(3) = W24, b2, a(3) = 2

But how to compute the total number of free parameters in this DNN?

One Answer

Weights at first layer will depend on the Dimension of the input too.

Let's assume, This is your Network. Input has 5 Features ML Visuals by dair.ai $hspace{6cm}$ Image credit - ML Visuals by dair.ai


Per layer weights and biases begin{array} {|r|r|} hline &Weights &Biases\ hline Layer-1 &5*3 = 15 &3\ hline Layer-2 &3*3 = 9 &3\ hline Layer-3 &3*4 = 12 &4\ hline Layer-4 &4*2 = 8 &2\ hline end{array}

Check with Code

import tensorflow as tf
from tensorflow import keras
model = keras.Sequential()

model.add(keras.layers.Dense( 3, activation="linear",input_shape=(5,)))
model.add(keras.layers.Dense( 3, activation="linear"))
model.add(keras.layers.Dense( 4, activation="linear"))
model.add(keras.layers.Dense( 2, activation="linear"))

model.summary()
#model.get_weights() #Can check weights matrix with this

Output

enter image description here

Correct answer by 10xAI on December 14, 2020

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP