TransWikia.com

can't understand the Architecture of Neural Network

Data Science Asked by Shaelander Chauhan on August 16, 2021

enter image description here

Please explain how Z1 is working I just want to know why W is of shape (4,3) I understand that there are four Weights we are performing (4,3)*(3,1) + (4,1) but I don’t understand what is 3 in (4,3)

Just write the full equation of Z1 rest is self-explanatory.

I New to This field so, please spare me its very common to forget basic concepts in beginnig

3 Answers

Neural Network is nothing but an accumulation of multiple Logistic Regressors. What is the Equation of Logistic Regression?

hypothesis = $W^T * x$ + b activation = Sigmoid (hypothesis)

Now, what does z1 tell us it tells us that first take the dot product between the input (x) and the weight matrix then apply the non-linearity. you have 3 input examples. $<x_1, x_2, x_3>$. and a weight matrix,

[[ $w^{1}_{11} , w^{1}_{12} , w^{1}_{13} , w^{1}_{14}$ ]

[ $w^{1}_{21} , w^{1}_{22} , w^{1}_{23} , w^{1}_{24}$ ]

[ $w^{1}_{31} , w^{1}_{32} , w^{1}_{33} , w^{1}_{34}$ ]],

Now you're multiplying the x with the transpose of W, i.e. $W^{t} * x$ Take the transpose of the Matrix W,

[[$w^{1}_{11} , w^{1}_{21} , w^{1}_{31}$]

[$w^{1}_{12} , w^{1}_{22} , w^{1}_{32}$]

[$w^{1}_{13} , w^{1}_{23} , w^{1}_{33}$]

[$w^{1}_{14} , w^{1}_{24} , w^{1}_{34}$]]

So, the input dimensions are (3, 1) Now the transpose matrix dimensions are (4, 3) and $W^T*x$ dimensions are (4, 3) * (3, 1) after multiplication the dimensions would be (4, 1). Now, Z1 is computed in this way.

The $W^1$ is telling the layer number means for which hidden layer this weight is initialized in your case your weight matrix is for hidden layer number 1. And the subscript $W_{11}$ or $W_{12}$...So on, represents the corresponding matrix multiplication indices $W_{1j}$ this first number in the subscript tells us the input number like in this case $x_1$. and the second subscript associated with the $W_{i2}$ tells us the neuron number in the corresponding hidden layer like in this case it is 2.

Now, lets compute $Z_1$,

$Z_1 = (w^{1}_{11} * x_1 + w^{1}_{21} * x_2 + w^{1}_{31} * x_2)$ + bias $ = sigmoid(Z_1)$ The rest are computed in the same and now you have the weight matrix and input vector you can compute the rest by yourself.

Correct answer by Mohammad Ahmed on August 16, 2021

First, welcome to the deep learning world. Take as simple as it is:

  • The dimensions of weights in any layer = (number of neurons in the next layer, number of neurons of the previous layer). In your example, the weights between the first hidden layer (4 neurons) and the input layer (3 inputs).
  • The dimensions of bias = (number of neurons of the connected layer, 1)
  • Z can be computed as in the following matrix: enter image description here

This is a step-by-step reference for your question: Using the right dimensions for your Neural Network

I hope that helps!

Answered by Hamzawi on August 16, 2021

Another way to look at it, visually, is that the fully connected layer is characterized by the set of edges between the inputs and the outputs, and each edge is represented by a single weight. In this particular construction, there are three inputs and four outputs, which gives a 4 x 3 matrix of weights.

In other words, you have four weights per input, and since there are three inputs, you have 4 x 3 weights.

Answered by XYZ on August 16, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP