How do we generate the samples of hidden root nodes in the Bayes network (Sigmoid Belief Networks) of a generative model

Question

Following is a Sigmoid Belief Networks where we can only observe the bottom observable layer $v.$ Usually we use Wake-Sleep to trainning the bottom-to-up Recognition Weight $r_{ji}$ and up-to-bottom Generative Weight $w_{ij}.$ The conditional probability of all the nodes except for the top layer root nodes $(h^{(2)}_1,h^{(2)}_2)$ are assumed to be the Bernoulli distribution with Logistic probability function. There is no assumption of the probability function on $(h^{(2)}_1,h^{(2)}_2).$
As a generative model, how do we generate the samples of roots $(h^{(2)}_1,h^{(2)}_2)?$

TheCG · Answer

From this tutorial:

To do this, for each unit of the top layer, the sigmoid function is simply applied to the additive constant in (1), as there are no subsequent layers providing influence.

This means only a bias is associated with the top random variables to parametrize the logistic function:
$p(h^{(2)}_1=1)=frac{1}{1 + exp(-b^{(2)}_1)}$ and $p(h^{(2)}_2=1)=frac{1}{1 + exp(-b^{(2)}_2)}$.

How do we generate the samples of hidden root nodes in the Bayes network (Sigmoid Belief Networks) of a generative model

One Answer

Add your own answers!

Ask a Question