Graphic model factorizing, marginalization

Question

This is actually a probability marginalization question that I encountered in graphic models section of PRML by Bishop (question about equation 8.26 page 391). Assume I have the following graphic model

Therefore the joint probability density of the variables will factor to $p(a,b,c)=p(a)p(c|a)p(b|c)$. Now assume I want to marginalize over $c$, the book says
$$sum_c p(a,b,c) = sum_c p(a)p(c|a)p(b|c) = p(a) sum_c p(c|a)p(b|c)= p(a)p(b|a)$$
This means that $p(b|a) = sum_c p(c|a)p(b|c)$, how to prove this?!
If it was like this $sum_c p(c|a)p(b|c,a)$ then one can reason that $sum_c p(c|a)p(b|c,a) = sum_c frac{p(a,c)}{p(a)}frac{p(a,b,c)}{p(a,c)} = sum_c p(b,c|a)=p(b|a)$. But I can't conclude the same result with $sum_c p(c|a)p(b|c)$. What am I getting wrong?
Thanks in advance

TheCG · Answer

You need to use the fact that $p(b|c,a)=p(b|c)$ i.e. $a$ and $b$ conditionally independent given $c$. This is directly visible on the graph and also proved with:
$p(b|c,a)=frac{p(b,c,a)}{p(c,a)}=frac{p(a)p(c|a)p(b|c)}{sum_{b}p(a)p(c|a)p(b|c)}=frac{p(b|c)}{sum_{b}p(b|c)}=p(b|c).$
Use this additional equality at the beginning of your development to get the result.

Graphic model factorizing, marginalization

One Answer

Add your own answers!

Ask a Question