Conditional Multivariate Gaussian Identity

Question

I'm trying to verify the form of a multivariate Gaussian provided in a paper I'm reading. It should be pretty elementary.
Let $Y=X+varepsilon$ where $Xsim N(0,C)$ and $varepsilonsim N(0,sigma^2mathbf{I})$. The authors then claim that
$$
X|Y,C,sigma^2 sim N(mu,Sigma),
$$
where
$$
mu := C(C+sigma^2mathbf I)^{-1}Y\
Sigma:=sigma^2C(C+sigma^2mathbf I)^{-1}.
$$
My first thought was to consider the joint distribution
$$
begin{pmatrix}
X\
Y
end{pmatrix}sim NBig(begin{pmatrix}
0\
0
end{pmatrix},begin{pmatrix}
C & C\
C^top & sigma^2mathbf I+C
end{pmatrix}Big)
$$
and apply the conditional Gaussian identities. Unfortunately this approach gives me the right $mu$, but I can't see how their form of $Sigma$ comes about. Any thoughts?

Xi'an · Accepted Answer

This is a correct representation of the conditional variance.
Since
$$begin{pmatrix}
X\
epsilon
end{pmatrix}sim NBig(begin{pmatrix}
0\
0
end{pmatrix},begin{pmatrix}
C & mathbf O\
mathbf O & sigma^2mathbf I
end{pmatrix}Big)$$
and
$$begin{pmatrix}
X\
Y
end{pmatrix} =  begin{pmatrix}
mathbf 1^text{T} &mathbf 0^text{T} \
mathbf 1^text{T} &mathbf 1^text{T}
end{pmatrix}begin{pmatrix}
X\
epsilon
end{pmatrix}$$
the distribution of $begin{pmatrix}
X\
Y
end{pmatrix}$ is
$$begin{pmatrix}
X\
Y
end{pmatrix}sim NBig(begin{pmatrix}
0\
0
end{pmatrix},underbrace{begin{pmatrix}
mathbf 1^text{T} &mathbf 0^text{T} \
mathbf 1^text{T} &mathbf 1^text{T}
end{pmatrix}begin{pmatrix}
C & mathbf O\
mathbf O & sigma^2mathbf I
end{pmatrix}begin{pmatrix}
mathbf 1 &mathbf 1 \
mathbf 0 &mathbf 1
end{pmatrix}}_{begin{pmatrix}
C & C\
C & sigma^2mathbf I
end{pmatrix}
}Big)$$
indeed. With
$$mathbb E[X|Y] = 0 + C (C+sigma^2mathbf I)^{-1}
Y $$
and
$$text{var}(X|Y) = C - C (C+sigma^2I)^{-1} C
$$
Applying the Woodbury matrix inversion lemma
$$(A+B)^{-1}=A^{-1}-A^{-1}(B^{-1}+A^{-1})^{-1}A^{-1}$$
one gets that
begin{align*} C - C (C+sigma^2I)^{-1} C &= C - C (C^{-1}-
C^{-1}(C^{-1}+sigma^{-2}mathbf I)^{-1}C^{-1})C\ &= C - C
+(C^{-1}+sigma^{-2}mathbf I)^{-1}\ &= 
(C^{-1}mathbf I+sigma^{-2}C^{-1}C)^{-1}\ &= sigma^2 C (sigma^2mathbf I+C)^{-1}
end{align*}
The apparent lack of symmetry in the expression may sound suspicious but actually$$C (sigma^2mathbf I+C)^{-1} = (sigma^2mathbf I+C)^{-1} C$$

Conditional Multivariate Gaussian Identity

One Answer

Add your own answers!

Ask a Question