Looking to build the Mathematical proof that $ Var(hat{y}) = sigma^2textbf{H} $

Question

I'm looking to build the proof that
$$
Var(hat{y}) = sigma^2textbf{H}
$$
So I was going down this road and seem to have gotten a bit confused:
$$
Var(hat{y}) = sigma^2textbf{X} (textbf{X}'textbf{X})^{-1}textbf{X}'\
mathrm{Var}(hat{y}) = mathrm{var}(Xhat{beta})\
= Var[X(X'X)^{-1} X'] var(y)
$$
Am I on the right track or where did I go wrong?

Linxing Yao · Answer

Suppose the estimated parameters are:
$$hat{beta}=(X^TX)^{-1}X^Ty$$
therefore,
$$hat{y}=Xhat{beta}=X(X^TX)^{-1}X^Ty$$
since $Var(AX)=AVar(X)A^T$, we can have:
begin{align}
Var(hat{y})&=Var(X(X^TX)^{-1}X^Ty)\
            &=X(X^TX)^{-1}X^TVar(y)(X(X^TX)^{-1}X^T)^T\
            &={sigma}^2X(X^TX)^{-1}X^T(X(X^TX)^{-1}X^T)\
            &={sigma}^2Xbbox[5px,#ffd,border:0.25px solid green]{(X^TX)^{-1}X^TX}(X^TX)^{-1}X^T\
            &={sigma}^2IX(X^TX)^{-1}X^T\
            &={sigma}^2H  
end{align}
given $(X^TX)$ is Idempotent, $(AX)^T=X^TA^T$ and $H=X(X^TX)^{-1}X^T$.

Thomas Lumley · Answer

Moving a linear constant factor out of the variance involves multiplying by that factor twice.  So $mathrm{var}[2Y] = 4mathrm{var}[Y]$, and
$$ mathrm{var}[X(X'X)^{-1}X'Y]= X(X'X)^{-1}X'mathrm{var}[Y]X(X'X)^{-1}X'$$

Looking to build the Mathematical proof that $ Var(hat{y}) = sigma^2textbf{H} $

2 Answers

Add your own answers!

Ask a Question