Derivative of vectorized function wrt to a Cholesky decompositiion

Question

Let $Sigma$ be a symmetric, positive definite $ptimes p$ covariance matrix, and let $f(Sigma)$ be it's Cholesky factor. That is, $f(Sigma)$ is a lower triangular $ptimes p$ matrix such that $Sigma = f(Sigma) f(Sigma)^{top}$. Further let $Lambda := operatorname{diag}(f(Sigma))$ be a diagonal matrix holding the diagonal elements of $f(Sigma)$ on its diagonal, i.e. the standard deviations given by $Sigma$, and finally, let $P = Lambda^{-1} Sigma Lambda^{-1}$ denote the correlation matrix.
I am wondering if, with $mathcal{P} := P - I_p + Lambda$, the derivative
$$
frac{mathrm{d}operatorname{vec}left( mathcal{P} right)}{mathrm{d} operatorname{vec} left( f(Sigma) right)} 
$$
is known, where $operatorname{vec}$ is the vectorization function and $I_p$ the $p$-dimensional identity matrix.
I found questions answering related questions, as for example here and here and here; however due to my limited knowledge of matrix calculus I don't know how to combine these sources nor if a closed form solution exists.

greg · Answer

Let's use a naming convention where matrices and vectors are denoted
by upper and lower case Latin letters, respectively. Further, the symbol $odot$ will denote the Hadamard product and $otimes$ the Kronecker product.
Finally, let's use $X$ as the independent variable.
Then rewrite the problem using this convention.
$$eqalign{
S &= XX^T,quad 
A = Iodot X,quad 
V=A^{-1} \
P &= VSV + A - I \
}$$
Each of these matrices (except for $X$) is symmetric, and $(A,V,I)$ are diagonal.
Apply the vec operation ($K$ denotes the Commutation matrix)
$$eqalign{
y &= {rm vec}(I) \
x &= {rm vec}(X) quadimpliesquad{rm vec}(X^T) &doteq Kx \
a &= y odot x 
 ;=; {rm Diag}(y),x 
 &doteq Yx \
da &= Y,dx \
\
s &= (Iotimes X)Kx ;=; (Xotimes I),x \
ds &= Big((Iotimes X)K+(Xotimes I)Big),dx &doteq N,dx\
\
p &= (Votimes V)s + a-y &doteq Bs + a-y \
 &= (VSotimes I)v + a-y &doteq Hv + a-y  \
 &= (Iotimes VS)v + a-y &doteq Jv + a-y  \
}$$
Finally, calculate the differentials of $v$ and $p$
$$eqalign{
dv &= {rm vec}(-V,dA,V) = -(Votimes V),da \
   &= -B,da ;=; -BY,dx \
\
dp &= da + B,ds + H,dv + J,dv \
   &= Y,dx + BN,dx - (H+J)BY,dx \
   &= Big(Y + BN - (H+J)BYBig),dx \
}$$
and the gradient with respect to $x$
$$eqalign{
frac{partial p}{partial x}
   &= Y + BN - (H+J)BY \
   &= Y
    + (Votimes V)Big((Iotimes X)K+(Xotimes I)Big)
    - (VSotimes I + Iotimes VS)(Votimes V)Y \
   &= Y + (Votimes VX)K+(VXotimes V)
    - big(VSVotimes V + Votimes VSVbig)Y \
}$$

Derivative of vectorized function wrt to a Cholesky decompositiion

One Answer

Add your own answers!

Ask a Question