What is the derivative of $log det X$ when $X$ is symmetric?

$ defp#1#2{frac{partial #1}{partial #2}} defg#1#2{p{#1}{#2}} defm#1{left[begin{array}{r}#1end{array}right]} $To summarize the example used in the accepted answer $$eqalign{ phi(A) &= logdet(A) = log(ab-x^2) \ A &= m{a&x\x&b} quadimpliesquad g{phi}{A} = frac{1}{ab-x^2}m{b&-2x\-2x&a} \ }$$ Let's use this in a first-order Taylor expansion (use a colon to denote the matrix inner product) $$eqalign{ phi(A+dA) &= phi(A) + g{phi}{A}:dA \ dphi &= g{phi}{A}:dA \ &= frac{1}{ab-x^2}m{b&-2x\-2x&a}:m{da&dx\dx&db} \ &= frac{a,db-4x,dx+b,da}{ab-x^2} \ }$$ which disagrees with the direct (non-matrix) calculation $$eqalign{ dlog(ab-x^2) &= frac{a,db-2x,dx+b,da}{ab-x^2} \ }$$ On the other hand, using Boyd's result for the matrix calculation yields $$eqalign{ dphi &= g{phi}{A}:dA = frac{1}{ab-x^2}m{b&-x\-x&a}:m{da&dx\dx&db} = frac{a,db-2x,dx+b,da}{ab-x^2} \ }$$ which is correct.

Carefully read the Srinivasan-Panda paper (which has been mentioned in other answers) for an explanation of why Harville (and many other references) are mistaken.

Harville's quantity may be useful in certain contexts, but it is not a gradient.

This is a really well done paper that describes what is going on:

Shriram Srinivasan, Nishant Panda. (2020) "What is the gradient of a scalar function of a symmetric matrix?" https://arxiv.org/pdf/1911.06491.pdf

Their conclusion is that Boyd's formula is the correct one, which comes by restricting the Frechet derivative (defined in $mathbb{R}^{n times n}$) to the subspace of symmetric n x n matrices, denoted $mathbb{S}^{n times n}$. Deriving the gradient in the reduced space of $n(n+1)/2$ dimensions and then mapping back to $mathbb{S}^{n times n}$ is subtle and can't be done so simply, leading to the inconsistent result by Harville.

Let me call $X_0$ the symmetric matrix with entries $(X_0)_{i,j} = x_{i,j}$. We have by assumptions $x_{i,j}=x_{j,i}$. Since $X_0$ is symmetric it can be diagonalized (if it's real). Its determinant is the product of the eigenvalues $lambda_k$. So for a symmetric matrix $X$

$$ lndet X = sum_k ln(lambda_k ) $$

Assume $X$ depends on a parameter $t$. It's derivative would be

$$ frac{d}{dt} lndet X(t) = sum_k frac{dot{lambda}_k}{lambda_k} $$

Say we want the derivative of $X_0$ with respect to $x_{i,j}$ for $ineq j$. Then, defining

begin{align} V &= |irangle langle j | + |jrangle langle i | \ X(t) &= X_0 +tV, end{align}

($V$ is the matrix with all zeros except ones at position $(i,j)$ and $(j,i)$). We have

$$ frac{partial}{partial x_{i,j}} lndet X_0 = left . frac{d}{dt} lndet X(t) right vert_{t=0}= sum_k frac{dot{lambda}_k}{lambda_k} $$

Now

$$ dot{lambda}_k = langle v_k | V| v_k rangle $$

where $|v_k rangle$ is the eigenvector of $X_0$ corresponding to $lambda_k$. Hence (for $ineq j$)

begin{align} frac{partial}{partial x_{i,j}} lndet X_0 & = sum_k frac{ langle j| v_k rangle langle v_k |i rangle }{lambda_k} + i leftrightarrow j \ &= left ( X^{-1} right)_{j,i} +left ( X^{-1} right)_{i,j} \ &= 2left ( X^{-1} right)_{i,j} end{align}

Let us now compute the derivative with respect to $x_{i,i}$. We reason exactly as before with $V = |irangle langle i |$ and we get

begin{align} frac{partial}{partial x_{i,i}} lndet X_0 & = sum_k frac{ langle i| v_k rangle langle v_k |i rangle }{lambda_k} \ &= left ( X^{-1} right)_{i,i}. end{align}

Hence the second formula is the correct one for a symmetric matrix. The first formula is correct for a non symmetric matrix. All formulae require of course the matrix to be non-singular.

Added

Let's explain the subtlety with one example that should clarify the matter. Conside the following symmetric matrix:

$$ A=left(begin{array}{cc} a & x\ x & b end{array}right) $$

Now,

$$logdet(A) = log(ab-x^2)$$

and so

begin{align} frac{partial logdet(A)}{partial a } &= frac{b}{ab-x^2} \ frac{partial logdet(A)}{partial x } &= - frac{2x}{ab-x^2} \ frac{partial logdet(A)}{partial b } &= frac{a}{ab-x^2} end{align}

And compare this with

$$ A^{-1} = frac{1}{(ab-x^2)} left(begin{array}{cc} b & -x\ -x & a end{array}right) $$

This simple calculation agrees with the formula above (cfr. the factor of 2). As I said in the comment, the point is to be clear about what are the independent variables or what is the variation that we are using. Here I considered variation $V$ which is symmetric, as this seems to be the problem's assumption.

Obviously if you consider

$$ A'=left(begin{array}{cc} a & y\ x & b end{array}right) $$

you will obtain $nabla A' sim {A'}^{-1}$