Quantum relative entropy with respect to a pure state

Question

I want to evalualte the quantum relative entropy $S(rho|| sigma)=-{rm tr}(rho {rm log}(sigma))-S(rho)$, where $sigma=|PsiranglelanglePsi|$ is a density matrix corresponding to a pure state and $rho$ is a density matrix corresponding to an arbitrary mixed state. Here, $S(rho)$ simply denotes the Von Neumann entropy of $rho$. Given that $sigma$ is diagonal, with eigenvalues $0$ and $1$ it seems that the first term in the quantum relative entropy will in general be infinite. As $S(rho)leq {rm log}(d)$, where $rho in L({mathcal H}^{d})$, the first term dominates and the quantum relative entropy is also infinite. Is this correct? And if so, what's the intuition behind this fact?

Angelo Lucia · Accepted Answer

If $sigma$ is not full rank, then the correct way to interpret the quantum relative entropy formula you wrote is to assign it the value of $+infty$ when the support of $rho$ is not included in the support of $sigma$. Wikipedia has a nice explanation of how to interpret this, but you can think that the reason for which the quantum relative entropy is finite in that case is that $lim_{xto 0} x log(x) = 0$.
In your case, the support of $sigma$ is simply the one-dimensional subspace spanned by $|Psirangle$. For $rho$ to be supported on this subspace, it must hold that $rho$ is also pure, so it must be also equal to $|Psirangle!langlePsi|$, and so $rho=sigma$. In this case, the quantum relative entropy vanishes.
So in summary, if $sigma$ is pure, than $S(rho||sigma)$ is either $0$ (when $rho=sigma$) or $+infty$ (in the other cases).

Rammus · Answer

I'd like to add to Angelo Lucia's answer slightly. It is not very surprising that $S(rho | sigma)$ can take the value $+infty$ once we realise that the relative entropy is a generalization of the Kullback-Liebler divergence $D(p | q)$ between probability distributions $p$ and $q$. Formally, given two distributions $p,q$ over some finite set $mathcal{X}$ the KL-divergence is defined as
$$
D(p| q) = begin{cases}
sum_{x in mathcal{x}} p(x) log frac{p(x)}{q(x)} quad & text{if } mathrm{supp}(p) subseteq mathrm{supp}(q) 
+ infty & text{otherwise}
end{cases}
$$
where $mathrm{supp}(p) = {x in mathcal{X} : p(x)> 0}$. Note that if we fix a basis and consider only diagonal states in that basis, i.e. $rho = sum_{x} p(x) |xrangle langle x |$ and $sigma = sum_x q(x) |xrangle langle x |$, then computing $S(rho | sigma)$ we recover exactly the KL-divergence $D(p | q)$. The analogous situation to taking $sigma$ to be a pure state is taking $q$ to be some point distribution (delta-distribution). In this case we see that $D(p| q)$ is finite iff $p=q$, which is exactly what we observe in the quantum case for pure states.

Quantum relative entropy with respect to a pure state

2 Answers

Add your own answers!

Ask a Question