Do information entropy and thermodynamic entropy evolve the same way?

Question

Recently I asked a question about the relationship between Von Neumann entropy and heat transfer. The link is here.

Is the Von Neumann entropy related to heat transfer?

I feel as though the answer is not quite so clear so I will ask a couple of questions that I think may essentially be the same question as this previous one or, if these new questions aren't the same as the old question, your answers will help me clarify some key points.

Original question: Is the change in Von Neumann entropy related to heat transfer? I think I can rephrase and ask if information theory can explain heat transfer in terms of evolving information.

New Question #1:

The Wikipedia page "Maximum entropy thermodynamics" says under "Second Law": (paraphrased) Liouville's equation shows how the phase space evolves from a maximum entropy distribution to one that is not necessarily maximum entropy, yet the thermodynamic entropy is certainly maximized.

Does this mean what I think it does: that the evolution of information entropy and thermodynamic entropy are not the same? When I see people comparing information and thermodynamic entropy they always do it for a single state. But I don't think I've ever seen a discussion of whether, if the state evolves, the two entropies evolve to the same value. The Wiki article suggests to me that they don't. So, if they don't behave exactly the same, aren't they different things?

Here's the link:  https://en.wikipedia.org/wiki/Maximum_entropy_thermodynamics

New Question #2:

Another Wiki article "Entropy in thermodynamics" says, under "Criticisms": (paraphrased) information has no concept of temperature or energy so thermodynamic and information entropies are not the same. Does anybody disagree with this?

Here's the link: https://en.wikipedia.org/wiki/Entropy_in_thermodynamics_and_information_theory#Criticism

Themis · Answer

Question 1

Liouville's equation shows how the phase space evolves from a maximum entropy distribution to one that is not necessarily maximum entropy [...]

I cannot find a reference to Liouville's equation in Wikipedia's entry on the second law. In fact, the above statement is wrong: Liouville's equation preserves entropy at all times, that is, the Liouville equation is capable of describing only an isentropic process.

Question 2

Another Wiki article "Entropy in thermodynamics" says, under "Criticisms": (paraphrased) information has no concept of temperature or energy so thermodynamic and information entropies are not the same. Does anybody disagree with this?

The criticism is unfounded on several levels:

Shannon only claimed to have identified the entropy functional. He was not interested in "energy" or "temperature". That doesn't mean that these concepts cannot exist within the formalism of information theory. 
"Temperature" and "energy" are just names of variables that appear in certain physical problems. Jaynes realized this and drew the connection between  Information Theory and Statistical Mechanics. Briefly, any positive variable $x$ whose mean is known is "energy"; "temperature" arises as a Lagrange multiplier when one seeks the probability distribution of $x$ that maximizes entropy given that its mean is known. 
It is possible to recover the entire calculus of thermodynamics starting with the notion of the most probable distribution (https://arxiv.org/abs/1809.07367).

Do information entropy and thermodynamic entropy evolve the same way?

One Answer

Add your own answers!

Ask a Question