Statistical physics and quantum mechanics

In classical physics,  the state of a system is the juxtaposition of the states of subsystems.  That means that the state of a 2-particle system is the state of a 1-particle system and another state of a 1-particle system.  The state of a 5-particle system consists of 5 sub-states, each describing one particle.

In quantum physics, that doesn't hold any more.  The state of the system describes the whole system and is more involved than just the juxtaposition of the states of the individual states.  In quantum theory, a sub-system doesn't have a well-defined state in general.  The property that makes this happen, is called "entanglement".  A pure quantum state can only be defined for the whole system, as components of the system can get entangled.  A typical example is a quantum system made up of two spins.  If sub-system A has spin-1/2 and sub-system B has spin-1/2 and they are both part of an overall 2-spin system AB, then AB can be in a singlet state, or in one of three different triplet states.  The singlet state of AB doesn't assign a precise state to A, nor to B.  In fact, the singlet state AB is a combination of the up-state of A, and the down state of B, together with the down-state of A, together with the up state of B.  The second triplet state is that too, but with different phases between the components.  So a pure quantum state only pertains to the whole system.

Technically, a pure quantum state can be described by its projector operator on the quantum state: | ψ > < ψ |

Of course, just as was the case with classical statistical physics, we can consider a statistical ensemble of pure quantum states, instead of one well-determined pure quantum state.  We can say that the system is in quantum state U with probability p1, is in quantum state V with probability p2 and so on.  Technically, such an ensemble can be described by a density operator:

ρ = Σ pi | ψi > < ψi |

The above property is very remarkable: an ensemble of different quantum states has itself a quantum-mechanical description.  This is entirely different in classical physics, where an "ensemble of classical states" doesn't have a classical description as a mechanical state itself.  In quantum theory, an ensemble of pure quantum states can be "incorporated" in the formalism very easily.   Such a kind of quantum state, which describes a whole ensemble, is called a mixed state.

In fact, quantum mechanics comes to the rescue of a problem in classical statistical physics.  Indeed, we said that the entropy of a classical system is given by the Gibbs formula.  However, for that to work, one would need a finite, discrete set of different states in the ensemble. But classical states have a continuous description.  We didn't mention that, and it is usually circumvented by imposing a finite "resolution" on position and velocity (in other terms, by considering elementary volumes in classical phase space).  In quantum theory, on the other hand, the many pure quantum states of a system are, in as much as that system has finite spatial extend, discrete.   As such, there is a genuine discrete, finite number of possible states in any ensemble, and the Gibbs formula can work without any trick such as imposing a finite resolution on space and velocity.  There are a finite number of states.  Such states are pure states of the whole system.  A quantum system in a (known) pure state has entropy 0.  If it can be in several pure states, then it is actually in a mixed state.  Gibbs formula still holds:

S = kB. Σ pi ln(1/pi)

where the probabilities are those making up the statistical mixture of quantum states.  In fact there is an equivalent formula:

S = - kB tr [ ρ ln(ρ) ] in thermodynamic units, or

S = - tr [ ρ log2(ρ) ] in bits

Earlier, we said that a pure quantum state only makes sense "for the whole system".  However, this is a very strange concept, as "system" is something that we have confined to some arbitrarily chosen part of the physical world.  We could have chosen a part of that part as being "the system", but we said that a pure quantum state doesn't necessarily make sense for a subsystem alone.

However, if a "system" is in a pure state, and we only consider a sub-system of that system, we can still use a quantum description of only that sub-part.

We have that the quantum-mechanical description of a sub-system A of a whole system AB is given by:

ρA = trB [ ρ ]

Here, ρ is the density matrix of the whole system (which may very well describe a pure state of the whole system), and trB is the partial trace, which takes the trace only over the degrees of freedom of the sub-system B.  ρA can be the density operator of a genuine mixture.

This is the way mixtures can be seen as resulting from looking only at a sub-system of an entangled bigger system where we do not look into the rest.  This is also what happens with a measurement.  During a measurement, a quantum system interacts, and gets entangled with the measurement device and with everything else in the neighbourhood of that measurement device.    This turns the quantum state of the system into a mixture: the probabilities of the outcomes of the measurement!

This is why a quantum measurement has an irreducible entropy before we know the result of the measurement.   A quantum measurement is the "purest form" of the rolling of a dice and the information entropy related to it.

In fact, the notions of entropy, ensemble and information have a much more natural link with physics in quantum physics than in classical physics.

The difficulty of maintaining a quantum system in a pure state are related with the fact that from the moment that the system has the slightest interaction with its environment, one cannot consider the system by itself any more, and once the environment is involved, the "things one needs to include in the system" grow without any control until it includes the whole part of the universe within the forward light cone of the first interaction of the system with the environment.  The original system is now not in a pure state any more, but in the mixed state of a sub-system.  One says that decoherence has set in.

Decoherence always implies the generation of entropy.

Entropy, gravity and black holes

The most enigmatic application of information and entropy to physics is probably the link between entropy and the size of a black hole.  There seemed to be a problem with black holes, as it can be shown that a black hole has only 3 properties: mass, (electrical) charge, and angular momentum.  A black hole has a certain mass, is rotating in a certain way, and may be electrically charged.  As such, it wasn't clear, if heat went into a black hole, where the entropy went.  To make a long story short, if we have a black hole of mass M, then the Schwarzschild radius (the radius of the event horizon) is given by:

RS = 2 G M / c2  with G Newton's gravity constant, and c the velocity of light in vacuum.

If we define the event horizon surface as A = 4 π (RS)2 then it can be shown that if we take as (thermodynamic) entropy of a black hole:

S = kB A c3 / ( 4 G h ) with h Planck's constant, and as temperature:

T = h c3 /  (8 π G M kB),

the we see that if we add energy ΔE = c2 ΔM to a black hole, that ΔA = 32 π G2 M / c6  ΔE from which one has:

ΔS = 8 π G M  kB / (  h c3  ) Δ E =  ΔE / T  as given by the second law of thermodynamics.

Apart from unit conversion constants, we see that the entropy of a black hole is the surface of its event horizon, its temperature is the inverse of its mass, and its entropy grows with every amount of energy that goes (irreversibly) into it, as if it were a heat reservoir.

It can be shown that there cannot be any larger concentration of entropy than the entropy of a black hole, so a black hole is the ultimate density of information.

Conclusion

Information and its associated concept of ignorance (entropy) which didn't enter the realm of physics until the end of the 19th century pop up as a unifying concept throughout the most modern and diverse aspects of modern physics.  From an elusive and abstract quantity in the theory of heat (thermodynamics) to the geometrical properties of black holes and the decoherence and measurement in quantum mechanics, information seems to be present now as a fundamental concept in many diverse branches of physics.