< Back to Article

The two kinds of free energy and the Bayesian revolution

Fig 2

Minimizing the free energy from constraints (7) requires to trade off the competing terms of energy and entropy H(p), here shown exemplarily for the case of three elements.

Assuming there exists a unique minimal element , then minimizing only over all probability distributions p results in the (Dirac delta) distribution δ_z* that assigns zero probability to all z_i ≠ z* and probability one to z_i = z*, and therefore has zero entropy. In contrast, minimizing only the term is equivalent to maximizing H(p) and therefore would result in the uniform distribution that gives equal probability to all elements. The resulting Boltzmann distribution p* interpolates between these two extreme solutions of minimal energy (β → ∞) and maximum entropy (β → 0).

doi: https://doi.org/10.1371/journal.pcbi.1008420.g002