Kolmogorov–Sinai entropy

Summary

The Kolmogorov–Sinai entropy, or metric entropy, of a measure-preserving transformation measures how much new information each step of the map produces. It is defined from the Shannon entropy of finite partitions, refined along the orbit and taken per step. It is an isomorphism invariant. It tells apart systems with the same spectral data, such as two Bernoulli shifts whose probability vectors have different entropies. By the Ornstein isomorphism theorem it is a complete invariant for Bernoulli shifts.