Asymptotic equipartition property
The
asymptotic equipartition property (AEP) is a direct consequence of the weak
law of large numbers and is used extensively in
information theory. It can be summed up by the phrase 'Almost everything is almost equally probable.'
Given independent identically-distributed random variables
X1, ..., Xn with entropy H(X) and p(Xj, ..., Xk) the probability of observing the sequence X1, ..., Xk, the AEP states that in probability,
The AEP is used to define the
typical set, which is used extensively in theories of
compression.