scipy.stats.entropy

scipy.stats.entropy(pk, qk=None, base=None, axis=0)[source]

Calculate the entropy of a distribution for given probability values.

If only probabilities pk are given, the entropy is calculated as S = -sum(pk * log(pk), axis=axis).

If qk is not None, then compute the Kullback-Leibler divergence S = sum(pk * log(pk / qk), axis=axis).

This routine will normalize pk and qk if they don’t sum to 1.

Parameters
pksequence

Defines the (discrete) distribution. pk[i] is the (possibly unnormalized) probability of event i.

qksequence, optional

Sequence against which the relative entropy is computed. Should be in the same format as pk.

basefloat, optional

The logarithmic base to use, defaults to e (natural logarithm).

axis: int, optional

The axis along which the entropy is calculated. Default is 0.

Returns
Sfloat

The calculated entropy.

Examples

>>> from scipy.stats import entropy

Bernoulli trial with different p. The outcome of a fair coin is the most uncertain:

>>> entropy([1/2, 1/2], base=2)
1.0

The outcome of a biased coin is less uncertain:

>>> entropy([9/10, 1/10], base=2)
0.46899559358928117

Relative entropy:

>>> entropy([1/2, 1/2], qk=[9/10, 1/10])
0.5108256237659907
scipy.stats.mvsdist scipy.stats.differential_entropy