standarddeviationcalculator.net

Updated Free · runs in your browser

Statistics

Shannon entropy calculator

Measure the uncertainty in a distribution. Enter probabilities, counts or a piece of text and the calculator gives the Shannon entropy with each −p log p term, the maximum possible entropy and how even the distribution is.

One count per category, e.g. individuals of each species.

Shannon entropy H (bits) 2.04145
Categories with p > 0 (S)5
Maximum entropy log₂ S2.32193
Evenness J = H / H_max0.8792
Effective number of categories e^H′4.1166
H in bits2.04145
H in nats (Shannon index H′)1.41502
CategoryCountplog₂ p−p log₂ p
1400.4−1.32190.52877
2250.25−20.5
3200.2−2.32190.46439
4100.1−3.32190.33219
550.05−4.32190.2161
0.4 1 0.25 2 0.2 3 0.1 4 0.05 5 p
Show the working, step by step

  1. Turn each count into a proportion by dividing by the total.

    p = n ÷ 100

  2. Multiply each proportion by its log₂, add them and change the sign.

    H = −Σ p log₂ p H = −(0.4·log₂(0.4) + 0.25·log₂(0.25) + 0.2·log₂(0.2) + 0.1·log₂(0.1) + 0.05·log₂(0.05)) H = 2.04145 bits

  3. Compare with the maximum, reached when all S categories are equally likely.

    H_max = log₂ 5 = 2.32193 J = 2.04145 ÷ 2.32193 = 0.8792

Categories with p = 0 add nothing, because p log p → 0 as p → 0. Ecologists usually report H′ with natural logs; information theory uses bits.

The formula

H = −Σ pᵢ log pᵢ H_max = log S J = H ÷ H_max

pᵢ is the probability (or proportion) of category i and S is the number of categories with pᵢ > 0. Categories with p = 0 contribute nothing, since p log p tends to 0. The base of the logarithm sets the unit.

A worked example from counts

The default is 100 observations in five categories with counts 40, 25, 20, 10 and 5, for example the individuals of five species in a sample. The proportions are 0.4, 0.25, 0.2, 0.1 and 0.05.

plog₂ p−p log₂ p
0.40−1.32190.5288
0.25−2.00000.5000
0.20−2.32190.4644
0.10−3.32190.3322
0.05−4.32190.2161

The terms add up to H = 2.041 bits. With five categories the maximum is log₂ 5 = 2.322 bits, so the evenness is 2.041 ÷ 2.322 = 0.879. In natural logs the same distribution has H′ = 1.415 nats, the figure an ecologist would report as the Shannon diversity index, and e^1.415 = 4.12 is the effective number of categories: the distribution is as uncertain as a fair choice among about 4.1 equally likely options.

Other inputs

With the probabilities 0.5, 0.25, 0.125 and 0.125 the entropy is exactly 1.75 bits, and a code that spends 1, 2, 3 and 3 bits on the four outcomes achieves that average length. In text mode each distinct character is a category. "hello world" has 11 characters, with l three times, o twice and the other six once each, giving 2.845 bits per character.

Interpreting the result

Higher entropy means outcomes are harder to predict. For a source of symbols, H in bits is a lower bound on the average code length per symbol when symbols are independent. For a community, higher H′ means more species, more evenly represented. Values of H′ in field studies commonly fall between about 1.5 and 3.5. Because H mixes richness and evenness, report S and J alongside it.

Entropy and sample size

The formula uses the observed proportions as if they were the true probabilities. With small samples this underestimates the entropy, because rare categories are often missed entirely. The Miller–Madow correction adds (S − 1) ÷ 2N nats to H′; for the default counts that is 4 ÷ 200 = 0.02 nats, negligible here but noticeable for samples of a few dozen.

Common mistakes

  • Mixing log bases. Bits and nats differ by a factor of ln 2 = 0.693.
  • Forgetting the minus sign, which gives a negative entropy.
  • Entering probabilities that do not sum to 1. The calculator rejects totals far from 1 and rescales small rounding gaps.
  • Comparing H′ across samples of very different sizes. Small samples miss rare categories and understate entropy.
Shannon entropy calculator: the worked example on this page, with its result and chart
Shannon entropy calculator: the worked example above, at a glance.

Common questions

What does Shannon entropy measure?

The average uncertainty, or surprise, in the outcome of a random variable. In information theory it is the average number of bits needed per symbol to encode messages from that source. In ecology the same formula, usually with natural logs, is the Shannon diversity index H′.

What is the difference between bits, nats and dits?

Only the base of the logarithm. Base 2 gives bits, base e gives nats and base 10 gives dits (also called hartleys or bans). To convert, 1 nat = 1.4427 bits and 1 dit = 3.3219 bits.

What is the maximum possible entropy?

log S, reached when all S categories are equally likely. A fair coin has 1 bit, a fair six-sided die log₂ 6 = 2.585 bits. Any imbalance lowers the entropy; a certain outcome has entropy 0.

What is Pielou evenness?

J = H ÷ log S, the entropy as a fraction of its maximum. It runs from 0 to 1 and does not depend on the log base. Ecologists use it to separate how many species there are from how evenly the individuals are spread among them.

Is the Shannon index the same as Simpson’s index?

No. Both measure diversity, but Shannon's H′ gives more weight to rare species and Simpson's index is dominated by the common ones. They often rank communities the same way, but not always.

Written and reviewed by our editorial team. Last updated . Method and sources: how these numbers are computed.