Non-informative prior distribution

TL;DR

Used in Bayesian inference when little or no prior knowledge about outcomes is available.
Common examples include the uniform distribution and the Jeffreys prior.
Provides an objective baseline for estimation and comparison, but can be inappropriate when substantial prior information exists.

Definition

A non-informative prior distribution is a type of probability distribution used in Bayesian statistics that does not contain any information about the likelihood of certain events occurring.

Explanation

Non-informative priors are chosen to reflect a lack of prior knowledge about the parameters or outcomes of interest. They are applied when it is not possible or desirable to encode prior beliefs into the analysis. By avoiding informative assumptions, these priors aim to produce estimates that are less influenced by subjective beliefs.

The uniform distribution is a simple non-informative prior that assigns equal probability to all outcomes within a specified range. The Jeffreys prior is another example; the source describes it as based on the principle of maximum entropy and commonly used when no prior knowledge is available.

Advantages noted in the source include enabling more objective and unbiased probability estimates when prior knowledge is limited and providing a baseline for comparing different estimates (for example, comparing experts). Limitations described in the source include reduced accuracy when substantial information exists and inappropriateness when significant prior knowledge is available—situations in which more informative priors may better reflect existing knowledge.

Examples

Uniform distribution

The uniform distribution assigns equal probability to all outcomes within a given range. Example from the source: when estimating the probability of a coin landing heads or tails with no prior knowledge, a uniform prior would assign a probability of 0.5 to both heads and tails.

Jeffreys prior

The Jeffreys prior is presented in the source as a commonly used non-informative prior and is described as being based on the principle of maximum entropy.

Use cases

Applied when there is little or no prior knowledge about the likelihood of outcomes.
Used as a baseline to compare different probability estimates (for example, comparing estimates from different experts).
Employed in highly uncertain prediction tasks cited in the source, such as predicting the outcome of an election or the performance of a new product on the market.

Notes or pitfalls

May not provide the most accurate estimates when a significant amount of information about the likelihood of events is available.
May be inappropriate when substantial prior knowledge exists; in such cases, a more informative prior may better reflect existing knowledge.

Bayesian statistics
Uniform distribution
Jeffreys prior
Maximum entropy