Bernoulli Distribution

📐 Discrete Probability Distribution 🕒 Last Updated: March 2025 👁️ 142K Views

Distribution Parameters

TypeDiscrete

Supportk ∈ {0, 1}

Parameterp ∈ (0, 1]

Meanp

Variancep(1 − p)

Skewness\(\frac{1-2p}{\sqrt{p(1-p)}}\)

Kurtosis3 - \(\frac{1}{p(1-p)}\)

Entropy\(-p\ln p - (1-p)\ln(1-p)\)

Overview

The Bernoulli distribution is a discrete probability distribution named after Swiss mathematician Jacob Bernoulli. It models a single experiment or trial that results in exactly two possible outcomes: success (typically encoded as 1) with probability \(p\), and failure (typically encoded as 0) with probability \(1 - p\). It serves as the foundational building block for many more complex distributions in probability theory, statistics, and machine learning.

Common real-world examples include coin tosses, pass/fail tests, system uptime checks, and binary classification predictions. When \(p = 0.5\), the distribution is uniform over \(\{0, 1\}\) and is often referred to as a fair coin flip.

Mathematical Definition

Probability Mass Function (PMF)

For a random variable \(X\) following a Bernoulli distribution with parameter \(p\), the probability mass function is defined as:

P(X = k) = p^k (1 - p)^{1 - k} \quad \text{for } k \in \{0, 1\}

Equivalently, using indicator notation:

P(X = 1) = p, \quad P(X = 0) = 1 - p

Cumulative Distribution Function (CDF)

The cumulative distribution function describes the probability that \(X\) takes a value less than or equal to \(x\):

F(x) = \begin{cases} 0 & x < 0 \\ 1 - p & 0 \le x < 1 \\ 1 & x \ge 1 \end{cases}

Key Properties

The Bernoulli distribution exhibits several fundamental statistical properties that make it mathematically tractable and widely applicable:

Expected Value: \(\mathbb{E}[X] = p\)
Variance: \(\text{Var}(X) = p(1 - p)\)
Moment Generating Function: \(M_X(t) = 1 - p + pe^t\)
Characteristic Function: \(\phi_X(t) = 1 - p + pe^{it}\)

Note: The variance \(p(1-p)\) is maximized when \(p = 0.5\), yielding \(\text{Var}(X) = 0.25\). This reflects maximum uncertainty in a binary outcome.

Relations to Other Distributions

Bernoulli & Binomial

If \(X_1, X_2, \dots, X_n\) are independent and identically distributed (i.i.d.) Bernoulli random variables with parameter \(p\), then their sum follows a binomial distribution:

Y = \sum_{i=1}^{n} X_i \sim \text{Binomial}(n, p)

This relationship demonstrates that the binomial distribution is essentially a sum of \(n\) Bernoulli trials.

Bernoulli & Categorical

The Bernoulli distribution is a special case of the categorical distribution where the number of categories \(k = 2\). It is also equivalent to the two-parameter multinomial distribution restricted to a single trial.

Applications

The Bernoulli distribution underpins numerous fields across science, engineering, and data science:

Machine Learning: Binary classification models (logistic regression, neural networks) output Bernoulli-distributed probabilities for class membership.
Quality Control: Modeling defect/non-defect outcomes in manufacturing processes.
Finance: Default/no-default modeling in credit risk assessment.
Medical Trials: Success/failure of treatments in clinical studies.
Information Theory: Fundamental to entropy calculations and binary source coding.

Worked Example

Consider a new drug trial where the probability of a patient experiencing a positive response is \(p = 0.65\). Let \(X\) represent the outcome for a single patient.

P(X = 1) = 0.65, \quad P(X = 0) = 1 - 0.65 = 0.35 \] \[ \mathbb{E}[X] = 0.65, \quad \text{Var}(X) = 0.65 \times 0.35 = 0.2275

If 100 patients are tested independently, the number of successful responses \(Y\) follows \(\text{Binomial}(100, 0.65)\), with expected successes \(\mathbb{E}[Y] = 65\) and variance \(\text{Var}(Y) = 22.75\).

References

Bernoulli, J. (1713). Ars Coniendandi (The Art of Conjecturing). Posthumously published 1713.
Feller, W. (1968). An Introduction to Probability Theory and Its Applications, Vol. I (3rd ed.). Wiley.
Casella, G., & Berger, R. L. (2002). Statistical Inference (2nd ed.). Duxbury Press.
Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective . MIT Press.
Aevum Encyclopedia Editorial Board. (2024). Foundations of Discrete Distributions . Aevum Press.