Home › Statistics › Probability Distributions › Wishart Distribution

Wishart Distribution

📐 Probability & Statistics 📅 Updated: Oct 2025 👁️ 48.2K views ✅ Peer-Reviewed

The Wishart distribution is a continuous probability distribution over real, symmetric, positive-definite matrices. It serves as the multivariate generalization of the chi-squared distribution and is named after the Scottish mathematician John Wishart, who introduced it in 1928[1]. The distribution plays a foundational role in multivariate statistical analysis, Bayesian inference, and modern machine learning algorithms.

If a random vector follows a multivariate normal distribution, the sample covariance matrix (scaled by the degrees of freedom) follows a Wishart distribution. This makes it indispensable for hypothesis testing, confidence region estimation, and covariance modeling in high-dimensional settings.

Mathematical Definition

A random matrix \( \mathbf{X} \) of dimension \( p \times p \) is said to follow a Wishart distribution with \( \nu \) degrees of freedom and scale matrix \( \mathbf{\Sigma} \), denoted as \( \mathbf{X} \sim W_p(\nu, \mathbf{\Sigma}) \), if its probability density function (PDF) is given by:

\[ f(\mathbf{X}) = \frac{1}{2^{\nu p / 2} |\mathbf{\Sigma}|^{\nu / 2} \Gamma_p(\nu / 2)} |\mathbf{X}|^{(\nu - p - 1) / 2} \exp\left( -\frac{1}{2} \operatorname{tr}(\mathbf{\Sigma}^{-1} \mathbf{X}) \right) \]

where \( \mathbf{X} \) is symmetric and positive-definite, \( \Gamma_p(\cdot) \) is the multivariate gamma function, and \( \operatorname{tr}( ) \) denotes the matrix trace. The support is the space of \( p \times p \) symmetric positive-definite matrices.

Parameters

  • Dimension \( p \): The size of the square matrices. Determines the number of variables in the underlying multivariate distribution.
  • Degrees of freedom \( \nu \): A scalar parameter such that \( \nu \geq p \) for the distribution to be proper. Controls the spread and concentration of the density.
  • Scale matrix \( \mathbf{\Sigma} \): A \( p \times p \) symmetric positive-definite matrix that acts as a covariance-like scaling factor. It is not necessarily the covariance matrix of \( \mathbf{X} \).

Key Properties

Expectation & Moments

The expected value of a Wishart-distributed matrix is straightforward:

\[ \mathbb{E}[\mathbf{X}] = \nu \mathbf{\Sigma} \]

The variance of the individual elements \( X_{ij} \) is given by:

\[ \operatorname{Var}(X_{ij}) = \nu \left( \Sigma_{ij}^2 + \Sigma_{ii}\Sigma_{jj} \right) \]

For \( \nu > p + 1 \), the mode (most probable matrix) is:

\[ \operatorname{mode}(\mathbf{X}) = (\nu - p - 1) \mathbf{\Sigma} \]

Additivity & Marginals

If \( \mathbf{X}_1 \sim W_p(\nu_1, \mathbf{\Sigma}) \) and \( \mathbf{X}_2 \sim W_p(\nu_2, \mathbf{\Sigma}) \) are independent, then \( \mathbf{X}_1 + \mathbf{X}_2 \sim W_p(\nu_1 + \nu_2, \mathbf{\Sigma}) \). Submatrices of a Wishart-distributed matrix also follow a Wishart distribution with reduced dimension and the same scale submatrix.

Related Distributions

  • Chi-squared distribution: When \( p = 1 \), the Wishart distribution reduces to a scaled chi-squared distribution: \( W_1(\nu, \sigma^2) \equiv \sigma^2 \chi^2_\nu \).
  • Inverse Wishart distribution: If \( \mathbf{X} \sim W_p(\nu, \mathbf{\Sigma}) \), then \( \mathbf{X}^{-1} \sim \mathcal{IW}_p(\nu, \mathbf{\Sigma}^{-1}) \). It is the conjugate prior for the covariance matrix in Bayesian multivariate normal models.
  • Multivariate normal distribution: The Wishart distribution naturally arises as the sampling distribution of the sum of outer products of multivariate normal vectors.
  • Multivariate t-distribution: Arises from marginalizing a multivariate normal distribution with an Inverse Wishart prior on the covariance matrix.

Applications

The Wishart distribution is extensively used across quantitative disciplines:

  • Bayesian Inference: Conjugate prior for covariance matrices in multivariate normal models, enabling closed-form posterior updates.
  • Signal Processing: Modeling sample covariance matrices in radar, wireless communications, and array processing (e.g., spatial covariance estimation).
  • Finance & Econometrics: Modeling time-varying volatility and correlation structures in portfolio optimization and risk management.
  • Machine Learning: Used in probabilistic PCA, factor analysis, Gaussian mixture models with covariance sharing, and Bayesian neural networks.
  • Hypothesis Testing: Central to Wilks' lambda, Hotelling's \( T^2 \), and other multivariate test statistics.

Sampling & Generation

Direct sampling from the Wishart distribution is typically performed via the Bartlett decomposition. If \( \mathbf{L} \) is the Cholesky factor of \( \mathbf{\Sigma} \) (i.e., \( \mathbf{\Sigma} = \mathbf{L}\mathbf{L}^T \)), then \( \mathbf{X} = \mathbf{L} \mathbf{A} \mathbf{A}^T \mathbf{L}^T \), where \( \mathbf{A} \) is a lower-triangular matrix with independent entries:

\[ A_{ij} \sim \begin{cases} \mathcal{N}(0, 1) & i > j \\ \sqrt{\chi^2_{\nu - i + 1}} & i = j \end{cases} \]

Algorithmically, this is efficient and numerically stable. Modern libraries (e.g., NumPy, SciPy, PyTorch, Stan) implement optimized routines based on this decomposition.

References & Further Reading

  1. Wishart, J. (1928). "The Generalized Distance in Statistics". Philosophical Transactions of the Royal Society A, 221(594-604), 31-52.
  2. Muirhead, R. J. (1982). Aspects of Multivariate Statistical Theory. Wiley.
  3. Gelman, A., et al. (2020). Bayesian Data Analysis (3rd ed.). CRC Press. (Ch. 5.6)
  4. Kotz, S., Balakrishnan, N., & Johnson, N. L. (2003). Continuous Multivariate Distributions, Vol. 1. Wiley.
  5. Anderson, T. W. (2003). An Introduction to Multivariate Statistical Analysis (3rd ed.). Wiley.
Multivariate Statistics Probability Theory Bayesian Inference Covariance Modeling Matrix Distributions