The exponential family of probability distributions is a unifying framework that encapsulates a broad class of continuous and discrete distributions. Entries under this tag explore natural parameters, sufficient statistics, conjugate priors, information geometry, and applications in Bayesian inference and machine learning.
A comprehensive introduction to the mathematical definition, canonical form, and key properties of the exponential family. Covers natural parameter space, base measures, and conditions for regularity.
Derives the canonical representation of the Bernoulli distribution, identifies its natural parameter (log-odds), and demonstrates conjugate updating with the Beta prior in Bayesian inference.
Explores how the normal distribution fits into the exponential family framework, detailing the natural parameter vector (μ/σ², -1/2σ²), sufficient statistics, and connections to precision matrices.
Explains why exponential family distributions naturally yield conjugate priors, derives the general form, and walks through practical examples including Dirichlet-Multinomial and Normal-Gamma models.
Examines the Poisson distribution's exponential family representation, moment generating functions, and its role in modeling count data. Includes links to overdispersion alternatives like Negative Binomial.
Connects exponential family distributions to GLMs, explaining canonical link functions, iterative reweighted least squares (IRLS), and assumptions for regression with non-Gaussian responses.
Details the relationship between exponential family structure, minimal sufficiency, and the Cramér-Rao lower bound. Includes derivations of Fisher information matrices and asymptotic normality.
Covers the Dirichlet as a conjugate prior for categorical and multinomial likelihoods, explores concentration parameters, posterior predictive distributions, and applications in topic modeling.