Normal-Wishart

Notation	$({\boldsymbol {\mu )),{\boldsymbol {\Lambda )))\sim {\mathrm {NW} }({\boldsymbol {\mu ))_{0},\lambda ,{\mathbf {W} },\nu )$
Parameters	${\boldsymbol {\mu ))_{0}\in {\mathbb {R} }^{D}\,$ location (vector of real) $\lambda >0\,$ (real) ${\mathbf {W} }\in {\mathbb {R} }^{D\times D}$ scale matrix (pos. def.) $\nu >D-1\,$ (real)
Support	${\boldsymbol {\mu ))\in {\mathbb {R} }^{D};{\boldsymbol {\Lambda ))\in {\mathbb {R} }^{D\times D}$ covariance matrix (pos. def.)
PDF	$f({\boldsymbol {\mu )),{\boldsymbol {\Lambda ))|{\boldsymbol {\mu ))_{0},\lambda ,{\mathbf {W} },\nu )={\mathcal {N))({\boldsymbol {\mu ))|{\boldsymbol {\mu ))_{0},(\lambda {\boldsymbol {\Lambda )))^{-1})\ {\mathcal {W))({\boldsymbol {\Lambda ))|{\mathbf {W} },\nu )$

In probability theory and statistics, the normal-Wishart distribution (or Gaussian-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and precision matrix (the inverse of the covariance matrix).^[1]

Definition

Suppose

${\displaystyle {\boldsymbol {\mu ))|{\boldsymbol {\mu ))_{0},\lambda ,{\boldsymbol {\Lambda ))\sim {\mathcal {N))({\boldsymbol {\mu ))_{0},(\lambda {\boldsymbol {\Lambda )))^{-1})}$

has a multivariate normal distribution with mean ${\boldsymbol {\mu ))_{0}$ and covariance matrix $(\lambda {\boldsymbol {\Lambda )))^{-1}$, where

${\boldsymbol {\Lambda ))|{\mathbf {W} },\nu \sim {\mathcal {W))({\boldsymbol {\Lambda ))|{\mathbf {W} },\nu )$

has a Wishart distribution. Then $({\boldsymbol {\mu )),{\boldsymbol {\Lambda )))$ has a normal-Wishart distribution, denoted as

$({\boldsymbol {\mu )),{\boldsymbol {\Lambda )))\sim {\mathrm {NW} }({\boldsymbol {\mu ))_{0},\lambda ,{\mathbf {W} },\nu ).$

Characterization

Probability density function

$f({\boldsymbol {\mu )),{\boldsymbol {\Lambda ))|{\boldsymbol {\mu ))_{0},\lambda ,{\mathbf {W} },\nu )={\mathcal {N))({\boldsymbol {\mu ))|{\boldsymbol {\mu ))_{0},(\lambda {\boldsymbol {\Lambda )))^{-1})\ {\mathcal {W))({\boldsymbol {\Lambda ))|{\mathbf {W} },\nu )$

Properties

Scaling

Marginal distributions

By construction, the marginal distribution over ${\boldsymbol {\Lambda ))$ is a Wishart distribution, and the conditional distribution over ${\boldsymbol {\mu ))$ given ${\boldsymbol {\Lambda ))$ is a multivariate normal distribution. The marginal distribution over ${\boldsymbol {\mu ))$ is a multivariate t-distribution.

Posterior distribution of the parameters

After making $n$ observations ${\displaystyle {\boldsymbol {x))_{1},\dots ,{\boldsymbol {x))_{n))$, the posterior distribution of the parameters is

${\displaystyle ({\boldsymbol {\mu )),{\boldsymbol {\Lambda )))\sim \mathrm {NW} ({\boldsymbol {\mu ))_{n},\lambda _{n},\mathbf {W} _{n},\nu _{n}),}$

where

${\displaystyle \lambda _{n}=\lambda +n,}$

${\displaystyle {\boldsymbol {\mu ))_{n}={\frac {\lambda {\boldsymbol {\mu ))_{0}+n{\boldsymbol {\bar {x)))){\lambda +n)),}$

${\displaystyle \nu _{n}=\nu +n,}$

${\displaystyle \mathbf {W} _{n}^{-1}=\mathbf {W} ^{-1}+\sum _{i=1}^{n}({\boldsymbol {x))_{i}-{\boldsymbol {\bar {x))})({\boldsymbol {x))_{i}-{\boldsymbol {\bar {x))})^{T}+{\frac {n\lambda }{n+\lambda ))({\boldsymbol {\bar {x))}-{\boldsymbol {\mu ))_{0})({\boldsymbol {\bar {x))}-{\boldsymbol {\mu ))_{0})^{T}.}$^[2]

Generating normal-Wishart random variates

Generation of random variates is straightforward:

1. Sample ${\boldsymbol {\Lambda ))$ from a Wishart distribution with parameters $\mathbf {W}$ and $\nu$
2. Sample ${\boldsymbol {\mu ))$ from a multivariate normal distribution with mean ${\boldsymbol {\mu ))_{0}$ and variance $(\lambda {\boldsymbol {\Lambda )))^{-1}$

Related distributions

• The normal-inverse Wishart distribution is essentially the same distribution parameterized by variance rather than precision.
• The normal-gamma distribution is the one-dimensional equivalent.
• The multivariate normal distribution and Wishart distribution are the component distributions out of which this distribution is made.