The compound distribution ("unconditional distribution") is the result of marginalizing (integrating) over the latent random variable(s) representing the parameter(s) of the parametrized distribution ("conditional distribution").
A compound probability distribution is the probability distribution that results from assuming that a random variable is distributed according to some parametrized distribution with an unknown parameter that is again distributed according to some other distribution . The resulting distribution is said to be the distribution that results from compounding with . The parameter's distribution is also called the mixing distribution or latent distribution. Technically, the unconditional distribution results from marginalizing over , i.e., from integrating out the unknown parameter(s) . Its probability density function is given by:
The same formula applies analogously if some or all of the variables are vectors.
From the above formula, one can see that a compound distribution essentially is a special case of a marginal distribution: The joint distribution of and is given by
, and the compound results as its marginal distribution:
If the domain of is discrete, then the distribution is again a special case of a mixture distribution.
The compound distribution will depend on the specific expression of each distribution, as well as which parameter of is distributed according to the distribution , and the parameters of will include any parameters of that are not marginalized, or integrated, out.
The support of is the same as that of , and if the latter is a two-parameter distribution parameterized with the mean and variance, some general properties exist.
The compound distribution's first two moments are given by:
Convolution of probability distributions (to derive the probability distribution of sums of random variables) may also be seen as a special case of compounding; here the sum's distribution essentially results from considering one summand as a random location parameter for the other summand.
Compound distributions derived from exponential family distributions often have a closed form.
If analytical integration is not possible, numerical methods may be necessary.
Compound distributions may relatively easily be investigated using Monte Carlo methods, i.e., by generating random samples. It is often easy to generate random numbers from the
distributions as well as and then utilize these to perform collapsed Gibbs sampling to generate samples from .
A compound distribution may usually also be approximated to a sufficient degree by a mixture distribution using a finite number of mixture components, allowing to derive approximate density, distribution function etc.
Compounding a Bernoulli distribution with probability of success distributed according to a distribution that has a defined expected value yields a Bernoulli distribution with success probability . An interesting consequence is that the dispersion of does not influence the dispersion of the resulting compound distribution.
^Gelman, A.; Carlin, J. B.; Stern, H.; Rubin, D. B. (1997). "9.5 Finding marginal posterior modes using EM and related algorithms". Bayesian Data Analysis (1st ed.). Boca Raton: Chapman & Hall / CRC. p. 276.
^Gneiting, T. (1997). "Normal scale mixtures and dual probability densities". Journal of Statistical Computation and Simulation. 59 (4): 375–384. doi:10.1080/00949659708811867.
^Mood, A. M.; Graybill, F. A.; Boes, D. C. (1974). Introduction to the theory of statistics (3rd ed.). New York: McGraw-Hill.
^Johnson, N. L.; Kemp, A. W.; Kotz, S. (2005). "6.2.2". Univariate discrete distributions (3rd ed.). New York: Wiley. p. 253.
^Gelman, A.; Carlin, J. B.; Stern, H.; Dunson, D. B.; Vehtari, A.; Rubin, D. B. (2014). Bayesian Data Analysis (3rd ed.). Boca Raton: Chapman & Hall / CRC.
^Lawless, J.F. (1987). "Negative binomial and mixed Poisson regression". The Canadian Journal of Statistics. 15 (3): 209–225. doi:10.2307/3314912. JSTOR3314912.
Lindsay, B. G. (1995), Mixture models: theory, geometry and applications, NSF-CBMS Regional Conference Series in Probability and Statistics, vol. 5, Hayward, CA, USA: Institute of Mathematical Statistics, pp. i–163, ISBN978-0-940600-32-4, JSTOR4153184
Mood, A. M.; Graybill, F. A.; Boes, D. C. (1974), "III.4.3 Contagious distributions and truncated distributions", Introduction to the theory of statistics (3rd ed.), New York: McGraw-Hill, ISBN978-0-07-042864-5
Johnson, N. L.; Kemp, A. W.; Kotz, S. (2005), "8 Mixture distributions", Univariate discrete distributions, New York: Wiley, ISBN978-0-471-27246-5