Exam notes

Useful distributions

univariate normal

A random variable \(X \in \mathbb{R}\) has a \(N(\theta, \sigma^2)\) distribution if \(\sigma^2 > 0\) and

\(p(x | \theta, \sigma^2) = (2 \pi \sigma^2)^{-\frac{1}{2}} e^{-\frac{1}{2\sigma^2}(x - \theta)^2} \ \ \ \text{ for } -\infty < x < \infty.\)

multivariate normal

A random vector \(X \in \mathbb{R}^p\) has a \(MVN(\theta, \Sigma)\) distribution if \(\Sigma > 0\) and

\(p(x| \theta, \Sigma) = (2\pi)^{-p/2} |\Sigma|^{-1/2} \exp\{ -\frac{1}{2}(x - \theta)^T \Sigma^{-1} (x - \theta) \}\)

gamma

A random variable \(X \in (0, \infty)\) has a gamma(a,b) distribution if \(a > 0, b > 0\) and

\(p(x |a,b) = \frac{b^a}{\Gamma(a)} x^{a - 1} e^{-bx} \ \ \ \text{ for } x > 0.\)
\(E[X | a, b] = a/b\), \(Var[X | a,b] = a / b^2\)

inverse-gamma

A random variable \(X \in (0, \infty)\) has an inverse-gamma(a,b) distribution if 1/X has a gamma(a,b) distribution. If \(X\) is inverse-gamma(a,b) then the density of X is
\(p(x|a,b) = \frac{b^a}{\Gamma(a)} x^{-a-1} e^{-b/x} \ \ \ \text{ for } x > 0.\)
\(E[X|a,b] = \frac{b}{a-1}\) if \(a>=1\), \(\infty\) if \(0<a<1\)
\(Var[X|a,b] = \frac{b^2}{(a-1)^2(a-2)}\) if \(a\geq2\), \(\infty\) if \(0<a<2\)

inverse-Wishart

A random \(p \times p\) matrix \(\Sigma\) has an inverse-Wishart distribution if \(p(\Sigma | \nu_0, S_0^{-1}) \propto |\Sigma|^{-(\nu_0 + p + 1)/2} \times \exp \{ -\frac{1}{2}tr(S_0 \Sigma^{-1})\}\).

• the support is \(\Sigma > 0\) and \(\Sigma\) symmetric \(p \times p\) matrix. \(\nu_0 \in \mathbb{N}^+\) and \(\nu_0 \geq p\). \(S_0\) is a \(p \times p\) symmetric positive definite matrix.

• \(E[\Sigma^{-1}] = \nu_0 S_0^{-1}\) and \(E[\Sigma] = \frac{1}{\nu_0 - p - 1} S_0\).

binomial

A random variable \(X \in \{0, 1, \ldots, n\}\) has a binomial\((n, \theta)\) distribution if \(\theta \in [0, 1]\) and

\(p(X = x| \theta, n) = {n \choose x} \theta^x (1- \theta)^{n-x} \ \ \ \text{ for } x\in \{0, 1, \ldots, n \}\)
\(E[X|\theta] = n\theta\), \(Var[X|\theta] = n\theta(1-\theta)\)

beta

A random variable \(X \in [0, 1]\) has a beta(a,b) distribution if \(a > 0, b > 0\) and

\(p(x|a,b) = \frac{\Gamma(a + b)}{\Gamma(a)\Gamma(b)} x^{a-1} (1-x)^{b-1} \ \ \ \text{ for } 0 \leq x \leq 1.\)
\(E[X|a,b] = \frac{a}{a + b}\), \(Var[X|a,b] = \frac{ab}{(a + b + 1)(a + b)^2}\)

Poisson

A random variable \(X \in \{0, 1, 2, \ldots \}\) has a Poisson(\(\theta\)) distribution if \(\theta > 0\) and
\(p(X = x | \theta) = \theta^x \frac{e^{-\theta}}{x!} \ \ \ \text{ for } x \in \{0, 1, 2, \ldots\}\)
\(E[X|\theta] = \theta\), \(Var[X|\theta] = \theta\)

exponential

A random variable \(X \in [0, \infty)\) has a exponential(\(\theta\)) distribution if \(\theta >0\) and
\(p(x | \theta) = \theta e^{-\theta x}\)
\(E[X|\theta] = \frac{1}{\theta}\), \(Var[X|\theta] = \frac{1}{\theta^2}\)