Dirichlet distribution
Date: 18/11/2024
Whazzat!
A dirichlet distribution is a multivariate generalisation of beta distribution. Thus,,dirichlet distribution is used as conjugate prior for categorical distribution, just like how beta distribution is for bernouilli and binomial distribution. It is parameterised by \(\mathbf{\alpha} = [\alpha_1, \alpha_2,\cdots \alpha_n]^T\). The following image is taken from wiki where n = 3 and the \(\alpha_1 = \alpha_2 = \alpha_3\).
To play with each \(\alpha_i\) Click here.
Density function
\[\text{Dir}(x;\mathbf{\alpha}) = \frac{1}{B(\mathbf{\alpha})} \prod_{i=1}^{n} x_i^{\alpha_i-1} \\ B(\mathbf{\alpha}) = \frac{\prod_{i=1}^{n}\Gamma(\alpha_i)}{\Gamma(\alpha_0)}\]The mean is given by \(\mathbb{E}[X_i]==\frac{\alpha_i}{\alpha_0}\), where \(\alpha_0 = \sum_{i=1}^{n} \alpha_i\).
Use cases
- Using Dirichlet distribution as means to sample a probability values for categorical distribution allows one to control heterogeneity. Fo instance having constant vector \(\mathbf{\alpha} \approx 0\) would give highly heterogeneous sample as compared to \(\mathbf{\alpha} \approx 10^6\).