Expectation Maximization

- Iterative approach to MLE when latent variables are present. - Gaussian mixture models approach to density estimation, paramter of distributions are fit using em algorithm.

Problem: Estimate joint probability distribution for dataset

Density estimation: Select probability distribution and parameters to best explain jpd.

Assumption on MLE: All variables that are relevant to the problem are present (no hidden = latent variables).

Alternate formulation of maximum likelihood is required for searching appropriate model paramters ⇒ EM Algorithm

Two modes:

* E-Step: Estimate missing (=latent) variables * M-Step: Maximizte parameters of the model in the presence of the data

Usually applied for unsupervised learning (density estimation, clustering).

Unspecied combination of multiple probability distribution functions. GMM ⇒ estimate stddev and mean for each pdf.