- Iterative approach to MLE when latent variables are present. - Gaussian mixture models approach to density estimation, paramter of distributions are fit using em algorithm.
Problem: Estimate joint probability distribution for dataset
Density estimation: Select probability distribution and parameters to best explain jpd.
Assumption on MLE: All variables that are relevant to the problem are present (no hidden = latent variables).
Alternate formulation of maximum likelihood is required for searching appropriate model paramters ⇒ EM Algorithm
Two modes:
* E-Step: Estimate missing (=latent) variables * M-Step: Maximizte parameters of the model in the presence of the data
Usually applied for unsupervised learning (density estimation, clustering).
Unspecied combination of multiple probability distribution functions. GMM ⇒ estimate stddev and mean for each pdf.