EM Algorithm with Split and Merge Operations for Mixture Models

Naonori UEDA  Ryohei NAKANO  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E83-D   No.12   pp.2047-2055
Publication Date: 2000/12/25
Online ISSN: 
DOI: 
Print ISSN: 0916-8532
Type of Manuscript: INVITED PAPER (Special Issue on the 1999 IEICE Excellent Paper Award)
Category: Biocybernetics, Neurocomputing
Keyword: 
EM algorithm,  split and merge operations,  mixture models,  maximum likelihood estimates,  dimensionality reduction,  

Full Text: PDF>>
Buy this Article




Summary: 
The maximum likelihood estimate of a mixture model is usually found by using the EM algorithm. However, the EM algorithm suffers from a local optima problem and therefore we cannot obtain the potential performance of mixture models in practice. In the case of mixture models, local maxima often have too many components of a mixture model in one part of the space and too few in another, widely separated part of the space. To escape from such configurations we proposed a new variant of the EM algorithm in which simultaneous split and merge operations are repeatedly performed by using a new criterion for efficiently selecting the split and merge candidates. We apply the proposed algorithm to the training of Gaussian mixtures and the dimensionality reduction based on a mixture of factor analyzers using synthetic and real data and show that the proposed algorithm can markedly improve the ML estimates.