For Full-Text PDF, please login, if you are a member of IEICE,|
or go to Pay Per View on menu list, if you are a nonmember of IEICE.
EM Algorithm with Split and Merge Operations for Mixture Models
Naonori UEDA Ryohei NAKANO
IEICE TRANSACTIONS on Information and Systems
Publication Date: 2000/12/25
Print ISSN: 0916-8532
Type of Manuscript: INVITED PAPER (Special Issue on the 1999 IEICE Excellent Paper Award)
Category: Biocybernetics, Neurocomputing
EM algorithm, split and merge operations, mixture models, maximum likelihood estimates, dimensionality reduction,
Full Text: PDF>>
The maximum likelihood estimate of a mixture model is usually found by using the EM algorithm. However, the EM algorithm suffers from a local optima problem and therefore we cannot obtain the potential performance of mixture models in practice. In the case of mixture models, local maxima often have too many components of a mixture model in one part of the space and too few in another, widely separated part of the space. To escape from such configurations we proposed a new variant of the EM algorithm in which simultaneous split and merge operations are repeatedly performed by using a new criterion for efficiently selecting the split and merge candidates. We apply the proposed algorithm to the training of Gaussian mixtures and the dimensionality reduction based on a mixture of factor analyzers using synthetic and real data and show that the proposed algorithm can markedly improve the ML estimates.