Critical lines in symmetry of mixture models and its application to component splitting.pdf

Critical Lines in Symmetry of Mixture Models and its Application to Component Splitting Kenji Fukumizu Institute of Statistical Mathematics Tokyo 106-8569 Japan fukumizu@ism.ac.jp Shotaro Akaho AIST Tsukuba 305-8568 Japan s.akaho@aist.go.jp Shun-ichi Amari RIKEN Wako 351-0198 Japan amari@brain.riken.go.jp Abstract We show the existence of critical points as lines for the likelihood func- tion of mixture-type models. They are given by embedding of a critical point for models with less components. A sufficient condition that the critical line gives local maxima or saddle points is also derived. Based on this fact, a component-split method is proposed for a mixture of Gaus- sian components, and its effectiveness is verified through experiments. 1 Introduction The likelihood function of a mixture model often has a complex shape so that calculation of an estimator can be difficult, whether the maximum likelihood or Bayesian approach is used. In the maximum likelihood estimation, convergence of the EM algorithm to the global maximum is not guaranteed, while it is a standard method. Investigation of the like- lihood function for mixture models is important to develop effective methods for learning. This paper discusses the critical points of the likelihood function for mixture-type models by analyzing their hierarchical symmetric structure. As generalization of [1], we show that, given a critical point of the likelihood for the model with (H ? 1) components, duplication of any of the components gives critical points as lines for the model with H components. We call them critical lines of mixture models. We derive also a sufficient condition that the critical lines give maxima or saddle points of the larger model, and show that given a maximum of the likelihood for a mixture of Gaussian components, an appropriate split of any component always gives an ascending direction of the likelihood. Based on this theory, we propose a stable method of splitting a component, which work