Hostname: page-component-745bb68f8f-hvd4g Total loading time: 0 Render date: 2025-01-11T18:51:46.627Z Has data issue: false hasContentIssue false
  • English
  • Français

On EM algorithms and their proximal generalizations

Published online by Cambridge University Press:  08 May 2008

Stéphane Chrétien  and
Alfred O. Hero
Stéphane Chrétien
Affiliation:
Université de Franche-Comté, Laboratoire de Mathématiques, UMR CNRS 6623, 16 route de Gray, 25030 Besançon, France; chretien@math.univ-fcomte.fr
Alfred O. Hero
Affiliation:
Department of Electrical Engineering and Computer Science, 1301 Beal St., University of Michigan, Ann Arbor, MI 48109-2122, USA; hero@eecs.umich.edu
Get access

Abstract

In this paper, we analyze the celebrated EM algorithm from the point of view of proximal point algorithms. More precisely, we study a new type of generalization of the EM procedure introduced in [Chretien and Hero (1998)] and called Kullback-proximal algorithms. The proximal framework allows us to prove new results concerning the cluster points. An essential contribution is a detailed analysis of the case where some cluster points lie on the boundary of the parameter space.

Keywords

Maximum Likelihood Estimation (MLE)EM algorithmproximal point algorithmKarush-Kuhn-Tucker conditionmixture densitiescompeting risks models
Type
Research Article
Information
ESAIM: Probability and Statistics , Volume 12 , 2008 , pp. 308 - 326
Copyright
© EDP Sciences, SMAI, 2008

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ahn, H., Moon, H. and Kodell, R.L., Attribution of tumour lethality and estimation of the time to onset of occult tumours in the absence of cause-of-death information. J. Roy. Statist. Soc. Ser. C 49 (2000) 157–169. CrossRef
Box, M.J., A new method of constrained optimization and a comparison with other methods. Comp. J. 8 (1965) 42–52. CrossRef
G. Celeux, S. Chretien, F. Forbes and A. Mkhadri, A component-wise EM algorithm for mixtures. J. Comput. Graph. Statist. 10 (2001), 697–712 and INRIA RR-3746, Aug. 1999.
S. Chretien and A.O. Hero, Acceleration of the EM algorithm via proximal point iterations, in Proceedings of the International Symposium on Information Theory, MIT, Cambridge (1998) 444.
Chrétien, S. and Hero, A., Kullback proximal algorithms for maximum-likelihood estimation. IEEE Trans. Inform. Theory 46 (2000) 1800–1810. CrossRef
Csiszár, I., Information-type measures of divergence of probability distributions and indirect observations. Studia Sci. Math. Hung. 2 (1967) 299–318.
A.P. Dempster, N.M. Laird and D.B. Rubin, Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc., Ser. B 39 (1977) 1–38.
I.A. Ibragimov and R.Z. Has'minskii, Statistical estimation: Asymptotic theory. Springer-Verlag, New York (1981).
Journal of Statistical Planning and Inference No. 107 (2002) 1–2.
Kalai, A.T. and Vempala, S., Simulated annealing for convex optimization. Math. Oper. Res. 31 (2006) 253–266. CrossRef
Martinet, B., Régularisation d'inéquation variationnelles par approximations successives. Revue Francaise d'Informatique et de Recherche Operationnelle 3 (1970) 154–179.
G.J. McLachlan and T. Krishnan, The EM algorithm and extensions, Wiley Series in Probability and Statistics: Applied Probability and Statistics. John Wiley and Sons, Inc., New York (1997).
Moon, H., Ahn, H., Kodell, R. and Pearce, B., A comparison of a mixture likelihood method and the EM algorithm for an estimation problme in animal carcinogenicity studies. Comput. Statist. Data Anal. 31 (1999) 227–238. CrossRef
A.M. Ostrowski, Solution of equations and systems of equations. Pure and Applied Mathematics, Vol. IX. Academic Press, New York-London (1966).
Rockafellar, R.T., Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14 (1976) 877–898. CrossRef
Teboulle, M., Entropic proximal mappings with application to nonlinear programming. Math. Oper. Res. 17 (1992) 670–690. CrossRef
Tseng, P., An analysis of the EM algorithm and entropy-like proximal point methods. Math. Oper. Res. 29 (2004) 27–44. CrossRef
On, C.F.J. Wu the convergence properties of the EM algorithm. Ann. Stat. 11 (1983) 95–103.
Z.B. Zabinsky, Stochastic adaptive search for global optimization. Nonconvex Optimization and its Applications 72. Kluwer Academic Publishers, Boston, MA (2003).
W.I. Zangwill and B. Mond, Nonlinear programming: a unified approach. Prentice-Hall International Series in Management. Prentice-Hall, Inc., Englewood Cliffs, N.J. (1969).