Open Access
2020 Perspective maximum likelihood-type estimation via proximal decomposition
Patrick L. Combettes, Christian L. Müller
Electron. J. Statist. 14(1): 207-238 (2020). DOI: 10.1214/19-EJS1662

Abstract

We introduce a flexible optimization model for maximum likelihood-type estimation (M-estimation) that encompasses and generalizes a large class of existing statistical models, including Huber’s concomitant M-estimator, Owen’s Huber/Berhu concomitant estimator, the scaled lasso, support vector machine regression, and penalized estimation with structured sparsity. The model, termed perspective M-estimation, leverages the observation that convex M-estimators with concomitant scale as well as various regularizers are instances of perspective functions, a construction that extends a convex function to a jointly convex one in terms of an additional scale variable. These nonsmooth functions are shown to be amenable to proximal analysis, which leads to principled and provably convergent optimization algorithms via proximal splitting. We derive novel proximity operators for several perspective functions of interest via a geometrical approach based on duality. We then devise a new proximal splitting algorithm to solve the proposed M-estimation problem and establish the convergence of both the scale and regression iterates it produces to a solution. Numerical experiments on synthetic and real-world data illustrate the broad applicability of the proposed framework.

Citation

Download Citation

Patrick L. Combettes. Christian L. Müller. "Perspective maximum likelihood-type estimation via proximal decomposition." Electron. J. Statist. 14 (1) 207 - 238, 2020. https://doi.org/10.1214/19-EJS1662

Information

Received: 1 April 2019; Published: 2020
First available in Project Euclid: 8 January 2020

zbMATH: 07154987
MathSciNet: MR4048598
Digital Object Identifier: 10.1214/19-EJS1662

Subjects:
Primary: 62J02 , 90C25
Secondary: 46N30 , 62P10

Keywords: concomitant M-estimator , Convex optimization , heteroscedastic model , perspective function , proximal algorithm , robust regression

Vol.14 • No. 1 • 2020
Back to Top