Aggregation for Gaussian regression - Mathematics > Statistics TheoryReport as inadecuate

Aggregation for Gaussian regression - Mathematics > Statistics Theory - Download this document for free, or read online. Document in PDF available to download.

Abstract: This paper studies statistical aggregation procedures in the regressionsetting. A motivating factor is the existence of many different methods ofestimation, leading to possibly competing estimators. We consider here threedifferent types of aggregation: model selection MS aggregation, convex Caggregation and linear L aggregation. The objective of MS is to select theoptimal single estimator from the list; that of C is to select the optimalconvex combination of the given estimators; and that of L is to select theoptimal linear combination of the given estimators. We are interested inevaluating the rates of convergence of the excess risks of the estimatorsobtained by these procedures. Our approach is motivated by recently publishedminimax results Nemirovski, A. 2000. Topics in non-parametric statistics.Lectures on Probability Theory and Statistics Saint-Flour, 1998. LectureNotes in Math. 1738 85-277. Springer, Berlin; Tsybakov, A. B. 2003. Optimalrates of aggregation. Learning Theory and Kernel Machines. Lecture Notes inArtificial Intelligence 2777 303-313. Springer, Heidelberg. There existcompeting aggregation procedures achieving optimal convergence rates for eachof the MS, C and L cases separately. Since these procedures are notdirectly comparable with each other, we suggest an alternative solution. Weprove that all three optimal rates, as well as those for the newly introducedS aggregation subset selection, are nearly achieved via a single``universal- aggregation procedure. The procedure consists of mixing theinitial estimators with weights obtained by penalized least squares. Twodifferent penalties are considered: one of them is of the BIC type, the secondone is a data-dependent $\ell 1$-type penalty.

Author: Florentina Bunea, Alexandre B. Tsybakov, Marten H. Wegkamp


Related documents