Abstract

High dimensional asymptotics is an active research area that provides exact risk or distributional characterizations for statistical estimators. Because of the precise nature of the theory, this type of problems has gained prominence in recent years in a number of important high dimensional problems. A common theme of this line of theory is to work under two common premises: (i) the so-called proportional regime, where the signal dimension is proportional to the sample size, and (ii) a standard Gaussian design setting with i.i.d. standard normal entries (with suitable variance scaling).

This talk will review some recent progress in moving beyond these two common premises, with an emphasis on statistical estimators in the linear model.

In the first part of the talk, we consider the problem of estimating an unknown signal under a convex constraint in the linear model with a Gaussian design. We show that the risk of the natural convex constrained least squares estimator (LSE) can be characterized exactly in high dimensional limits, by that of the convex constrained LSE in the corresponding Gaussian sequence model at a different noise level. Importantly, the risk characterization holds all the way down to the parametric rate. An interesting finding of this full-regime risk characterization is that the sample complexity for weak signal recovery in noisy linear inverse problems can be much lower than exact recovery in the noiseless settings.

In the second part of the talk, we consider the universality problem for a broad class of regularized regression estimators. Here universality means that if a "structure" is satisfied by the regression estimator under a standard Gaussian design, then it will also be satisfied under a general non-Gaussian design with independent entries. In particular, we show that with a good enough coordinate-wise bound for the regression estimator, natural structure properties that hold under Gaussian designs also hold under general designs with independent entries. As a major statistical implication, we validate inference procedures using the degrees-of-freedom adjusted debiased Lasso under general design and error distributions.

Some parts of the talk are based on joint work with Yandi Shen (Chicago).