(Translated by https://www.hiragana.jp/)
Search | arXiv e-print repository
Skip to main content

Showing 1–17 of 17 results for author: Sur, P

Searching in archive stat. Search in all archives.
.
  1. arXiv:2406.13944  [pdf, other

    math.ST cs.LG stat.ME stat.ML

    Generalization error of min-norm interpolators in transfer learning

    Authors: Yanke Song, Sohom Bhattacharya, Pragya Sur

    Abstract: This paper establishes the generalization error of pooled min-$\ell_2$-norm interpolation in transfer learning where data from diverse distributions are available. Min-norm interpolators emerge naturally as implicit regularized limits of modern machine learning algorithms. Previous work characterized their out-of-distribution risk when samples from the test distribution are unavailable during trai… ▽ More

    Submitted 19 June, 2024; originally announced June 2024.

    Comments: 53 pages, 2 figures

  2. arXiv:2406.11666  [pdf, other

    math.ST cs.LG stat.ML

    ROTI-GCV: Generalized Cross-Validation for right-ROTationally Invariant Data

    Authors: Kevin Luo, Yufan Li, Pragya Sur

    Abstract: Two key tasks in high-dimensional regularized regression are tuning the regularization strength for good predictions and estimating the out-of-sample risk. It is known that the standard approach -- $k$-fold cross-validation -- is inconsistent in modern high-dimensional settings. While leave-one-out and generalized cross-validation remain consistent in some high-dimensional cases, they become incon… ▽ More

    Submitted 17 June, 2024; originally announced June 2024.

    Comments: 25 pages, 3 figures

  3. arXiv:2406.11184  [pdf, other

    stat.ME math.ST

    HEDE: Heritability estimation in high dimensions by Ensembling Debiased Estimators

    Authors: Yanke Song, Xihong Lin, Pragya Sur

    Abstract: Estimating heritability remains a significant challenge in statistical genetics. Diverse approaches have emerged over the years that are broadly categorized as either random effects or fixed effects heritability methods. In this work, we focus on the latter. We propose HEDE, an ensemble approach to estimate heritability or the signal-to-noise ratio in high-dimensional linear models where the sampl… ▽ More

    Submitted 16 June, 2024; originally announced June 2024.

    Comments: 58 pages, 7 figures

  4. arXiv:2403.16336  [pdf, other

    stat.ML cs.LG math.ST stat.ME

    Predictive Inference in Multi-environment Scenarios

    Authors: John C. Duchi, Suyash Gupta, Kuanhao Jiang, Pragya Sur

    Abstract: We address the challenge of constructing valid confidence intervals and sets in problems of prediction across multiple environments. We investigate two types of coverage suitable for these problems, extending the jackknife and split-conformal methods to show how to obtain distribution-free coverage in such non-traditional, hierarchical data-generating scenarios. Our contributions also include exte… ▽ More

    Submitted 24 March, 2024; originally announced March 2024.

  5. arXiv:2309.07810  [pdf, other

    math.ST cs.IT stat.ME stat.ML

    Spectrum-Aware Debiasing: A Modern Inference Framework with Applications to Principal Components Regression

    Authors: Yufan Li, Pragya Sur

    Abstract: Debiasing is a fundamental concept in high-dimensional statistics. While degrees-of-freedom adjustment is the state-of-the-art technique in high-dimensional linear regression, it is limited to i.i.d. samples and sub-Gaussian covariates. These constraints hinder its broader practical use. Here, we introduce Spectrum-Aware Debiasing--a novel method for high-dimensional regression. Our approach appli… ▽ More

    Submitted 22 July, 2024; v1 submitted 14 September, 2023; originally announced September 2023.

    Comments: Minor changes and updated references

  6. arXiv:2210.12082  [pdf, other

    stat.ML cs.LG math.ST

    A Non-Asymptotic Moreau Envelope Theory for High-Dimensional Generalized Linear Models

    Authors: Lijia Zhou, Frederic Koehler, Pragya Sur, Danica J. Sutherland, Nathan Srebro

    Abstract: We prove a new generalization bound that shows for any class of linear predictors in Gaussian space, the Rademacher complexity of the class and the training error under any continuous loss $\ell$ can control the test error under all Moreau envelopes of the loss $\ell$. We use our finite-sample bound to directly recover the "optimistic rate" of Zhou et al. (2021) for linear regression with the squa… ▽ More

    Submitted 21 October, 2022; originally announced October 2022.

    Comments: As published at NeurIPS 2022

  7. arXiv:2207.04588  [pdf, other

    stat.ML cs.LG

    Multi-Study Boosting: Theoretical Considerations for Merging vs. Ensembling

    Authors: Cathy Shyr, Pragya Sur, Giovanni Parmigiani, Prasad Patil

    Abstract: Cross-study replicability is a powerful model evaluation criterion that emphasizes generalizability of predictions. When training cross-study replicable prediction models, it is critical to decide between merging and treating the studies separately. We study boosting algorithms in the presence of potential heterogeneity in predictor-outcome relationships across studies and compare two multi-study… ▽ More

    Submitted 12 July, 2022; v1 submitted 10 July, 2022; originally announced July 2022.

  8. arXiv:2205.10198  [pdf, other

    math.ST econ.EM stat.ME stat.ML

    A New Central Limit Theorem for the Augmented IPW Estimator: Variance Inflation, Cross-Fit Covariance and Beyond

    Authors: Kuanhao Jiang, Rajarshi Mukherjee, Subhabrata Sen, Pragya Sur

    Abstract: Estimation of the average treatment effect (ATE) is a central problem in causal inference. In recent times, inference for the ATE in the presence of high-dimensional covariates has been extensively studied. Among the diverse approaches that have been proposed, augmented inverse probability weighting (AIPW) with cross-fitting has emerged a popular choice in practice. In this work, we study this cro… ▽ More

    Submitted 28 October, 2022; v1 submitted 20 May, 2022; originally announced May 2022.

    Comments: 132 pages, 7 figures; In V2, we added extensive comparisons with the classical variance formula (c.f.~Sec 3, Fig 2, Fig 4) and elaborated on the non-trivial cross-fit covariance phenomenon further

  9. arXiv:2204.04476  [pdf, other

    math.ST cs.LG math.PR stat.ML

    High-dimensional Asymptotics of Langevin Dynamics in Spiked Matrix Models

    Authors: Tengyuan Liang, Subhabrata Sen, Pragya Sur

    Abstract: We study Langevin dynamics for recovering the planted signal in the spiked matrix model. We provide a "path-wise" characterization of the overlap between the output of the Langevin algorithm and the planted signal. This overlap is characterized in terms of a self-consistent system of integro-differential equations, usually referred to as the Crisanti-Horner-Sommers-Cugliandolo-Kurchan (CHSCK) equa… ▽ More

    Submitted 9 April, 2022; originally announced April 2022.

    Comments: 26 pages

    Journal ref: Information and Inference: A Journal of the IMA, 12(4):2720-2752, 2023

  10. arXiv:2006.11478  [pdf, ps, other

    cs.LG stat.ML

    Representation via Representations: Domain Generalization via Adversarially Learned Invariant Representations

    Authors: Zhun Deng, Frances Ding, Cynthia Dwork, Rachel Hong, Giovanni Parmigiani, Prasad Patil, Pragya Sur

    Abstract: We investigate the power of censoring techniques, first developed for learning {\em fair representations}, to address domain generalization. We examine {\em adversarial} censoring techniques for learning invariant representations from multiple "studies" (or domains), where each study is drawn according to a distribution on domains. The mapping is used at test time to classify instances from a new… ▽ More

    Submitted 19 June, 2020; originally announced June 2020.

  11. arXiv:2004.01840  [pdf, other

    cs.LG stat.ML

    Abstracting Fairness: Oracles, Metrics, and Interpretability

    Authors: Cynthia Dwork, Christina Ilvento, Guy N. Rothblum, Pragya Sur

    Abstract: It is well understood that classification algorithms, for example, for deciding on loan applications, cannot be evaluated for fairness without taking context into account. We examine what can be learned from a fairness oracle equipped with an underlying understanding of ``true'' fairness. The oracle takes as input a (context, classifier) pair satisfying an arbitrary fairness definition, and accept… ▽ More

    Submitted 3 April, 2020; originally announced April 2020.

    Comments: 17 pages, 1 figure

  12. arXiv:2002.01586  [pdf, other

    math.ST cs.IT cs.LG stat.ML

    A Precise High-Dimensional Asymptotic Theory for Boosting and Minimum-$\ell_1$-Norm Interpolated Classifiers

    Authors: Tengyuan Liang, Pragya Sur

    Abstract: This paper establishes a precise high-dimensional asymptotic theory for boosting on separable data, taking statistical and computational perspectives. We consider a high-dimensional setting where the number of features (weak learners) $p$ scales with the sample size $n$, in an overparametrized regime. Under a class of statistical models, we provide an exact analysis of the generalization error of… ▽ More

    Submitted 22 July, 2021; v1 submitted 4 February, 2020; originally announced February 2020.

    Comments: 68 pages, 4 figures

    Journal ref: The Annals of Statistics, 50(3):1669-1695, 2022

  13. arXiv:2001.09351  [pdf, other

    math.ST stat.ME

    The Asymptotic Distribution of the MLE in High-dimensional Logistic Models: Arbitrary Covariance

    Authors: Qian Zhao, Pragya Sur, Emmanuel J. Candès

    Abstract: We study the distribution of the maximum likelihood estimate (MLE) in high-dimensional logistic models, extending the recent results from Sur (2019) to the case where the Gaussian covariates may have an arbitrary covariance structure. We prove that in the limit of large problems holding the ratio between the number $p$ of covariates and the sample size $n$ constant, every finite list of MLE coordi… ▽ More

    Submitted 4 January, 2023; v1 submitted 25 January, 2020; originally announced January 2020.

    Journal ref: Bernoulli 28 (3) 1835-1861, August 2022

  14. arXiv:1804.09753  [pdf, other

    stat.ME stat.ML

    The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression

    Authors: Emmanuel J. Candes, Pragya Sur

    Abstract: This paper rigorously establishes that the existence of the maximum likelihood estimate (MLE) in high-dimensional logistic regression models with Gaussian covariates undergoes a sharp `phase transition'. We introduce an explicit boundary curve $h_{\text{MLE}}$, parameterized by two scalars measuring the overall magnitude of the unknown sequence of regression coefficients, with the following proper… ▽ More

    Submitted 25 April, 2018; originally announced April 2018.

    Comments: 15 pages, 2 figures

  15. A modern maximum-likelihood theory for high-dimensional logistic regression

    Authors: Pragya Sur, Emmanuel J. Candes

    Abstract: Every student in statistics or data science learns early on that when the sample size largely exceeds the number of variables, fitting a logistic model produces estimates that are approximately unbiased. Every student also learns that there are formulas to predict the variability of these estimates which are used for the purpose of statistical inference; for instance, to produce p-values for testi… ▽ More

    Submitted 16 June, 2018; v1 submitted 19 March, 2018; originally announced March 2018.

    Comments: 29 pages, 14 figures, 4 tables

  16. arXiv:1706.01191  [pdf, other

    math.ST cs.IT math.PR stat.ML

    The Likelihood Ratio Test in High-Dimensional Logistic Regression Is Asymptotically a Rescaled Chi-Square

    Authors: Pragya Sur, Yuxin Chen, Emmanuel J. Candès

    Abstract: Logistic regression is used thousands of times a day to fit data, predict future outcomes, and assess the statistical significance of explanatory variables. When used for the purpose of statistical inference, logistic models produce p-values for the regression coefficients by using an approximation to the distribution of the likelihood-ratio test. Indeed, Wilks' theorem asserts that whenever we ha… ▽ More

    Submitted 5 June, 2017; originally announced June 2017.

    Comments: 58 pages, 7 figures

  17. arXiv:1309.0579  [pdf

    stat.ME

    Modeling Bimodal Discrete Data Using Conway-Maxwell-Poisson Mixture Models

    Authors: Pragya Sur, Galit Shmueli, Smarajit Bose, Paromita Dubey

    Abstract: Bimodal truncated count distributions are frequently observed in aggregate survey data and in user ratings when respondents are mixed in their opinion. They also arise in censored count data, where the highest category might create an additional mode. Modeling bimodal behavior in discrete data is useful for various purposes, from comparing shapes of different samples (or survey questions) to predi… ▽ More

    Submitted 23 January, 2014; v1 submitted 2 September, 2013; originally announced September 2013.

    Comments: 29 pages