(Translated by https://www.hiragana.jp/)
Deep Learning for Individual Heterogeneity: An Automatic Inference Framework
IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2010.14694.html
   My bibliography  Save this paper

Deep Learning for Individual Heterogeneity: An Automatic Inference Framework

Author

Listed:
  • Max H. Farrell
  • Tengyuan Liang
  • Sanjog Misra

Abstract

We develop methodology for estimation and inference using machine learning to enrich economic models. Our framework takes a standard economic model and recasts the parameters as fully flexible nonparametric functions, to capture the rich heterogeneity based on potentially high dimensional or complex observable characteristics. These "parameter functions" retain the interpretability, economic meaning, and discipline of classical parameters. Deep learning is particularly well-suited to structured modeling of heterogeneity in economics. We show how to design the network architecture to match the structure of the economic model, delivering novel methodology that moves deep learning beyond prediction. We prove convergence rates for the estimated parameter functions. These functions are the key inputs into the finite-dimensional parameter of inferential interest. We obtain inference based on a novel influence function calculation that covers any second-stage parameter and any machine-learning-enriched model that uses a smooth per-observation loss function. No additional derivations are required. The score can be taken directly to data, using automatic differentiation if needed. The researcher need only define the original model and define the parameter of interest. A key insight is that we need not write down the influence function in order to evaluate it on the data. Our framework gives new results for a host of contexts, covering such diverse examples as price elasticities, willingness-to-pay, and surplus measures in binary or multinomial choice models, effects of continuous treatment variables, fractional outcome models, count data, heterogeneous production functions, and more. We apply our methodology to a large scale advertising experiment for short-term loans. We show how economically meaningful estimates and inferences can be made that would be unavailable without our results.

Suggested Citation

  • Max H. Farrell & Tengyuan Liang & Sanjog Misra, 2020. "Deep Learning for Individual Heterogeneity: An Automatic Inference Framework," Papers 2010.14694, arXiv.org, revised Jul 2021.
    • Handle: RePEc:arx:papers:2010.14694
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2010.14694
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Leslie E. Papke, 1995. "Participation in and Contributions to 401(k) Pension Plans: Evidence from Plan Data," Journal of Human Resources, University of Wisconsin Press, vol. 30(2), pages 311-325.
    2. Diebold, Francis X. & Shin, Minchul, 2019. "Machine learning for regularized survey forecast combination: Partially-egalitarian LASSO and its derivatives," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1679-1691.
    3. Max H. Farrell & Tengyuan Liang & Sanjog Misra, 2021. "Deep Neural Networks for Estimation and Inference," Econometrica, Econometric Society, vol. 89(1), pages 181-213, January.
    4. Hidehiko Ichimura & Whitney K. Newey, 2022. "The influence function of semiparametric estimators," Quantitative Economics, Econometric Society, vol. 13(1), pages 29-61, January.
    5. Tetsuya Kaji & Elena Manresa & Guillaume Pouliot, 2020. "An adversarial approach to structural estimation," CeMMAP working papers CWP39/20, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    6. Li, Qi, et al, 2002. "Semiparametric Smooth Coefficient Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(3), pages 412-422, July.
    7. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    8. Xiaohong Chen & Sydney C. Ludvigson, 2009. "Land of addicts? an empirical investigation of habit-based asset pricing models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 24(7), pages 1057-1093.
    9. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    10. Matias D. Cattaneo & Michael Jansson & Whitney K. Newey, 2018. "Inference in Linear Regression Models with Many Covariates and Heteroscedasticity," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1350-1361, July.
    11. Mitsuru Igami, 0. "Artificial intelligence as structural estimation: Deep Blue, Bonanza, and AlphaGo," Econometrics Journal, Royal Economic Society, vol. 23(3), pages 1-24.
    12. Sendhil Mullainathan & Jann Spiess, 2017. "Machine Learning: An Applied Econometric Approach," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 87-106, Spring.
    13. Cattaneo, Matias D., 2010. "Efficient semiparametric estimation of multi-valued treatment effects under ignorability," Journal of Econometrics, Elsevier, vol. 155(2), pages 138-154, April.
    14. Farrell, Max H., 2015. "Robust inference on average treatment effects with possibly more covariates than observations," Journal of Econometrics, Elsevier, vol. 189(1), pages 1-23.
    15. Nevo, Aviv, 2001. "Measuring Market Power in the Ready-to-Eat Cereal Industry," Econometrica, Econometric Society, vol. 69(2), pages 307-342, March.
    16. Powell, James L & Stock, James H & Stoker, Thomas M, 1989. "Semiparametric Estimation of Index Coefficients," Econometrica, Econometric Society, vol. 57(6), pages 1403-1430, November.
    17. Jianhua Z. Huang & Haipeng Shen, 2004. "Functional Coefficient Regression Models for Non‐linear Time Series: A Polynomial Spline Approach," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 31(4), pages 515-534, December.
    18. Leeb, Hannes & Pötscher, Benedikt M., 2005. "Model Selection And Inference: Facts And Fiction," Econometric Theory, Cambridge University Press, vol. 21(1), pages 21-59, February.
    19. Papke, Leslie E & Wooldridge, Jeffrey M, 1996. "Econometric Methods for Fractional Response Variables with an Application to 401(K) Plan Participation Rates," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 11(6), pages 619-632, Nov.-Dec..
    20. Graham, Bryan S. & Pinto, Cristine Campos de Xavier, 2022. "Semiparametrically efficient estimation of the average linear regression function," Journal of Econometrics, Elsevier, vol. 226(1), pages 115-138.
    21. Tetsuya Kaji & Elena Manresa & Guillaume Pouliot, 2020. "An Adversarial Approach to Structural Estimation," Working Papers 2020-144, Becker Friedman Institute for Research In Economics.
    22. Puneet Manchanda & Pradeep K. Chintagunta, 2004. "Responsiveness of Physician Prescription Behavior to Salesforce Effort: An Individual Level Analysis," Marketing Letters, Springer, vol. 15(2_3), pages 129-145, July.
    23. Newey, Whitney K, 1990. "Semiparametric Efficiency Bounds," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 5(2), pages 99-135, April-Jun.
    24. Tengyuan Liang & Hai Tran-Bach, 2020. "Mehler’s Formula, Branching Process, and Compositional Kernels of Deep Neural Networks," Working Papers 2020-151, Becker Friedman Institute for Research In Economics.
    25. Susan Athey & Guido W. Imbens & Stefan Wager, 2018. "Approximate residual balancing: debiased inference of average treatment effects in high dimensions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 80(4), pages 597-623, September.
    26. Matias D. Cattaneo & Richard K. Crump & Michael Jansson, 2013. "Generalized Jackknife Estimators of Weighted Average Derivatives," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(504), pages 1243-1256, December.
    27. Patrick Bajari & Denis Nekipelov & Stephen P. Ryan & Miaoyu Yang, 2015. "Machine Learning Methods for Demand Estimation," American Economic Review, American Economic Association, vol. 105(5), pages 481-485, May.
    28. Mitsuru Igami, 2020. "Artificial intelligence as structural estimation: Deep Blue, Bonanza, and AlphaGo," The Econometrics Journal, Royal Economic Society, vol. 23(3), pages 1-24.
    29. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
    30. Sendhil Mullainathan & Ziad Obermeyer, 2023. "Diagnosing Physician Error: A Machine Learning Approach to Low-Value Health Care," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 137(2), pages 679-727.
    31. Steven T. Berry, 1994. "Estimating Discrete-Choice Models of Product Differentiation," RAND Journal of Economics, The RAND Corporation, vol. 25(2), pages 242-262, Summer.
    32. Mert Demirer & Vasilis Syrgkanis & Greg Lewis & Victor Chernozhukov, 2019. "Semi-Parametric Efficient Policy Learning with Continuous Actions," Papers 1905.10116, arXiv.org, revised Jul 2019.
    33. Matias D. Cattaneo, 2010. "multi-valued treatment effects," The New Palgrave Dictionary of Economics,, Palgrave Macmillan.
    34. Edward H. Kennedy & Zongming Ma & Matthew D. McHugh & Dylan S. Small, 2017. "Non-parametric methods for doubly robust estimation of continuous treatment effects," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(4), pages 1229-1245, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Christensen, Peter & Francisco, Paul & Myers, Erica & Shao, Hansen & Souza, Mateus, 2024. "Energy efficiency can deliver for climate policy: Evidence from machine learning-based targeting," Journal of Public Economics, Elsevier, vol. 234(C).
    2. Pranjal Rawat, 2024. "A Deep Learning Approach to Heterogeneous Consumer Aesthetics in Retail Fashion," Papers 2405.10498, arXiv.org.
    3. Simon, Frederik & Weibels, Sebastian & Zimmermann, Tom, 2023. "Deep parametric portfolio policies," CFR Working Papers 23-01, University of Cologne, Centre for Financial Research (CFR).
    4. Smeele, Nicholas V.R. & Chorus, Caspar G. & Schermer, Maartje H.N. & de Bekker-Grob, Esther W., 2023. "Towards machine learning for moral choice analysis in health economics: A literature review and research agenda," Social Science & Medicine, Elsevier, vol. 326(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kyle Colangelo & Ying-Ying Lee, 2019. "Double debiased machine learning nonparametric inference with continuous treatments," CeMMAP working papers CWP72/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    2. Kyle Colangelo & Ying-Ying Lee, 2020. "Double Debiased Machine Learning Nonparametric Inference with Continuous Treatments," Papers 2004.03036, arXiv.org, revised Sep 2023.
    3. Hui Chen & Antoine Didisheim & Simon Scheidegger, 2021. "Deep Structural Estimation:With an Application to Option Pricing," Cahiers de Recherches Economiques du Département d'économie 21.14, Université de Lausanne, Faculté des HEC, Département d’économie.
    4. Victor Chernozhukov & Juan Carlos Escanciano & Hidehiko Ichimura & Whitney K. Newey & James M. Robins, 2022. "Locally Robust Semiparametric Estimation," Econometrica, Econometric Society, vol. 90(4), pages 1501-1535, July.
    5. Michael C. Knaus, 2021. "A double machine learning approach to estimate the effects of musical practice on student’s skills," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 282-300, January.
    6. Kyle Colangelo & Ying-Ying Lee, 2019. "Double debiased machine learning nonparametric inference with continuous treatments," CeMMAP working papers CWP54/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    7. Rahul Singh & Liyuan Xu & Arthur Gretton, 2020. "Kernel Methods for Causal Functions: Dose, Heterogeneous, and Incremental Response Curves," Papers 2010.04855, arXiv.org, revised Oct 2022.
    8. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    9. Farrell, Max H., 2015. "Robust inference on average treatment effects with possibly more covariates than observations," Journal of Econometrics, Elsevier, vol. 189(1), pages 1-23.
    10. Ganesh Karapakula, 2023. "Stable Probability Weighting: Large-Sample and Finite-Sample Estimation and Inference Methods for Heterogeneous Causal Effects of Multivalued Treatments Under Limited Overlap," Papers 2301.05703, arXiv.org, revised Jan 2023.
    11. Su, Liangjun & Ura, Takuya & Zhang, Yichong, 2019. "Non-separable models with high-dimensional data," Journal of Econometrics, Elsevier, vol. 212(2), pages 646-677.
    12. Michael Pollmann, 2020. "Causal Inference for Spatial Treatments," Papers 2011.00373, arXiv.org, revised Jan 2023.
    13. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    14. Chunrong Ai & Oliver Linton & Kaiji Motegi & Zheng Zhang, 2021. "A unified framework for efficient estimation of general treatment models," Quantitative Economics, Econometric Society, vol. 12(3), pages 779-816, July.
    15. Wei Huang & Oliver Linton & Zheng Zhang, 2021. "A Unified Framework for Specification Tests of Continuous Treatment Effect Models," Papers 2102.08063, arXiv.org, revised Sep 2021.
    16. Matias D Cattaneo & Michael Jansson & Xinwei Ma, 2019. "Two-Step Estimation and Inference with Possibly Many Included Covariates," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 86(3), pages 1095-1122.
    17. Max H. Farrell & Tengyuan Liang & Sanjog Misra, 2021. "Deep Neural Networks for Estimation and Inference," Econometrica, Econometric Society, vol. 89(1), pages 181-213, January.
    18. Michael Zimmert & Michael Lechner, 2019. "Nonparametric estimation of causal heterogeneity under high-dimensional confounding," Papers 1908.08779, arXiv.org.
    19. Liu, Lin & Mukherjee, Rajarshi & Robins, James M., 2024. "Assumption-lean falsification tests of rate double-robustness of double-machine-learning estimators," Journal of Econometrics, Elsevier, vol. 240(2).
    20. Qi Li & Jeffrey Scott Racine, 2006. "Nonparametric Econometrics: Theory and Practice," Economics Books, Princeton University Press, edition 1, volume 1, number 8355.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2010.14694. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.