(Translated by https://www.hiragana.jp/)
Selection Bias in College Admissions Test Scores
IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/14265.html
   My bibliography  Save this paper

Selection Bias in College Admissions Test Scores

Author

Listed:
  • Melissa Clark
  • Jesse Rothstein
  • Diane Whitmore Schanzenbach

Abstract

Data from college admissions tests can provide a valuable measure of student achievement, but the non-representativeness of test-takers is an important concern. We examine selectivity bias in both state-level and school-level SAT and ACT averages. The degree of selectivity may differ importantly across and within schools, and across and within states. To identify within-state selectivity, we use a control function approach that conditions on scores from a representative test. Estimates indicate strong selectivity of test-takers in "ACT states," where most college-bound students take the ACT, and much less selectivity in SAT states. To identify within- and between-school selectivity, we take advantage of a policy reform in Illinois that made taking the ACT a graduation requirement. Estimates based on this policy change indicate substantial positive selection into test participation both across and within schools. Despite this, school-level averages of observed scores are extremely highly correlated with average latent scores, as across-school variation in sample selectivity is small relative to the underlying signal. As a result, in most contexts the use of observed school mean test scores in place of latent means understates the degree of between-school variation in achievement but is otherwise unlikely to lead to misleading conclusions.

Suggested Citation

  • Melissa Clark & Jesse Rothstein & Diane Whitmore Schanzenbach, 2008. "Selection Bias in College Admissions Test Scores," NBER Working Papers 14265, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:14265
    Note: CH LS
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w14265.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. David Card & Alan B. Krueger, 2005. "Would the Elimination of Affirmative Action Affect Highly Qualified Minority Applicants? Evidence from California and Texas," ILR Review, Cornell University, ILR School, vol. 58(3), pages 416-434, April.
    2. Dynarski, Mark, 1987. "The Scholastic Aptitude Test: Participation and performance," Economics of Education Review, Elsevier, vol. 6(3), pages 263-273, June.
    3. Gronau, Reuben, 1974. "Wage Comparisons-A Selectivity Bias," Journal of Political Economy, University of Chicago Press, vol. 82(6), pages 1119-1143, Nov.-Dec..
    4. Jesse M. Rothstein, 2006. "Good Principals or Good Peers? Parental Valuation of School Characteristics, Tiebout Equilibrium, and the Incentive Effects of Competition among Jurisdictions," American Economic Review, American Economic Association, vol. 96(4), pages 1333-1350, September.
    5. Eric A. Hanushek & Lori L. Taylor, 1990. "Alternative Assessments of the Performance of Schools: Measurement of State Variations in Achievement," Journal of Human Resources, University of Wisconsin Press, vol. 25(2), pages 179-201.
    6. Katharine G. Abraham & Melissa A. Clark, 2006. "Financial Aid and Students’ College Decisions: Evidence from the District of Columbia Tuition Assistance Grant Program," Journal of Human Resources, University of Wisconsin Press, vol. 41(3).
    7. Card, David & Payne, A. Abigail, 2002. "School finance reform, the distribution of school spending, and the distribution of student test scores," Journal of Public Economics, Elsevier, vol. 83(1), pages 49-82, January.
    8. Krueger, Alan B & Whitmore, Diane M, 2001. "The Effect of Attending a Small Class in the Early Grades on College-Test Taking and Middle School Test Results: Evidence from Project STAR," Economic Journal, Royal Economic Society, vol. 111(468), pages 1-28, January.
    9. Murphy, Kevin M & Topel, Robert H, 2002. "Estimation and Inference in Two-Step Econometric Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 88-97, January.
    10. Daniel M. Koretz, 2002. "Limitations in the Use of Achievement Tests as Measures of Educators' Productivity," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 752-777.
    11. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    12. Jacob L. Vigdor & Charles T. Clotfelter, 2003. "Retaking the SAT," Journal of Human Resources, University of Wisconsin Press, vol. 38(1).
    13. Brian A. Jacob & Steven D. Levitt, 2003. "Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheating," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 843-877.
    14. Dynarski, Mark & Gleason, Philip, 1993. "Using scholastic aptitude test scores as indicators of state educational performance," Economics of Education Review, Elsevier, vol. 12(3), pages 203-211, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jesse M. Rothstein, 2006. "Good Principals or Good Peers? Parental Valuation of School Characteristics, Tiebout Equilibrium, and the Incentive Effects of Competition among Jurisdictions," American Economic Review, American Economic Association, vol. 96(4), pages 1333-1350, September.
    2. Kaitlin Anderson & Gema Zamarro & Jennifer Steele & Trey Miller, 2021. "Comparing Performance of Methods to Deal With Differential Attrition in Randomized Experimental Evaluations," Evaluation Review, , vol. 45(1-2), pages 70-104, February.
    3. Card, David & Rothstein, Jesse, 2007. "Racial segregation and the black-white test score gap," Journal of Public Economics, Elsevier, vol. 91(11-12), pages 2158-2184, December.
    4. Goodman, Joshua & Hurwitz, Michael & Smith, Jonathan & Fox, Julia, 2015. "The relationship between siblings’ college choices: Evidence from one million SAT-taking families," Economics of Education Review, Elsevier, vol. 48(C), pages 75-85.
    5. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    6. Goodman, Joshua & Hurwitz, Michael & Smith, Jonathan & Fox, Julia, 2016. "Reprint of “The relationship between siblings’ college choices: Evidence from one million SAT-taking families”," Economics of Education Review, Elsevier, vol. 51(C), pages 125-135.
    7. Guyonne Kalb & Sholeh Maani, 2011. "How important are omitted variables, censored scores and self-selection in analysing high-school academic achievement?," Australian Journal of Labour Economics (AJLE), Bankwest Curtin Economics Centre (BCEC), Curtin Business School, vol. 14(3), pages 307-332.
    8. Sezgin Polat & Jean-Jacques Paul, 2016. "How to predict university performance: a case study from a prestigious Turkish university?," Investigaciones de Economía de la Educación volume 11, in: José Manuel Cordero Ferrera & Rosa Simancas Rodríguez (ed.), Investigaciones de Economía de la Educación 11, edition 1, volume 11, chapter 22, pages 423-434, Asociación de Economía de la Educación.
    9. Raj Aggarwal & Joanne Goodell & John Goodell, 2014. "Culture, Gender, and GMAT Scores: Implications for Corporate Ethics," Journal of Business Ethics, Springer, vol. 123(1), pages 125-143, August.
    10. Sarena Goodman, 2013. "Learning from the test: raising selective college enrollment by providing information," Finance and Economics Discussion Series 2013-69, Board of Governors of the Federal Reserve System (U.S.).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:mpr:mprres:5535 is not listed on IDEAS
    2. Card, David & Rothstein, Jesse, 2007. "Racial segregation and the black-white test score gap," Journal of Public Economics, Elsevier, vol. 91(11-12), pages 2158-2184, December.
    3. Card, David & Rothstein, Jesse, 2007. "Racial segregation and the black-white test score gap," Journal of Public Economics, Elsevier, vol. 91(11-12), pages 2158-2184, December.
    4. repec:pri:cepsud:109rothstein is not listed on IDEAS
    5. Gordon B. Dahl, 2002. "Mobility and the Return to Education: Testing a Roy Model with Multiple Markets," Econometrica, Econometric Society, vol. 70(6), pages 2367-2420, November.
    6. Hægeland, Torbjørn & Raaum, Oddbjørn & Salvanes, Kjell G., 2012. "Pennies from heaven? Using exogenous tax variation to identify effects of school resources on pupil achievement," Economics of Education Review, Elsevier, vol. 31(5), pages 601-614.
    7. Angrist, Joshua D., 1997. "Conditional independence in sample selection models," Economics Letters, Elsevier, vol. 54(2), pages 103-112, February.
    8. Jacob L. Vigdor & Charles T. Clotfelter, 2003. "Retaking the SAT," Journal of Human Resources, University of Wisconsin Press, vol. 38(1).
    9. David Card & Jesse Rothstein, 2005. "Racial Segregation and the Black-White Test Score Gap," Working Papers 879, Princeton University, Department of Economics, Industrial Relations Section..
    10. Martin Huber & Anna Solovyeva, 2020. "Direct and Indirect Effects under Sample Selection and Outcome Attrition," Econometrics, MDPI, vol. 8(4), pages 1-25, December.
    11. Committee, Nobel Prize, 2021. "Answering causal questions using observational data," Nobel Prize in Economics documents 2021-2, Nobel Prize Committee.
    12. Martin Huber, 2014. "Treatment Evaluation in the Presence of Sample Selection," Econometric Reviews, Taylor & Francis Journals, vol. 33(8), pages 869-905, November.
    13. Card, David & Payne, A. Abigail, 2002. "School finance reform, the distribution of school spending, and the distribution of student test scores," Journal of Public Economics, Elsevier, vol. 83(1), pages 49-82, January.
    14. Southwick, Lawrence Jr & Gill, Indermit S., 1997. "Unified salary schedule and student SAT scores: Adverse effects of adverse selection in the market for secondary school teachers," Economics of Education Review, Elsevier, vol. 16(2), pages 143-153, April.
    15. Tahir Andrabi & Jishnu Das & Asim Ijaz Khwaja & Tristan Zajonc, 2011. "Do Value-Added Estimates Add Value? Accounting for Learning Dynamics," American Economic Journal: Applied Economics, American Economic Association, vol. 3(3), pages 29-54, July.
    16. Fernando Rios-Avila & Gustavo Canavire-Bacarreza, 2018. "Standard-error correction in two-stage optimization models: A quasi–maximum likelihood estimation approach," Stata Journal, StataCorp LP, vol. 18(1), pages 206-222, March.
    17. William Greene, 2001. "Estimating Econometric Models With Fixed Effects," Working Papers 01-10, New York University, Leonard N. Stern School of Business, Department of Economics.
    18. Piopiunik, Marc & Schwerdt, Guido & Woessmann, Ludger, 2013. "Central school exit exams and labor-market outcomes," European Journal of Political Economy, Elsevier, vol. 31(C), pages 93-108.
    19. Xi He, 2012. "Two Sides of a Coin: Endogenous and Exogenous Effects of Corporate Diversification on Firm Value," International Review of Finance, International Review of Finance Ltd., vol. 12(4), pages 375-397, December.
    20. David Card & Martin D. Dooley & A. Abigail Payne, 2010. "School Competition and Efficiency with Publicly Funded Catholic Schools," American Economic Journal: Applied Economics, American Economic Association, vol. 2(4), pages 150-176, October.
    21. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    22. P.W. Miller & S. Rummery, 1989. "Gender Wage Discrimination in Australia: A reassessment," Economics Discussion / Working Papers 89-21, The University of Western Australia, Department of Economics.

    More about this item

    JEL classification:

    • C24 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Truncated and Censored Models; Switching Regression Models; Threshold Regression Models
    • I2 - Health, Education, and Welfare - - Education
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:14265. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.