Hostname: page-component-745bb68f8f-v2bm5 Total loading time: 0 Render date: 2025-01-11T00:58:20.796Z Has data issue: false hasContentIssue false
  • English
  • Français

Estimating the Impact of State Policies and Institutions with Mixed-Level Data

Published online by Cambridge University Press:  25 January 2021

David M. Primo ,
Matthew L. Jacobsmeier  and
Jeffrey Milyo
David M. Primo
Affiliation:
University of Rochester
Matthew L. Jacobsmeier
Affiliation:
University of Rochester
Jeffrey Milyo
Affiliation:
University of Missouri at Columbia
Get access Rights & Permissions [Opens in a new window]

Abstract

Researchers are often interested in the effects of state policies and institutions on individual behavior or other outcomes in sub-state-level observational units, such as election results in state legislative districts. In this article, we examine the issue of clustered data in state and local politics research and the analytical problems it can cause. Standard estimation methods applied in most regression models do not properly account for the clustering of observations within states, leading analysts to overstate the statistical significance of coefficient estimates, especially of state-level factors. We discuss the theory behind two approaches for dealing with clustering—clustered standard errors and multilevel modeling—and argue that calculating clustered standard errors is a more straightforward and practical approach, especially when working with large datasets or many cross-level interactions. We demonstrate the relevance of this topic by replicating a recent study of the effects of state post-registration laws on voter turnout (Wolfinger, Highton, and Mullin 2005).

Type
The Practical Researcher
Information
State Politics & Policy Quarterly , Volume 7 , Issue 4 , Winter 2007 , pp. 446 - 459
Copyright
Copyright © The American Political Science Association, 2007

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abbe, Owen G., and Herrnson, Paul S.. 2003. “Campaign Professionalism in State Legislative Elections.” State Politics and Policy Quarterly 3(3):223–45.CrossRefGoogle Scholar
Beck, Nathaniel, and Katz, Jonathan N.. 1995. “What to Do (and Not to Do) with Time-Series Cross-Section Data.” American Political Science Review 89(3):634–47.CrossRefGoogle Scholar
Bertrand, Marianne, Duflo, Esther, and Mullainathan, Senhil. 2004. “How Much Should We Trust Differences-in-Differences Estimates?” Quarterly Journal of Economics 119(1):249–75.CrossRefGoogle Scholar
Bonneau, Chris W. 2005. “What Price Justice(s)? Understanding Campaign Spending in State Supreme Court Elections.” State Politics and Policy Quarterly 5(2):107–25.CrossRefGoogle Scholar
Bowers, Jake, and Drake, Katherine W.. 2005. “Applying a Two-Step Strategy to the Analysis of Cross-National Public Opinion Data.” Political Analysis 13(4):301–26.Google Scholar
Branton, Regina P. 2004. “Voting in Initiative Elections: Does the Context of Racial and Ethnic Diversity Matter?” State Politics and Policy Quarterly 4(3):294–317.CrossRefGoogle Scholar
Bryk, Stephen W, and Raudenbush, Anthony S.. 1992. Hierarchical Linear Models: Applications and Data Analysis Methods. Thousand Oaks, CA: Sage Press.Google Scholar
Buckley, Jack, and Westerland, Chad. 2004. “Duration Dependence, Functional Form, and Corrected Standard Errors: Improving EHA Models of State Policy Diffusion.” State Politics and Policy Quarterly 4(1):94–113.CrossRefGoogle Scholar
Bullock, Charles S., Hood, M.V. III, and Clark, Richard. 2005. “Punch Cards, Jim Crow, and Al Gore: Explaining Voter Trust in the Electoral System in Georgia, 2000.” State Politics and Policy Quarterly 5(3):283–94.CrossRefGoogle Scholar
Carson, Jamie L., and Crespin, Michael H.. 2004. “The Effect of State Redistricting Methods on Electoral Competition in United States House of Representatives Races.” State Politics and Policy Quarterly 4(4):455–69.CrossRefGoogle Scholar
Francia, Peter L., and Herrnson, Paul S.. 2004. “The Synergistic Effect of Campaign Effort and Election Reform on Voter Turnout in State Legislative Elections.” State Politics and Policy Quarterly 4(1):74–91.CrossRefGoogle Scholar
Franzese, Robert J. Jr. 2005. “Empirical Strategies for Various Manifestations of Multilevel Data.” Political Analysis 13(4):430–46.CrossRefGoogle Scholar
Froot, Kenneth A. 1989. “Consistent Covariance Matrix Estimation with Cross-Sectional Dependence and Heteroskedasticity in Financial Data.” Journal of Financial and Quantitative Analysis 24(3):333–55.CrossRefGoogle Scholar
Hansen, Christian. 2005. “Asymptotic Properties of a Robust Variance Matrix Estimator for Panel Data When T is Large.” Working paper, University of Chicago.Google Scholar
Huber, Peter J. 1967. “The Behavior of Maximum Likelihood Estimates under Nonstandard Conditions.” In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability. Berkeley, CA: University of California Press.Google Scholar
Kedar, Orit, and Phillips Shively, W.. 2005. “Introduction to the Special Issue.” Political Analysis 13(4):297–300.CrossRefGoogle Scholar
Kennedy, Peter. 2003. A Guide to Econometrics, 5th ed. Cambridge, MA: MIT Press.Google Scholar
Kezdi, Gabor. 2003. “Robust Standard Error Estimation in Fixed-Effects Panel Models.” Working paper, Central European University.CrossRefGoogle Scholar
King, Gary. 1997. A Solution to the Ecological Inference Problem: Reconstructing Individual Behavior from Aggregate Data. Princeton, NJ: Princeton University Press.Google Scholar
Lacy, Robert. 2005. “The Electoral Allure of Direct Democracy: The Effect of Initiative Salience on Voting, 1990–96.” State Politics and Policy Quarterly 5(2):168–81.Google Scholar
Moulton, Brent R. 1990. “An Illustration of a Pitfall in Estimatingthe Effects of Aggregate Variables in Micro Units.” Review of Economics and Statistics 72(2):334–38.CrossRefGoogle Scholar
Newey, Whitney K., and West, Kenneth D.. 1987. “A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix.” Econometrica 55(3):703–8.CrossRefGoogle Scholar
Raudenbush, Stephen W., and Bryk, Anthony S.. 2002. Hierarchical Linear Models: Applications and Data Analysis Methods. 2nd ed. Thousand Oaks, CA: Sage Press.Google Scholar
Rogers, William. 1993. “sg17: Regression Standard Errors in Clustered Samples.” Stata Technical Bulletin 13:19–23.Google Scholar
Steenbergen, Marco R., and Jones, Bradford S.. 2002. “Modeling Multilevel Data Structures.” American Journal of Political Science 46(1):218–37.CrossRefGoogle Scholar
Tolbert, Caroline J., and Grummel, John A.. 2003. “Revisiting the Racial Threat Hypothesis: White Voter Support for California's Proposition 209.” State Politics and Policy Quarterly 3(2):183–202.Google Scholar
Tolbert, Caroline J., McNeal, Ramona S., and Smith, Daniel A.. 2003. “Enhancing Civic Engagement: The Effect of Direct Democracy on Political Participation and Knowledge.” State Politics and Policy Quarterly 3(1):23–41.CrossRefGoogle Scholar
White, Halbert. 1980. “A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity.” Econometrica 48(4):817–38.CrossRefGoogle Scholar
Williams, Rick L. 2000. “A Note on Robust Variance Estimation for Cluster-Correlated Data.” Biometrics 56:645–6.CrossRefGoogle ScholarPubMed
Wolfinger, Raymond E., Highton, Benjamin, and Mullin, Megan. 2005. “How Postregistration Laws Affect the Turnout of Citizens Registered to Vote.” State Politics and Policy Quarterly 5(1):1–23.CrossRefGoogle Scholar
Wooldridge, Jeffrey M. 2002. Econometric Analysis of Cross Section and Panel Data. Cambridge, MA: MIT Press.Google Scholar
Wooldridge, Jeffrey M. 2003. “Cluster-Sample Methods in Applied Econometrics.” American Economic Review 93(2):133–8.CrossRefGoogle Scholar