Hostname: page-component-745bb68f8f-b95js Total loading time: 0 Render date: 2025-02-04T09:42:11.011Z Has data issue: false hasContentIssue false
  • English
  • Français

Improving Data Quality: Actors, Incentives, and Capabilities

Published online by Cambridge University Press:  04 January 2017

Yoshiko M. Herrera  and
Devesh Kapur
Yoshiko M. Herrera
Affiliation:
Department of Government, Harvard University, Davis Center, #S301, 1730 Cambridge Street, Cambridge, MA 02138. e-mail: herrera@fas.harvard.edu (corresponding author)
Devesh Kapur
Affiliation:
Centre for Advanced Study of India, University of Pennsylvania, 3600 Market Street, Suite 560, Philadelphia, PA 19104. e-mail: dkapur@sas.upenn.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper examines the construction and use of data sets in political science. We focus on three interrelated questions: How might we assess data quality? What factors shape data quality? and How can these factors be addressed to improve data quality? We first outline some problems with existing data set quality, including issues of validity, coverage, and accuracy, and we discuss some ways of identifying problems as well as some consequences of data quality problems. The core of the paper addresses the second question by analyzing the incentives and capabilities facing four key actors in a data supply chain: respondents, data collection agencies (including state bureaucracies and private organizations), international organizations, and finally, academic scholars. We conclude by making some suggestions for improving the use and construction of data sets.

It is a capital mistake, Watson, to theorise before you have all the evidence. It biases the judgment.

—Sherlock Holmes in “A Study in Scarlet”

Statistics make officials, and officials make statistics.”

—Chinese proverb

Type
Research Article
Information
Political Analysis , Volume 15 , Issue 4 , Autumn 2007 , pp. 365 - 386
Copyright
Copyright © The Author 2007. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abdelal, Rawi, Herrera, Yoshiko, Johnston, Alastair I., and McDermott, Rose. 2006. Identity as a variable. Perspectives on Politics 4: 695–711.Google Scholar
Achen, Christopher. 1985. Proxy variables and incorrect signs on regression coefficients. Political Methodology 11: 288–316.Google Scholar
Adcock, Robert, and Collier, David. 2001. Measurement validity: A shared standard for qualitative and quantitative research. American Political Science Review 95: 529–46.Google Scholar
Aiyar, Swamininathan Anklesaria. 2001. Poverty-stricken statistics. Economic Times, September 1, 2001.Google Scholar
Becker, Gary. 1968. Crime and punishment: An economic approach. Journal of Political Economy 76: 169–217.Google Scholar
Berkowitz, Daniel, Pistor, Katharina, and Richard, Jean-Francois. 2003. Economic development, legality, and the transplant effect. European Economic Review 47: 165–95.CrossRefGoogle Scholar
Brown, David, and Hunter, Wendy. 1999. Democracy and social spending in Latin America. American Political Science Review 93: 779–90.Google Scholar
Bueno de Mesquita, Bruce. 1981. The war trap. New Haven, CT: Yale University Press.Google Scholar
Cederman, Lars-Erik, and Girardin, Luc. 2005. Beyond fractionalization: Mapping ethnicity onto nationalist insurgencies. American Political Science Reviews 101: 173–85.Google Scholar
Chander, Ramesh. 1988. Strengthening information systems in SSA. Washington, DC: World Bank.Google Scholar
Chandra, Kanchan, ed. 2001. Symposium: Cumulative findings in the study of ethnic politics 2001. APSA-CP Newsletter 12(1): 7–25.Google Scholar
Chandra, Kanchan, Giffelquist, Rachel, Metz, Daniel, Wendt, Chris, and Ziegfeld, Adam. 2005. A constructivist dataset on ethnicity and institutions. In Identity as a variable, ed. Abdelal, Rawi, Herrera, Yoshiko, Ian Johnston, Alastair and McDermott, Rose. New York: New York University.Google Scholar
Cheibub, Jose Antonio. 1999. Data optimism in comparative politics: The importance of being earnest. APSA-CP 10(2): 21–5.Google Scholar
Collier, David, and Adcock, Robert. 1999. Democracy and dichotomies: A pragmatic approach to choices about concepts. Annual Review of Political Science 2: 537–65.Google Scholar
Coppedge, Michael. 2002. Democracy and dimensions: Comments on Munck and Verkuilen. Comparative Political Studies 35: 35–9.Google Scholar
Gleditsch, Kristian S., and Ward, Michael D. 1997. Double take: A reexamination of democracy and autocracy in modern polities. The Journal of Conflict Resolution 41: 361–83.Google Scholar
Goertz, Gary. 2005. Social science concepts: A user's guide. Princeton, NJ: Princeton University Press.Google Scholar
Goodhart, Charles. 1989. Money, information and uncertainty. 2nd ed. Cambridge, MA: MIT Press.Google Scholar
Hoskin, Keith. 1996. The ‘awful idea of accountability’: Inscribing people into the measurement of objects. In Accountability: Power, ethos, and the technologies of managing, ed. Munro, Rolland and Mouritsen, Jan, 265–82. London: Thomson International.Google Scholar
Kapur, Devesh, Lewis, John P., and Webb, Richard. 1997. The World Bank: Its first half century. Washington, DC: Brookings Institution.Google Scholar
Kaufmann, D., Kraay, A., and Mastruzzi, M. 2002. Governance matters III: Governance indicators for 1996-2002. World Bank Policy Research Working paper 3106.Google Scholar
Kaufmann, D., Kraay, A., and Zoido-Lobatón, P. 1999a. Aggregating governance indicators. World Bank Working paper 2195.Google Scholar
Kaufmann, D., Kraay, A., and Zoido-Lobatón, P. 1999b. Governance matters. World Bank Working paper 2196.Google Scholar
King, Gary, Murray, Christopher J. L., Salomon, Joshua A., and Tandon, Ajay. 2004. Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review 98: 191–207.Google Scholar
King, Gary, and Wand, Jonathan. 2007. Comparing incomparable survey responses: Evaluating and selecting anchoring vignettes. Political Analysis 15: 46–66.CrossRefGoogle Scholar
Krueger, Alan, and Laitin, David. 2004. Misunderestimating terrorism. Foreign Affairs 83(5): 8–13.Google Scholar
Kynge, James. 1999. China uncovers falsified accounts at state groups. Financial Times, December 24, 1999.Google Scholar
Laitin, David, and Posner, Daniel. 2001. The implications of constructivism for constructing ethnic fractionalization indices. APSA-CP Newsletter 12(1): 13–17.Google Scholar
Marshall, Monty G., Gurr, Ted Robert, Davenport, Christian, and Jaggers, Keith. 2002. Polity IV, 1800-1999: Comments on Munck and Verkuilen. Comparative Political Studies 35: 40–5.Google Scholar
Mishler, William, and Rose, Richard. 2001. Political support for incomplete democracies: Realist vs. idealist theories and measures. International Political Science Review 22: 303–20.Google Scholar
Munck, Gerardo L., and Verkuilen, Jay. 2002a. Conceptualizing and measuring democracy: Evaluating alternative indices. Comparative Political Studies 35: 5–34.Google Scholar
Munck, Gerardo L., and Verkuilen, Jay. 2002b. Generating better data: A response to discussants. Comparative Political Studies 35: 52–7.Google Scholar
Nagraj, R. 1999. How good are India's industrial statistics? An exploratory note. Economic and Political Weekly 34: 350–5.Google Scholar
Posner, Daniel N. 2004. Measuring ethnic fractionalization in Africa. American Journal of Political Science 48: 849–63.Google Scholar
Rawski, Thomas G. 2000. China by the numbers: How reform affected Chinese economic statistics. http://www.pitt.edu/∼tgrawski/papers2000/REVD00.HTM (accessed July 26, 2005).Google Scholar
Rose, Richard. 2002/2003. Economies in transition: A multidimensional approach to a cross-cultural problem. East European Constitutional Review 11/12 (4/1): 62–70.Google Scholar
Rozanski, J., and Yeats, A. 1994. On the (in)accuracy of economic observations: An assessment of trends in the reliability of international trade statistics. Journal of Development Economics 44: 103–30.CrossRefGoogle Scholar
Slantchev, Branislav L. 2004. How initiators end their wars. American Journal of Political Science 48: 813–29.Google Scholar
Srinivasan, T. N. 1994. Data base for development analysis: An overview. Journal of Development Economics 44: 3–27.CrossRefGoogle Scholar
Swonk, Diane. 2000. The value of good data. Financial Times, September 27, 2000.Google Scholar
The Economist. 2002. Roll over, Enron. August 3, p. 44.Google Scholar
Treier, Shawn, and Jackman, Simon. 2006. Democracy as a latent variable. http://www.tc.umn.edu/∼satreier/DemocracyAsLatentVariable_041906.pdf.Google Scholar
United Nations. 2004. Current status of the collection of international migration statistics. World Economic and Social Survey. New York: United Nations, 211–7.Google Scholar
United Nations Development Programme (UNDP). 2003. Human development report, 2003. New York: Oxford University Press.Google Scholar
Velkoff, Victoria A., and Miller, Jane E. 1995. Trends and differentials in infant mortality in the Soviet Union, 1970-90: How much is due to misreporting? Population Studies 49: 241–58.Google Scholar
Wallack, Jessica. 2006. The highs and lows of revenue estimating: Explaining bias and inaccuracy. San Diego, CA: University of California. http://irpshome.ucsd.edu/faculty/jwallack/revest_7_2006.pdf.Google Scholar
Ward, Michael D. 2002. Green binders in cyberspace: A modest proposal. Comparative Political Studies 35: 46–51.Google Scholar
Watson, Reg, and Pauly, Daniel. 2001. Systematic distortions in world fisheries catch trends. Nature 414: 534–6.Google Scholar
White, Halbert. 1994. Estimation, inference, and specification analysis. New York: Cambridge University Press.Google Scholar
Widner, Jennifer. 1999. Maintaining our knowledge base. APSA-CP 10(2): 17–21.Google Scholar
Wilkinson, Steve. 2002. Memo on developing better indicators of ethnic and non-ethnic identities. http://www.duke.edu/web/licep/5/wilkinson/wilkinson.pdf (accessed July 26, 2005).Google Scholar
Wong, George Y., and Mason, William M. 1991. Contextually specific effects and other generalizations of the hierarchical linear model for comparative analysis. Journal of the American Statistical Association 86: 487–503.CrossRefGoogle Scholar
World Bank. 2000. India: Policies to reduce poverty and accelerate sustainable development. Report No. 19471-IN. Washington, DC: World Bank.Google Scholar
Yeats, Alexander. 1990. On the accuracy of African observations: Do Sub-Saharan trade statistics mean anything? World Bank Economic Review 2: 135–56.Google Scholar