Hostname: page-component-745bb68f8f-b95js Total loading time: 0 Render date: 2025-02-10T06:34:56.879Z Has data issue: false hasContentIssue false
  • English
  • Français

A Convenient Solution: Using MTurk To Sample From Hard-To-Reach Populations

Published online by Cambridge University Press:  28 July 2015

Nicholas A. Smith ,
Isaac E. Sabat ,
Larry R. Martinez ,
Kayla Weaver  and
Shi Xu
Nicholas A. Smith*
Affiliation:
The Pennsylvania State University
Isaac E. Sabat
Affiliation:
George Mason University
Larry R. Martinez
Affiliation:
The Pennsylvania State University
Kayla Weaver
Affiliation:
The Pennsylvania State University
Shi Xu
Affiliation:
The Pennsylvania State University
*
Correspondence concerning this article should be addressed to Nicholas Smith, School of Hospitality Management, The Pennsylvania State University, 201 Mateer Building, University Park, PA 16802. E-mail: nas277@psu.edu
Get access Rights & Permissions [Opens in a new window]

Extract

We agree with Landers and Behrend's (2015) proposition that Amazon's Mechanical Turk (MTurk) may provide great opportunities for organizational research samples. However, some groups are characteristically difficult to recruit because they are stigmatized or socially disenfranchised (Birman, 2005; Miller, Forte, Wilson, & Greene, 2006; Sullivan & Cain, 2004; see Campbell, Adams, & Patterson, 2008, for a review). These groups may include individuals who have not previously been the focus of much organizational research, such as those of low socioeconomic status; individuals with disabilities; lesbian, gay, bisexual, or transgender (LGBT) individuals; or victims of workplace harassment. As Landers and Behrend (2015) point out, there is an overrepresentation of research using “Western, educated, industrialized, rich, and democratic” participants. It is important to extend research beyond these samples to examine workplace phenomena that are specific to special populations. We contribute to this argument by noting the particular usefulness that MTurk can provide for sampling from hard-to-reach populations, which we characterize as groups that are in the numerical minority in terms of nationwide representation. To clarify, we focus our discussion on populations that are traditionally hard to reach in the context of contemporary organizational research within the United States.

Type
Commentaries
Information
Industrial and Organizational Psychology , Volume 8 , Issue 2 , June 2015 , pp. 220 - 228
Copyright
Copyright © Society for Industrial and Organizational Psychology 2015 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Birman, D. (2005). Ethical issues in research with immigrants and refugees. In Trimble, J. E. & Fisher, C. B. (Eds.), The handbook of ethical research with ethnocultural populations and communities (pp. 155–178). Thousand Oaks, CA: Sage.Google Scholar
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3–5. doi:10.1177/1745691610393980Google Scholar
Campbell, R., Adams, A. E., & Patterson, D. (2008). Methodological challenges of collecting evaluation data from traumatized clients/consumers: A comparison of three methods. American Journal of Evaluation, 29, 369–381. doi:10.1177/1098214008320736Google Scholar
Carr, A. (2014). An exploration of Mechanical Turk as a feasible recruitment platform for cancer survivors (Unpublished undergraduate honors theses, Paper 59). Retrieved from http://scholar.colorado.edu/cgi/viewcontent.cgi?article=1058&context=honr_thesesGoogle Scholar
Eisenberg, M., & Wechsler, H. (2003). Substance use behaviors among college students with same-sex and opposite-sex experience: Results from a national study. Addictive Behaviors, 28, 899–913. doi:10.1016/S0306-4603(01)00286-6Google Scholar
Feitosa, J., Joseph, D. L., & Newman, D. A. (2015). Crowdsourcing and personality measurement equivalence: A warning about countries whose primary language is not English. Personality and Individual Differences, 75, 47–52. doi:10.1016/j.paid.2014.11.017CrossRefGoogle Scholar
Gonzalez, J. A., & DeNisi, A. S. (2009). Cross-level effects of demography and diversity climate on organizational attachment and firm effectiveness. Journal of Organizational Behavior, 30, 21–40. doi:10.1002/job.498Google Scholar
Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004). Should we trust web-based studies? A comparative analysis of six preconceptions about Internet questionnaires. American Psychologist, 59, 93–104. doi:10.1037/0003-066X.59.2.93CrossRefGoogle ScholarPubMed
Landers, R. N., & Behrend, T. S. (2015). An inconvenient truth: Arbitrary distinctions between organizational, Mechanical Turk, and other convenience samples. Industrial and Organizational Psychology: Perspectives on Science and Practice. Advance online publication. doi:10.1017/iop.2015.14Google Scholar
Law, C. L., Martinez, L. R., Ruggs, E. N., Hebl, M. R., & Akers, E. (2011). Trans-parency in the workplace: How the experiences of transsexual employees can be improved. Journal of Vocational Behavior, 79, 710–723. doi:10.1016/j.jvb.2011.03.018CrossRefGoogle Scholar
Lenox, R. A., & Subich, L. M. (1994). The relationship between self-efficacy beliefs and inventoried vocational interests. The Career Development Quarterly, 42, 302–313. doi:10.1002/j.2161-0045.1994.tb00514.xCrossRefGoogle Scholar
Levine, S., Ancill, R. J., & Roberts, A. P. (1989). Assessment of suicide risk by computer-delivered self-rating questionnaire: Preliminary findings. Acta Psychiatrica Scandinavica, 80, 216–220. doi:10.1111/j.1600-0447.1989.tb01330.xCrossRefGoogle ScholarPubMed
Locke, S. D., & Gilbert, B. O. (1995). Method of psychological assessment, self-disclosure, and experiential differences: A study of computer, questionnaire, and interview assessment formats. Journal of Social Behavior & Personality, 10, 255–263. Retrieved from http://psycnet.apa.org/psycinfo/1995-31530-001Google Scholar
McCormack, M. (2014). Innovative sampling and participant recruitment in sexuality research. Journal of Social and Personal Relationships, 31, 475–481. doi:10.1177/0265407514522889Google Scholar
McKay, P. F., Avery, D. R., & Morris, M. A. (2009). A tale of two climates: Diversity climate from subordinates’ and managers’ perspectives and their role in store unit sales performance. Personnel Psychology, 62, 767–791. doi:10.1111/j.1744-6570.2009.01157.xGoogle Scholar
Miller, R. L., Forte, D., Wilson, B. D. M., & Greene, G. J. (2006). Protecting sexual minority youth from research risks: Conflicting perspectives. American Journal of Community Psychology, 37, 341–348. doi:10.1007/s10464-006-9053-4CrossRefGoogle ScholarPubMed
Papa, A., Lancaster, N. G., & Kahler, J. (2014). Commonalities in grief responding across bereavement and non-bereavement losses. Journal of Affective Disorders, 161, 136–143. doi:10.1016/j.jad.2014.03.018Google Scholar
Skitka, L. J., & Sargis, E. G. (2005). Social psychological research and the Internet: The promise and peril of a new methodological frontier. In Amichai-Hamburger, Y. (Ed.), The social net: The social psychology of the Internet (pp. 1–26). Oxford, United Kingdom: Oxford University Press.Google Scholar
Storey, D. J. (1994). Understanding the small business sector. New York, NY: Routledge.Google Scholar
Sullivan, C. M., & Cain, D. (2004). Ethical and safety considerations when obtaining information from or about battered women for research purposes. Journal of Interpersonal Violence, 19, 603–618. doi:10.1177/0886260504263249CrossRefGoogle ScholarPubMed
Tenenbaum, R. Z., Byrne, C. J., & Dahling, J. J. (2014). Interactive effects of physical disability severity and age of disability onset on RIASEC self-efficacies. Journal of Career Assessment, 22, 274–289. doi:10.1177/1069072713493981CrossRefGoogle Scholar
Turner, C. F., Ku, L., Rogers, S. M., Lindberg, L. D., Pleck, J. H., & Sonenstein, F. L. (1998). Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science, 280, 867–873. doi:10.1126/science.280.5365.867CrossRefGoogle ScholarPubMed
U.S. Census Bureau. (2011). Statistics of U.S. businesses (SUSB) main [Data file]. Retrieved from http://www2.census.gov/econ/susb/data/2011/us_state_naicssector_small_emplsize_2011.xlsGoogle Scholar
Vaughn, A. A., Cronan, S. B., & Beavers, A. J. (2015). Resource effects on in-group boundary formation with regard to sexual identity. Social Psychological and Personality Science, 6, 292–299. doi:10.1177/1948550614559604Google Scholar