Highlights
-
• Strict inclusion/exclusion criteria, such as being diagnosed within the past 5 years, experiencing cancer pain during the past week, and having depressive symptoms in the past 2 weeks, made participant recruitment more challenging.
-
• Compounded difficulties due to cultural stigma attached to cancer, depressive symptoms, and expressing pain further complicated recruitment/retention.
-
• Community outreach was challenging, and a competitive research environment with other studies added to difficulties.
Introduction
A big challenge for researchers working with racial/ethnic minorities consists of the recruitment and retention of participants, frequently reported as the under-representation of racial/ethnic minorities in research [Reference Taani, Zabler, Fendrich and Schiffman1–Reference George, Duran and Norris3]. Suspicion about clinical research has been a major obstacle in recruiting racial/ethnicity minorities, who often question the benefits they might receive from participating in such studies [Reference Horowitz, Sabin and Ramos2,Reference George, Duran and Norris3]. Especially Asian Americans have a comparatively low participation rate in clinical trials, accounting for only 2% to 3% of participants in the U.S.; rates are higher for Black (15%–19%), Hispanic (10%–21%), and White (76%–81%) participants [Reference Lolic, Araojo, Okeke and Woodcock4]. Among Asian Americans, cultural hesitance stemming from misbeliefs and misconceptions related to specific conditions (e.g., cancer, mental illness) has been identified as a major contributing factor [Reference Su-Kubricht, Chen, Guo and Miller5].
Since the COVID-19 pandemic, the use of computer and mobile technologies in both quantitative and qualitative research has grown significantly [Reference Keen, Lomeli-Rodriguez and Joffe6,Reference Saltzman, Terzis, Hansel, Blakey, Logan and Bordnick7], and researchers are incorporating these technologies into interventions to promote health, prevent disease, and manage chronic conditions. In the early stages of this shift, there was optimism that technology would facilitate the recruitment and retention of research participants from hard-to-reach populations such as racial and ethnic minorities, given the anonymity and convenience of non–face-to-face interactions [Reference Im, Lee and Ji8]. However, technology-based research also faces multiple challenges in participant recruitment – the “digital divide,” limited digital literacy, mistrust, cultural irrelevance, engagement difficulties, technical and logistical barriers, and ethical concerns [Reference Ko, Gao and Wang9,Reference Chee, Ji and Kim10]. Unfortunately, despite such findings, little is still known about practical issues in recruiting and retaining Asian Americans into technology-based studies. In a PubMed search for the keywords, Asian Americans, recruitment, retention, and technology-based intervention, only two articles were retrieved; both are by the authors of the present study [Reference Im, Kim and Xu11,Reference Im, Chee and Kim12].
In this article, we therefore explore practical issues in recruiting and retaining racial and ethnic minorities in technology-based interventions, drawing on insights from a parent study of a web app-based intervention for Asian American breast cancer survivors with pain and depressive symptoms. Even though effective treatments for pain and depression are available, this specific population often underutilizes them because of strong stigma surrounding pain and mental health issues [Reference Tsai, Nusrath and Zhu13]. Thus, these individuals would be more difficult to recruit to a study compared with other Asian American subgroups. Technology-based interventions offer a solution, and identifying and highlighting challenges to such interventions can support researchers in addressing potential barriers to recruitment and retention in future studies involving computer and mobile technologies.
The parent study
In our ongoing parent study, we are examining the efficacy of a culturally tailored web -based cancer pain management program for improving health outcomes of Asian American breast cancer survivors with depressive symptoms. In that study and this, we consider depression as “a common and serious mental disorder that negatively affects how you feel, think, act, and perceive the world;” depressive symptoms include any symptoms related to depression, ranging from mild to severe [14]. The Web-based intervention, based on Bandura’s social cognitive theory [Reference Bandura15], offers individual and group coaching/support from interventionists and peers, focusing on breast cancer survivorship, cancer pain management, and depressive symptom management. Its goal is to change women’s attitudes, self-efficacy, perceived barriers, and social influences related to cancer pain management through various components, including subethnic social media sites, educational modules, online resources, a chat function, and machine learning-based individualization. The research initially targets Chinese, Korean, and Japanese women in the U.S., because Chinese are the largest Asian American group, Koreans are the most rapidly increasing group among Asian Americans [16], and Japanese have the highest breast cancer prevalence [17]. Therefore, the program is available in English, simplified and traditional Chinese, Korean, and Japanese. In this article, we focus on recruitment and retention challenges faced during data collection. More detailed information on the study is available elsewhere [Reference Im, Kim, Quan and Chee18]. The parent study was approved by the Institutional Review Board of the University of Texas at Austin (approval no. 00004807).
Methods
Approach
All research team members kept their own research diaries, and weekly research team meeting minutes were kept on Microsoft Teams. All these records were examined with the use of Weber’s content analysis [Reference Weber19]. A predetermined theoretical framework was not used; instead, themes for recruitment and retention issues emerged from the data. All authors coded the data independently. Words constituted the individual unit of analysis; all authors conducted line-by-line coding. The codes were categorized, with idea categories extracted from the data through iterative reviewing, re-reviewing, and comparing codes and categories to identify themes; and discrepancies were resolved in meetings to establish consensus.
Results
Issues
Recruitment and retention issues identified in this study include the following: (a) strict inclusion/exclusion criteria; (b) multiple stigmas attached to cancer, depressive symptoms, and pain; (c) lack of interest in research participation; (d) closed Asian American communities/groups; (e) frequent technological issues; and (f) potential unauthentic cases. Although the themes of closed Asian American communities/groups, frequent technological issues, and potential unauthentic cases have been reported previously in the literature [Reference Ko, Gao and Wang9–Reference Im, Chee and Kim12], the first three themes have rarely been reported.
Strict inclusion/exclusion criteria
The study’s strict inclusion/exclusion criteria made participant recruitment challenging. Eligible participants, on the basis of self-report, were women aged 18 years or older who identified as Chinese, Korean, or Japanese; had been diagnosed with breast cancer within the past 5 years; could read and write English, Mandarin (simplified or traditional), Korean, or Japanese; had internet access through computers or mobile devices; had cancer pain during the past week rated at least 2 on a 0–5 scale (where 2 indicated a somewhat bothering symptom); and scored between 1 and 10 on the Patient Health Questionnaire-9 (PHQ-9), indicating minimal to moderate depressive symptoms [Reference Kroenke, Spitzer and Williams20]. The pain threshold ensured that participants were experiencing bothersome symptoms sufficiently to seek pain management support.
Participants scoring between 1 and 10 on the PHQ-9 (range: 0–27) were included to capture relapsing and remitting depressive symptoms [Reference Kroenke, Spitzer and Williams20]. Good agreement between PHQ-9 diagnoses and those of professionals (overall accuracy, 85%; sensitivity, 75%; specificity, 90%) was reported in the literature [Reference Maurer, Raymond and Davis21]. Only women were included, because breast cancer occurs mainly in women; only about 1% of U.S. cases are men [Reference Gucalp, Traina and Eisner22], with even lower prevalence among Asian Americans [Reference Yom and Lor23]. Those under 18 years old were excluded because their cancer experience would be different from that of adults. Both those in treatment and those diagnosed within the past 5 years were eligible, because survivors beyond 5 years may have different needs. Those who participated in the research team’s previous studies were excluded. Finally, to ensure feasibility and cultural relevance [Reference Lor and Koleck24], only 4 languages – those most commonly spoken by the target population – were included [16].
Despite extensive efforts, no participants were recruited in the first 3 months of the project, due to the study’s strict inclusion/exclusion criteria. Online and offline communities and groups for Chinese, Korean, and Japanese in the U.S. were searched using the Google and Bing search engines as well as through personal connections. Each community/group’s website was reviewed for eligibility, yielding 6,472 potential communities/groups. Of those, 4,393 were contacted via email, phone, or in-person visits, and 75 study announcements were posted. Additional outreach was conducted through consultants’ national networks. However, ultimately, only 3 participants – all Japanese – were recruited.
Because of these recruitment difficulties, the pain level criterion was lowered from 2 to 1 with the approval by the funding agency. Over the next 4 months, 42 potential participants were screened, and 15 met the new criteria. This adjustment reflected evidence that Asians often underreport pain and/or related symptoms [Reference Shavers, Bakos and Sheppard25], and pain can fluctuate during survivorship, making a fixed threshold impractical. Recruitment was further complicated by participants’ shifting health status – some developed or recovered from pain or depressive symptoms during the 3-month period – highlighting the limitations of static eligibility criteria in a population with evolving needs.
Multiple stigmas attached to cancer, depressive symptoms, and pain
Cultural stigma around cancer and depression in Asian cultures is well known [Reference Kramer, Kwong, Lee and Chung26], and it posed significant challenges for the parent study, particularly in participant recruitment and retention. Early on, the research team used terms for depression that carried strong negative connotations, which deterred potential participation. In Chinese, the term 
could imply a mental disorder with strong stigma and negative personality traits, which prompted the research team to switch to 
, a phonetic translation of depressive symptoms with less stigma. A similar issue was found among Korean participants, where the term 
(the medical diagnosis of clinical depression) triggered strong stigma, unlike the terms 
(depressive symptom) and 
(feeling depressed). Although we revised the study materials with less stigmatized words, 3 participants still withdrew, citing discomfort upon realizing the study’s association with the medical diagnosis of clinical depression. Although the study’s consent form clearly stated that the study’s purpose was to improve the cancer pain experience of Asian American breast cancer survivors with “depressive symptoms,” some participants did not recognize that the study was related to depressive symptoms until they participated in the study’s information sessions.
In many Asian cultures, particularly Chinese and Korean, cancer is not only feared as a life-threatening disease, but it also carries symbolic meanings of hopelessness, suffering, and even personal failure [Reference Dong, Zheng, Chen, Wang, Zhou and Sun27–Reference Tripathi, Datta, Agrawal, Chatterjee and Ahmed31]. This perception reinforces social stigma, in which a diagnosis may be seen as a source of disgrace or punishment, resulting in feelings of guilt and isolation among survivors [Reference Jin, Xie, Zhang, Gong and Zhang28–Reference Lee, Chen, Ma, Fang, Oh and Scully30]. This stigma is further reinforced by concerns over self-image, because changes such as hair loss or mastectomy can heighten feelings of embarrassment and social isolation [Reference Tripathi, Datta, Agrawal, Chatterjee and Ahmed31]. The fear of being identified and of facing potential discriminatory attitudes within one’s community [Reference Lee, Chen, Ma, Fang, Oh and Scully30] may make some individuals reluctant to disclose their diagnosis or participate in research. This stigma extended to our research context, in which some Asian American groups declined to share study announcements, claiming that no cancer survivors existed in their communities and suggesting that even acknowledging cancer publicly was a taboo. However, when announcements were posted, several individuals discreetly reached out, indicating a hidden willingness to participate. The technology-based intervention was designed to address these barriers because it should reduce the perceived risk of identity exposure through non–face-to-face interactions, a method supported by the literature for engaging hidden populations with stigma [Reference Rodríguez-Rivas, Cangas, Cariola, Varela and Valdebenito32]. Nevertheless, the use of terms like “cancer survivors” in recruitment materials remained a deterrent.
In addition, cultural norms around pain expression posed another challenge. In traditional Asian cultures, expressing pain is sometimes perceived as a sign of weakness, and stoicism is highly valued [Reference Tung and Li33,Reference Perugino, De Angelis, Pompili and Martelletti34]. As a result, individuals may feel pressured to endure pain silently to avoid burdening their families or appearing vulnerable. Asians are expected to tolerate pain without complaint, because openly discussing discomfort could be seen as a lack of resilience or an inability to cope [Reference Edrington, Sun and Wong35]. Consequently, some potential participants underreported their pain during screening, making them ineligible and hindering accurate needs assessment.
Lack of interest in research participation
The research team often found that many potential participants referred by gatekeepers were not genuinely interested in research participation. Instead, they were primarily interested in the study’s financial incentives and withdrew their interest upon realizing that they would need to participate in the study for 3 months of weekly coaching/support sessions with only a small financial reward. Each participant received a $50 electronic gift card upon completion of the pretest and 1-month and 3-month follow-ups. Each participant also received a Fitbit that could be kept after completion of participation. Although the intervention was designed to offer information and coaching/support, many did not perceive benefits other than the financial incentives and Fitbits. Even those who initially expressed interest and began participating in the study sometimes withdrew after receiving the information and coaching/support they were interested in.
In the literature, 3 months appears to be an adequate intervention period for a technology-based behavior change intervention. The 3-month intervention period for this study was based on a literature review and our own preliminary studies [Reference Ge, Zhu, Wu and McConnell36,Reference Im, Chee and Paul37]. Yet the participants felt that the 3-month period was too long for continued participation. Some disappeared from the study because they traveled to their country of origin for medical care (translational care), were hospitalized for medical treatments in the U.S., or ignored follow-up emails and reminders, indicating a lack of sustained interest.
Closed Asian American communities/groups
The research team noted the closedness of Asian American communities/groups. For example, most churches/temples declined to share study announcements, mentioning that supporting research was not aligned with their mission and that members did not attend research-related activities. Even when one church/temple member was a community consultant for the study, the churches/temples declined to allow that member to announce the study among church/temple members. The response rate from churches/temples was notably low, with a rate of only 0.2%.
Competitions with other research studies were also noted. Many communities/groups were already helping other researchers whose studies had large incentives or a subcontract from a large grant, and they did not want to allow their members/groups to participate in other studies. Given the limited number of Asian American breast cancer survivors in a specific area, competition among researchers working with Asian American breast cancer survivors could be a major barrier for new researchers attempting to recruit potential participants from that area.
As reported in previous studies [Reference Im and Chee38,Reference Ta Park, Grill and Tsoh39], culturally matched research team members were also very important for contacting Asian American communities/groups. For example, when a Korean research assistant contacted a Chinese community/group, those contacted did not usually respond. However, when a Chinese research assistant contacted a Chinese community/group, the community or group did typically respond, regardless of whether their answer regarding participation was yes or no. Also noteworthy was that a Chinese research assistant from Taiwan should contact a Chinese community/group of people who came from Taiwan, whereas a Chinese research assistant from mainland China should contact a Chinese community/group of people who came from mainland China. Not only sub-ethnicity, but also the country of origin was an important factor for the cooperation of Asian American communities/groups in the research.
Potential unauthentic cases
The research team noted several potentially unauthentic cases. By the end of the first 6 months after the recruitment criteria were relaxed, a total of 109 potential participants had been screened, but only 39 met the inclusion/exclusion criteria. Despite the change of lowering the pain level to 1, recruitment did not drastically increase in the following 9 months. Among those who did not pass the screening questions, many unauthentic cases were found (e.g., cases from mainland China or South Africa).
Given that initial internet interactions were non–face-to-face, the research team expected some unauthentic cases from the beginning of the study as had been found before [Reference Im, Ji and Kim40]. However, the team also expected that unauthentic cases would be minimal because the present study required a 3-month involvement with weekly coaching/support and other online intervention components. Yet many of those who successfully went through the screening process disappeared from the study after receiving the first timepoint incentive (at the completion of the pretest questionnaire). A Zoom information session that was held for participants who completed the pretest questionnaire was therefore moved from the moment after providing the first timepoint incentive to before providing the first timepoint incentive. A few unusual cases emerged during the Zoom information sessions. When asked to turn on their cameras, several women who appeared to be of different ethnic backgrounds claimed to be Chinese, Korean, or Japanese. Suspecting possible mixed ethnicity, the research team asked follow-up questions to verify their identities, after which many withdrew from the study.
Frequent technological issues
Cognitive decline has been reported in cancer survivors, owing to chemotherapy, radiotherapy, and/or other treatment modalities [Reference Oliva, Giustiniani, Danesin, Burgio, Arcara and Conte41]. Perhaps for such reasons, many participants in the present study forgot their passwords and user IDs, which the technology assistant team frequently needed to reset. Many participants also reported difficulties in accessing and navigating the project website. To address these challenges, the research team conducted additional phone calls or Zoom meetings to resolve participants’ technological issues.
Despite preliminary Zoom sessions to assist with Fitbit setup, many participants, particularly older adults, struggled to sync their devices and to recall instructions. Given the participants’ average age of 50.36 ± 9.36 years, these challenges were expected. Nevertheless, as difficulties persisted, some participants became frustrated and withdrew due to the perceived burden. Issues in accessing chat sessions and web resources often required interventionists to schedule extra Zoom meetings for technological support.
Limited information is available on participants who withdrew from the study. Although some attrition may be attributed to time burden, several individuals dropped out before the intervention took place – either by missing the initial meeting or becoming unresponsive to follow-up emails. Without follow-ups, their reasons remain speculative. Some may have felt uncomfortable with the initial face-to-face or virtual introduction, perceiving it as intrusive. Alternatively, others may have been overwhelmed by the study requirements, such as coaching sessions and educational materials. This may have been especially true because women surviving breast cancer must often juggle caregiving roles and face cultural expectations to prioritize others’ needs, potentially lowering their threshold for continued participation.
Discussion
Recommendations for future recruitment and retention
In this article, we have presented 6 recruitment and retention issues from a technology-based intervention study with Asian American breast cancer survivors with depressive symptoms. Many issues have been reported in traditional studies among racial/ethnic minorities, yet little has been known about recruitment and retention issues in technology-based interventions using computers and mobile devices [Reference Ko, Gao and Wang9,Reference Im, Kim and Xu11]. We therefore suggest the following recommendations for future recruitment and retention of Asian American breast cancer survivors (see Table 1).
Table 1. Recommendations for recruitment and retention of Asian American breast cancer survivors
First, inclusion and exclusion criteria need to be carefully reviewed during the study planning process. Although our research team based these criteria on the literature related to breast cancer, pain, and depression, they proved impractical for this specific population. Although the research team had extensive experience in working with this group across multiple studies, the team did not combine multiple conditions (breast cancer, pain, and depressive symptoms) in a single study, which affected the feasibility of applying the criteria in real-world settings. Feedback on inclusion/exclusion criteria from potential users can offer a resource for examining the practicality/feasibility of inclusion and exclusion criteria. As an alternative, engaging community gatekeepers or leaders in focus groups may offer valuable insights into refining the criteria.
Second, any announcement of the study should take into account potential stigmas attached to the study’s topics, conditions, and populations. As discussed above, recruitment and retention for the present study was difficult because of multiple stigmas attached to cancer, depressive symptoms, and pain. Although non–face-to-face interactions through technology-based interventions might minimize the hesitation of hidden populations to participate in research studies [Reference Im, Chee and Kim12,Reference Rodríguez-Rivas, Cangas, Cariola, Varela and Valdebenito32], stigmas attached to specific topics, conditions, and populations could still be strong barriers to recruitment and retention. During the study planning process, researchers should consider potential stigmas attached to their research topics, conditions, and populations. Feedback on study topics, conditions, and populations from community gatekeepers and potential participants would help identify related stigmas.
Third, researchers need to carefully review and adopt strategies to motivate participation and strengthen communication with communities. Drawing from the literature [Reference Gill, King and Flego42], the research team implemented various strategies, including financial incentives, thank-you emails, newsletters, and reminders. Outreach efforts were conducted both online and offline, and they included attending community events, visiting local businesses, participating in cancer support groups and health fairs, and engaging with faith-based organizations through volunteering and relationship building. Transparent communication, culturally relevant messaging, and dissemination of findings helped foster trust, especially with faith-based organizations. Ethnic-specific media such as local newspapers, social media platforms (e.g., Instagram, Facebook), and communication apps like KakaoTalk, WeChat, and Line were also used to broaden the reach. Despite these efforts, recruitment and retention remained challenging because of the study population’s unique characteristics and conditions. Researchers should consider the unique characteristics of their study populations, including each population’s conditions and related contextual factors, to select specific motivational and communicative strategies for potential participants. For example, the monetary incentives in our study were based on pre-pandemic benchmarks, but shifts owing to inflation and increased telemedicine have altered participants’ expectations. Researchers need to carefully examine optimal monetary incentives through timely discussions with multiple stakeholders (other researchers, potential participants, and community gatekeepers/leaders), because social and cultural situations can change quickly with national and global events.
Fourth, researchers should establish and maintain trustable relationships with current and new stakeholders in communities/groups of potential participants and consider forming an advisory board of collaborators to obtain formalized feedback on study design and recruitment/retention. Although our research team had long-standing relationships with many Asian American communities and groups, the COVID-19 pandemic significantly disrupted those networks; many leaders relocated or retired, and several groups dissolved owing to declining participation, influenced partly by increases in anti-Asian racism [Reference Reny and Barreto43,Reference Gover, Harper and Langton44]. In response, the team focused on re-engaging former partners and identifying new ones, ultimately involving over 40 community consultants from local organizations, churches, and support groups. These trusted collaborators, many of whom had prior research involvement, played a key role in shaping study design and recruitment. Regular feedback, especially through quarterly Zoom meetings, proved essential for adapting to evolving community dynamics and improving participant engagement. Yet establishing a formal advisory board could further strengthen these efforts and ensure long-term collaboration.
Finally, researchers need to establish robust technological support systems and actively monitor unauthentic cases. Although technology-based interventions offer accessibility, they can pose challenges for certain groups, such as older adults or those with cognitive impairments [Reference Ko, Gao and Wang9]. In the present study, perhaps because of the effects of chemo-brain among cancer survivors, participants frequently required support as patients (e.g., ongoing tech support). In addition, concerns about fraudulent participation are well-documented in digital research [Reference Dal-Ré, Rid, Emanuel and Wendler45]. To address this, the research team’s engineering unit monitored website traffic and user behaviors to identify potential hacking or suspicious activities, and monitoring interpersonal interactions with participants also helped to identify unauthentic cases.
Conclusions
In this article, we have highlighted recruitment and retention issues in a technology-based cancer pain management intervention study among Asian American breast cancer survivors with depressive symptoms – a population rarely examined in this context. Such issues tend to be well-documented in traditional research with racial/ethnic minorities [Reference Taani, Zabler, Fendrich and Schiffman1–Reference George, Duran and Norris3], but they are seldom reported in technology-based research [Reference Ko, Gao and Wang9–Reference Im, Kim and Xu11]. By presenting insights specific to this understudied group, we offer guidance for designing culturally tailored, technology-based studies designed to improve participation and engagement among racial/ethnic minority populations.
Acknowledgements
The authors appreciate the efforts made by the research team, which included a dozen research assistants who have been involved in the study.
Author contributions
Wonshik Chee: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing-original draft, Writing-review and editing; Seulgi Ryu: Data curation, Formal analysis, Investigation, Writing-original draft, Writing-review and editing; Jianing Quan: Data curation, Formal analysis, Investigation, Writing-original draft, Writing-review and editing; Dongmi Kim: Data curation, Formal analysis, Investigation, Writing-original draft, Writing-review and editing; Eun-Ok Im: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Supervision, Validation, Writing-original draft, Writing-review and editing.
Funding statement
Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under Award Number R33CA280979. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Competing interests
The authors declared no conflict of interest.
Open access
