Statement of Research Significance
Research Topic(s):
-
• Neuropsychological testing is the evaluation of cognitive abilities. An individual’s scores on neuropsychological tests have to be benchmarked against those of well-performing, cognitively normal people. Our goal was to develop normative data for a comprehensive neuropsychological battery for Vietnamese adults and assess the influence of demographic characteristics on test performance.
Main Findings:
-
• We found age and education to be consistently and significantly associated with cognitive ability, such that people who are younger or more educated performed better on most or all tests. A sex effect was found in tests of memory and visuospatial function. Normative data are provided as tables and equations so that they can be readily applied in clinical settings.
Study Contributions:
-
• We generated normative values for commonly used neuropsychological tests and examined the influence of age, sex, and education on test performance in nearly 400 Vietnamese cognitively healthy adults, age 40 and above.
Introduction
Neuropsychological examination is an integral part of the comprehensive evaluation of brain functions. Neuropsychological reports can determine areas of strength and weakness in cognitive domains related to a wide range of neuropsychiatric disorders. It is particularly helpful in determining the presence and severity of dementia, cerebrovascular diseases, and post-traumatic brain injury (TBI) or concussions, all of which are among the fastest-growing causes of disability in Vietnam (Carr et al., Reference Carr, Kahn, Mathkour, Biro, Bui and Dumont2018). For instance, Vietnam ranked #21 among countries with the highest number of older adults in 2020 (https://www.prb.org/), leading to a corresponding rapid increase in the prevalence of dementia (Vu et al., Reference Vu, Nguyen, Nguyen, Nguyen, Tran, Nguyen, Hinton and Pham2024). Stroke incidence of Vietnam is among the world’s highest, at about 161 per 100000 (Mai et al., Reference Mai, Dao, Luong, Nguyen, Nguyen and Nguyen2022); while the incidence of TBI is at 291 per 100000, representing a 50.4% increase from 1990 to 2016 (James et al., Reference James, Theadom, Ellenbogen, Bannick, Montjoy-Venning, Lucchesi, Abbasi, Abdulkader, Abraha, Adsuar, Afarideh, Agrawal, Ahmadi, Ahmed, Aichour, Aichour, Aichour, Akinyemi, Akseer and Murray2019).
To interpret the results, scores from various cognitive tests must be benchmarked against normative data – a frame of reference considered to be “normal” performance generated by a cohort of cognitively healthy individuals (Mitrushina et al., Reference Mitrushina, Boone, Razani and D’Elia2005). Such normative values, while can be found for many neuropsychological tests among American and European individuals, remain scant in less developed countries, including Vietnam, primarily due to the challenging nature of conducting normative research (e.g., adequate sample size and funding are required). It is now well established that performance in neuropsychological tests is considerably influenced by age, sex, education, and cultural factors, and that tests of Western origins can potentially misclassify individuals from non-Western populations, ethnic minorities, or with disadvantageous socioeconomic background (Fillenbaum et al., Reference Fillenbaum, Mccurry, Kuchibhatla, Masaki, Borenstein, Foley, White, Heyman and Larson2005; Franzen et al., Reference Franzen, van den Berg, Goudsmit, Jurgens, van de Wiel, Kalkisim, Uysal-Bozkir, Ayhan, Nielsen and Papma2020; Nguyen et al., Reference Nguyen, Rampa, Staios, Nielsen, Zapparoli, Zhou, Mbakile-Mahlanza, Colon, Hammond, Hendriks, Kgolo, Serrano, Marquine, Dutt, Evans and Judd2024). In addition, comprehensive neuropsychological evaluation in Vietnam is sadly lacking, with no formal training or certification in neuropsychology being provided nationwide as of present. Therefore, it is important to generate normative performance from an adequately powered cohort to improve diagnostic accuracy and allow early detection of mild cognitive impairment (MCI) and dementia among Vietnamese individuals.
Previous research to generate normative values that account for sex, age, and educational background in the Vietnamese-speaking population was conducted more than 10 to 20 years ago, largely in participants residing in the United States (Dick et al., Reference Dick, Teng, Kempler, Davis and Taussig2002; Ngo et al., Reference Ngo, Le and Le2010), with small sample size (N = 30 – 60), or using primarily a single measure (e.g., verbal fluency in (Kempler et al., Reference Kempler, Teng, Dick, Taussig and Davis1998; Strauss et al., Reference Strauss, Sherman and Spreen2006) or Stroop task in (Doan & Swerdlow, Reference Doan and Swerdlow1999). At the present time, no normative data of neuropsychological tests in Vietnam have been published. Therefore, our study aims to extend prior efforts by developing a relatively comprehensive neuropsychological battery, assessing the influence of demographic variables of interest on test performance, and establishing updated normative data to inform clinical decision making for Vietnamese adults (McCauley et al., Reference McCauley, Nguyen, Nguyen, Strutt, Stinson, Windham and York2023).
Method
Participants
Participants were recruited from two medical centers, Hospital 30/4 (read “April 30”) and University Medical Center (UMC), in Ho Chi Minh city in southern Vietnam. Hospital 30/4 is a central hospital and is part of the healthcare system operated by the Ministry of Public Security. It primarily serves active and retired police officers and their families, although the general public can also seek and receive care. UMC is a university hospital, affiliated with the University of Medicine and Pharmacy at Ho Chi Minh city. Both are general hospitals with a neurology department and memory and dementia unit.
Participants were (i) individuals seeking pre-employment, employee health checkup, or general physical exams; (ii) spouses of people with dementia or MCI visiting the memory and dementia unit; or (iii) visitors of inpatients. Halfway through the data collection, it was found that adults over the age of 80 were significantly underrepresented in the study, hence recruitment efforts were also conducted at a community center where senior social clubs hold weekly gatherings in Ho Chi Minh city (in the same district as the two aforementioned hospitals). We included participants who met the following criteria: (i) age ≥ 40 years, (ii) self-identified as having normal cognition and daily function when interviewed (i.e., denied any cognitive complaints; able to perform basic and instrumental activities of daily living without assistance), (iii) Mini-Mental State Examination (MMSE) scores ≥ 26 (Folstein et al., Reference Folstein, Folstein and McHugh1975; Kvitting et al., Reference Kvitting, Fällman, Wressle and Marcusson2019), and (iv) could read and write. Participants who acknowledged a history of neuropsychological disorders, psychiatric illness (including schizophrenia and bipolar disorder), substance abuse, head injury with loss of consciousness, or having visual or hearing impairment were excluded. Sample size was calculated based on recommendations of at least 50 participants per age group for normative studies (Mitrushina et al., Reference Mitrushina, Boone, Razani and D’Elia2005), totaling 250 participants for five age groups (40–49, 50–59, 60–69, 70–79, and 80–89).
Data on the demographic characteristics (age, sex, education; with education recorded as less than high school, high school or equivalent, or college and beyond) and history of any chronic illnesses (e.g., cardiovascular disease, diabetes, cancer) of the participants were collected. All participants provided written informed consent. The research was approved by the Institutional Review Board of the University of Medicine and Pharmacy at Ho Chi Minh City (Approval No. 615/HDDD-DHYD on September 24 2020). The study was completed in accordance with the Helsinki Declaration. Data were collected between March 2021 and October 2023.
Test selection for the neuropsychological battery
The neuropsychological tests were selected based on the following three criteria. First, they should encompass a broad range of cognitive domains that are relevant to the diagnosis of cognitive impairment: memory, attention, executive function, processing speed, language, and visuospatial ability. Second, we opted for instruments that are well-established and commonly used (Rabin et al., Reference Rabin, Barr and Burton2005) rather than developing new tests, as this would enable meaningful comparison with other cross-cultural studies. Finally, test administration should be completed within 30 minutes to facilitate implementation in various clinical settings, including low- to medium-resource facilities. As a result, the following tests were chosen: Consortium to Establish a Registry for Alzheimer’s Disease (CERAD) Word List (WL) Tasks, Trail Making Test (TMT), Digit Span, Animal Fluency, and Clock Drawing Test.
Neuropsychological measures
As there are currently no neuropsychologists in Vietnam, neuropsychological tests were administered by nurses who were trained under the guidance of the study’s corresponding author (CTT), with all testing sessions conducted under the supervision of a Ph.D.-level neurologist (author HTTT). Tests were administered using a paper and pencil format, including:
CERAD WL Tasks (Morris et al., Reference Morris, Heyman, Mohs, Hughes, van Belle, Fillenbaum and Clark1989), consisting of the WL Memory, WL Recall, and WL Recognition. The WL Memory is a free immediate recall task of 10 printed words over three trials, with the 10 words presented in a new random order in trials 2 and 3. WL Recall and WL Recognition test for delayed recall and recognition of the 10 words, respectively. Permission to translate and adapt the CERAD WL Tasks into Vietnamese was granted by Dr Gerda Fillenbaum (personal email communication, February 1, 2020). We opted to employ the high frequency concept during translation, i.e., choosing words referring to objects familiar to the Vietnamese population rather than using word-for-word translation from the original CERAD WL. The original WL was constructed by selecting common English nouns, of one to two syllables, with a frequency of ≥ 100/million or 50-99/million words from the Thorndike-Lorge list (Morris et al., Reference Morris, Heyman, Mohs, Hughes, van Belle, Fillenbaum and Clark1989; Thorndike & Lorge, Reference Thorndike and Lorge1944). Following this conceptualization, we developed the Vietnamese version of CERAD WL with words that are Vietnamese nouns commonly used in daily life and reported to have high frequency of usage based on Google search engine (i.e., words that occur ≥ 10,000 times using the Google search engine for Vietnamese words). We conducted a pilot testing on a small sample of cognitively unimpaired individuals who sought health checkup at Hospital 30/4 to confirm that the Vietnamese words were indeed frequently used (e.g., to assess the Vietnamese word “head,” individuals were asked if the statement “The head is part of the human body” was true or false). Finally, a copy of the Vietnamese translation, including the English equivalent for each word, was sent to Dr Fillenbaum for review. Please refer to Supplementary Methods and Supplementary Table S1 for the Vietnamese version of WL Tasks. We have previously demonstrated that the Memory, Recall, and Recognition tasks all have high test-retest reliabilities with rs of 0.81, 0,86, 0.77, respectively (all p-values < .05) (Nguyen & Tran, Reference Nguyen and Tran2022). Following the administration of WL Memory, a gap of 10 to 15 minutes took place before the WL Recall and WL Recognition were given, during which we administered the TMT, Parts A and B.
TMT, Parts A (TMT-A) and B (TMT-B) (Reitan & Wolfson, Reference Reitan and Wolfson1993). The TMT measures attention, psychomotor speed, and mental flexibility. TMT-A requires participants to draw lines that consecutively connect 25 encircled numbers on a sheet of paper. TMT-B has a similar requirement except that one has to alternate between numbers and letters. Participants are allowed 180 s to complete Part A and 300 s to complete Part B. Variables of interest are time to completion on each part in seconds. If participants have not completed the Trail by the time limit, timing is stopped and scores are not recorded (though participants are allowed to progress if they are likely to finish soon as judged by the examiner (Heaton et al., Reference Heaton, Miller, Taylor and Grant2004). Our rationale for not assigning the maximum TMT-A/TMT-B score of 180/300 for participants who do not complete the task, as is conventionally done (Strauss et al., Reference Strauss, Sherman and Spreen2006), is that this practice potentially diminishes performance variability among people with varying levels of executive dysfunction (Correia et al., Reference Correia, Ahern, Rabinowitz, Farrer, Smith Watts, Salloway, Malloy and Deoni2015) and leads to violation of the linear regression assumptions to calculate norms in our cohort (see Results). While a few studies have included all TMT-B completion times rather than a maximum of 300 s (MacPherson et al., Reference MacPherson, Cox, Dickie, Karama, Starr, Evans, Bastin, Wardlaw and Deary2017), we opted to discontinue the task to minimize participants’ frustration, reduce testing time, and to conform with most standard administration instructions (Bowie & Harvey, Reference Bowie and Harvey2006; Strauss et al., Reference Strauss, Sherman and Spreen2006). Additionally, in TMT-B, we kept the letters as English alphabet (A B C D E …) instead of changing them into Vietnamese alphabet (A Ă Â B C D Đ E Ê …) as we anticipated that the participants would be familiar with the Latin alphabet.
Digit Span (Wechsler, Reference Wechsler2008), consisting of Digit Span Forward and Digit Span Backward, which are measures of attention and working memory, respectively. In the Digit Span Forward, participants are read sequences of numbers of increasing lengths and asked to repeat them in the same order. In the Digit Span Backward, participants are also read sequences of numbers of increasing lengths but are asked to repeat each sequence in reverse order. Variables of interest are the longest correct number of sequential digits.
Animal Fluency (Kertesz, Reference Kertesz1982), a test of free recall that evaluates verbal category fluency and semantic memory among other abilities, including executive functioning, attention, and speed of processing. Participants are asked to name as many animals as they can in 60 s. Variable of interest is the total word count.
Clock Drawing Test (Shulman et al., Reference Shulman, Shedletsky and Silver1986), which is commonly used to assess executive function, visuospatial organization, and abstract thinking. Participants are shown a pre-drawn circular contour, then asked to draw numbers within the circle to make it look like a clock. As there is no Vietnamese word-for-word equivalent of the English “10 after 11” or “10 past 11,” participants were instructed to set the time “at 11:10.” Possible scores range from 1 to 6 based on the modified Shulman’s scoring system, given as followed: 1 = perfect, 2 = minor visuospatial errors (e.g., mildly impaired spacing of times); 3 = inaccurate representation of 10 after 11 when visuospatial organization is perfect or shows only minor deviations (e.g., minute hand points to 10); 4 = moderate visuospatial disorganization of times such that accurate denotation of 10 after 11 is impossible; 5 = severe level of disorganization as with 4 points; and 6 = no reasonable representation of a clock.
Higher scores indicate better performance on all tests, except for TMT and Clock Drawing Test, for which a higher score indicates longer time to completion or more clock-drawing errors, respectively, and therefore worse performance. The manuscript reporting the validity of these tests for the Vietnamese population is currently in preparation.
Statistical analysis
Descriptive statistics are provided with continuous variables presented as mean and standard deviation, while categorical variables are presented as frequencies and proportions. Normative data were generated via two approaches, the traditional discrete norms and multiple regression equations. Using the first approach, normative data for each neuropsychological test are provided as statistics of raw test scores stratified by age, education, and sex, if the latter showed significant effect on test performance. For each demographic group, we report the mean, standard deviation, and test scores at the 3rd (–2 SD), 7.5th (–1.5 SD), 15th (–1 SD), 25th, 50th, 75th, and 95th percentiles. The –1.5 and –1 SD are presented because these have been used as cutoff values in neuropsychological criteria to diagnose mild neurocognitive disorder; the –2 SD is the cutoff value to determine the presence of major neurocognitive disorder (American Psychiatric Association, 2013).
In the second approach, hierarchical linear regression analyses were conducted with cognitive test scores as dependent variables and age, education, and sex as independent variables. Reference groups are education < high school and male. Education was entered first into the regression equations, followed by age and sex, as education has been documented to have the greatest influence on test performance (Dick et al., Reference Dick, Teng, Kempler, Davis and Taussig2002). If any of the three independent variables was found to have non-significant coefficient, it was dropped from the model, and the model was rerun with the remaining variables. We also calculated the proportion of variance (eta2) accounted for by education, age, and sex in each cognitive test. All analyses were performed using R version 4.2.0.
Results
Participant characteristics
Participant demographic, clinical, and cognitive characteristics are shown in Table 1. Table 2 shows the number of participants stratified by age group, level of education, and sex . Overall, a total of 385 participants were included in this study, with mean age of 61.4 (SD = 10.9, range 40–89). More than half of them were female (57.1%). Participants were highly educated, with nearly half having completed college or a higher degree. Hypertension was the most common comorbidity of the cohort (39.7%), followed by dyslipidemia (28.5%) and diabetes mellitus (14.5%). Participants needed on average 28 minutes (–89). More than half of them were female (57.1%). Participants were highly educated, with nearly half having completed college or a higher degree. Hypertension was the most common comorbidity of the cohort (39.7%), followed by dyslipidemia (28.5%) and diabetes mellitus (14.5%). Participants needed on average 28 minutes (SD = 6.7) to complete the neuropsychological battery. Data for all neuropsychological tests were mostly available (rate of missing data varied between 0.5% and 2.5%), except for TMT-B, where only 53% of the cohort were able to complete the test within the time limit. The remaining participants either did not complete TMT-B within 300 s (136/385 individuals, 35.3%) or could not complete it (43/385, 11.1%) due to several reasons, including not understanding test instructions or inability to recall the sequence of the English alphabet. Compared to those who completed TMT-B within the time limit, non-completers were significantly older, less educated, and demonstrated poorer performance in measures of memory, attention, executive, and visuospatial function, but not language (all p-values < .001, Supplementary Table S2). Assigning the maximum score of 300 to participants who could not complete within 5 minutes (Strauss et al., Reference Strauss, Sherman and Spreen2006; Weintraub et al., Reference Weintraub, Besser, Dodge, Teylan, Ferris, Goldstein, Giordani, Kramer, Loewenstein, Marson, Mungas, Salmon, Welsh-Bohmer, Zhou, Shirk, Atri, Kukull, Phelps and Morris2018) leads to heavily skewed distribution of TMT-B scores and violation of homoscedasticity assumption of the linear regression model to calculate continuous norms, hence the remaining analyses for TMT-B scores were conducted using data of participants who completed this test within the allotted time.
Table 1. Participant characteristics
Note: CERAD WL = Consortium to Establish a Registry for Alzheimer’s Disease Word List Tasks, MMSE = Mini-Mental State Examination. Data on continuous variables are presented as M (SD); data on categorical variables are presented as n (%). Min and max scores of neuropsychological data are presented in parentheses next to the neuropsychological entries.
a Missing in 10/385 participants (2.5%).
b Missing in 181/385 participants (47.0%).
c Missing in 2/385 participants (0.5%).
d Missing in 3/385 participants (0.7%).
e Missing in 5/385 participants (1.2%).
Table 2. Number of participants stratified by sex, age, and education
Discrete normative data
Given the small sample size of cells where participants were further broken down by sex, we subsequently provided discrete norms tables stratified by age group and education only. Summary statistics (mean, SD) of all neuropsychological scores are provided in Table 3, while discrete norms by age and education, including performance at the 3rd, 7.5th (–1.5 SD), 15th (–1 SD), 25th, 50th, 75th, and 95th percentiles, are provided in Supplementary Tables S3–S11. As an example, for a 71-year-old male with ≥ college education, a CERAD WL Memory score of 14 would correspond to performance at the 7.5th percentile (Supplementary Table S3).
Table 3. Discrete norms reported as mean and SD by age group and level of education
Note: CERAD WL = Consortium to Establish a Registry for Alzheimer’s Disease Word List. Data were presented as mean (standard deviation).
a The standard deviation of Trail Making Test, Part B for individuals aged 80–89 with less than high school education was not available as there was only one participant in this demographic.
Regression-based norms
Regression-based demographically corrected norms can be calculated from Table 4 using the regression coefficients and Root Mean Squared Error (RMSE) . Using the previous example, to obtain the expected performance (i.e., 50th or at z-score = 0) on CERAD WL Memory for a 71-year-old male with ≥ college education:
$$\begin{align} &+ [(-0.136) \times 71] + (2.430 \times 0) + (1.406 \times 0) + 2.302 \\&\,\,\,\,\,\,= 19.321 \ {\rm words}\end{align}$$
Table 4. Linear regression analyses for demographic variables and neuropsychological test performance
Note: CERAD WL = Consortium to Establish a Registry for Alzheimer’s Disease Word List, n.s. = non-significant, RMSE = Root Mean Squared Error, TMT = Trail Making Test. Education level is categorized into (1) < high school, (2) high school or equivalent, and (3) college and beyond. Reference groups are male and education < high school.
a Tests for which higher scores indicate worse performance.
b TMT-B was completed by 53% of participants.
*p < .05, **p < .01, ***p < .001.
In fact, his actual score was 14, demonstrating below average performance for his demographic group. The RMSE can be incorporated into the equation to obtain the –1.5 SD normative score on CERAD WL Memory for this individual:
$$\begin{align}+ [(-1.5) \times 3.553] = 13.991\; {\rm words}\end{align}$$
The result shows that the mentioned individual’s CERAD WL Memory score approaches abnormal performance, determined as 1.5 SD below age-, sex- and education-matched individuals. Alternatively, using this individual’s actual score, we can calculate the degree of deviation from the expected score as follows:
$$\begin{align}\\&{\rm (Actual \; score - Expected \; score) / RMSE} = (14 - 19.321) / 3.553 \\&\,\,\,\,\,\,= -1.497\end{align}$$
This z-score, –1.497, can be converted into its corresponding percentile, i.e., 7th. An Excel file to facilitate calculations of norms is provided in the Supplementary Materials and available from the Vietnam Alzheimer’s Disease & Neurocognitive Disorders Association (https://alzvietnam.org/).
Effects of demographic characteristics
Table 4 presents the effects of age, sex, and education on performance of the neuropsychological battery. Linear regression analyses showed that, on average, 14.6% the variance (adjusted R 2) in test performance was explained by demographic variables, ranging from 7% to 29%. Specifically, education explained 0.8% – 8.6%, age explained 3.8% – 14.2%, and sex explained 0.1% – 9.7% of the total variance. Age was consistently and significantly associated with cognitive ability, with younger age relating to better performance on all tests. We found significant effects of sex for measures of memory (CERAD WL Memory: b = 2.43, p < .001; CERAD WL Recall: b = 0.84, p < .001; CERAD WL Recognition: b = 0.40, p = .003) and visuospatial function (Clock Drawing Test: b = 0.16, p = .043), but not attention (TMT-A: b = 1.57, p = .59; Digit Span Forward: b = –0.02, p = .90; Digit Span Backward: b = 0.14, p = .52), executive function (TMT-B: b = 15.36, p = .082), nor language (b = 0.16, p = .043). Specifically, a female advantage was found for all three CERAD WL tasks. In contrast, a male advantage was reported on the Clock Drawing Test. Finally, with the exception of CERAD WL Recognition, education was significantly related to all tests, with a particularly robust effect on CERAD WL Memory, Digit Span Backward, and Animal Naming Test, as indicated by the regression coefficients of different levels of educational attainment.
Discussion
To our knowledge, this is the first study to generate normative data for a neuropsychological battery in cognitively healthy Vietnamese adults, and to examine the effects of demographic variables on test performance. We found that performance on neuropsychological tests was differentially influenced by age, education, and sex, with age showing the strongest effect. All tests were vulnerable to age, wherein greater age was associated with worse performance, consistent with prior literature documenting that except for general knowledge, the majority of fluid abilities are expected to decrease in later life (Salthouse, Reference Salthouse2019). While education exerted significant influence on most test scores, its effect was more robust in some tests than others. Our study coded educational attainment as a categorical variable, and only in the CERAD WL Memory, Digit Span Backward, and Animal Naming Test were consistent differences found across all three levels of education. In CERAD WL Recall, both parts of TMT, Digit Span Forward, and Clock Drawing Test, association of education was less robust, wherein only people with a college degree or beyond outperformed those who did not graduate from high school, but no differences were found between people with or without high school education. The only measure where education did not contribute significantly to performance was CERAD WL Recognition, a finding that is in agreement with previous normative studies on the CERAD neuropsychological battery. One explanation could be overlearning, as participants had to read aloud and memorize the words three times during WL Memory, and were further provided with visual cues during WL Recognition (Liu et al., Reference Liu, Kuo, Tang, Chau, Ho, Kwok, Chan, Choi, Lam and Chu2011). This also demonstrated a marked ceiling effect for WL Recognition, such that participants performed relatively well regardless of educational attainment (Kirsebom et al., Reference Kirsebom, Espenes, Hessen, Waterloo, Johnsen, Gundersen, Botne Sando, Rolfseng Grøntvedt, Timón and Fladby2019). In terms of sex, we found a female advantage on verbal learning and recall (all three CERAD WL tasks) and a male advantage in visuospatial perception (Clock Drawing Test)–results that have been well documented in the literature (Karstens et al., Reference Karstens, Christianson, Lundt, Machulda, Mielke, Fields, Kremers, Graff-Radford, Vemuri, Jack, Knopman, Petersen and Stricker2023).
Several caveats concerning the usage of our normative data in clinical practice should be taken into account. In particular, TMT-B was only completed within the time limit of 300 s by approximately one half of the cohort, presumably by those with good knowledge of the English or French alphabet. Among the remaining half, three-fourths did not complete TMT-B within 300 s and one-fourth failed to finish TMT-B due to several reasons, including not understanding test instructions or not recalling the sequence of the English alphabet. Underperformance on TMT-B was largely explained by lower educational level and more advanced age, comparable to previous reports (Hashimoto et al., Reference Hashimoto, Meguro, Lee, Kasai, Ishii and Yamaguchi2006; Seo et al., Reference Seo, Lee, Kim, Lee, Jhoo, Youn, Choo, Ha and Woo2006). Hence, the results of TMT-B from this demographic in our study should be interpreted with caution. In the future, we may consider attempts to improve the clinically relevant performance of TMT-B. For example, increasing the permitted time to complete the test (e.g., up to 480 s (Hashimoto et al., Reference Hashimoto, Meguro, Lee, Kasai, Ishii and Yamaguchi2006) or adding quantitative scores (number of correct lines and errors of commission (Weintraub et al., Reference Weintraub, Salmon, Mercaldo, Ferris, Graff-Radford, Chui, Cummings, DeCarli, Foster, Galasko, Peskind, Dietrich, Beekly, Kukull and Morris2009) can possibly unmask performance variability among individuals and enhance utility in parametric statistical reports (Correia et al., Reference Correia, Ahern, Rabinowitz, Farrer, Smith Watts, Salloway, Malloy and Deoni2015). Notably, even when participants finish the test within the allotted time, the average TMT-B scores of our cohort are considerably lower than those previously reported (Seo et al., Reference Seo, Lee, Kim, Lee, Jhoo, Youn, Choo, Ha and Woo2006; Wang et al., Reference Wang, Sun, Ma, Wang, Yao, Deng, Liu, Collier and Li2011; Weintraub et al., Reference Weintraub, Besser, Dodge, Teylan, Ferris, Goldstein, Giordani, Kramer, Loewenstein, Marson, Mungas, Salmon, Welsh-Bohmer, Zhou, Shirk, Atri, Kukull, Phelps and Morris2018), presumably because of participants’ unfamiliarity with timed assessments and the English alphabet. Hence, TMT-B items could be modified, such that English letters are substituted with Vietnamese letters. That being said, while this adaptation has been realized in multiple languages (e.g., Arabic (Stanczak et al., Reference Stanczak, Stanczak and Awadalla2001), Chinese (Lu & Bigler, Reference Lu and Bigler2000), Japanese (Hashimoto et al., Reference Hashimoto, Meguro, Lee, Kasai, Ishii and Yamaguchi2006), and Korean (Seo et al., Reference Seo, Lee, Kim, Lee, Jhoo, Youn, Choo, Ha and Woo2006), it may still have considerable limitations in evaluating older adults, especially those with low educational attainment, as shown in (Hashimoto et al., Reference Hashimoto, Meguro, Lee, Kasai, Ishii and Yamaguchi2006). Finally, approaches such as removing TMT-B while retaining TMT-A in the battery (Dick et al., Reference Dick, Teng, Kempler, Davis and Taussig2002) or replacing TMT with its relatively language-free and culturally fair equivalence (e.g, the Color Trails Tests (Dugbartey et al., Reference Dugbartey, Townes and Mahurin2000; McCauley et al., Reference McCauley, Nguyen, Nguyen, Strutt, Stinson, Windham and York2023) or TMT-Black and White (Kim et al., Reference Kim, Baek and Kim2014) are also feasible.
In addition, a number of distinctions between the application of regression-based norms versus discrete norms are worth mentioning. First, as we did not correct for sex in the discrete norms due to inadequate sample size, normative values of neuropsychological tests known to be affected by this demographic variable may be more accurate using the regression approach. This is particularly evident in CERAD WL Memory; such that given the same age, level of education, and recall performance, a female individual would be expected to correctly remember two items more than a male individual using the regression-based norms, while no sex effect is apparent from the discrete norms table. Second, discrete norms assigned several possible percentiles to a single raw score for some cognitive tests, such as CERAD WL Recognition (Supplementary Table S5) and Clock Drawing Test (Supplementary Table S11), leaving considerable scope for individual interpretation. This finding also demonstrates a more pronounced ceiling effect in the CERAD WL Recognition and Clock Drawing Test than in other measures, i.e., cognitively healthy individuals achieved relatively high scores in these two tests regardless of age, sex, or education. While low ceiling in CERAD WL Recognition was attributable to overlearning as mentioned, for Clock Drawing Test it could be due to our usage of the 5-point Shulman scoring system. Implementing more complex (and time-consuming) scoring methods, e.g., a 10- or 20-point scale, could potentially reduce the skewness of score distribution (Strauss et al., Reference Strauss, Sherman and Spreen2006). Additionally, it has been shown that the ceiling effect is diminished and education emerges as a protective factor in the performance of Clock Drawing Test among individuals that developed dementia (Lam et al., Reference Lam, Chiu, Ng, Chan, Chan, Li and Wong1998). One final distinction between the two norming methods is in the well-documented treatment of age (Kiselica et al., Reference Kiselica, Karr, Mikula, Ranum, Benge, Medina and Woods2024): the regression-based approach treats age as a continuous variable, whereas discrete norming necessitates the creation of artificial age bands. For instance, using discrete norms, a TMT-A score of 100 would correspond to performance between percentiles 7.5th and 15th (>1 SD below norms) for a 59-year-old individual, and between percentiles 15th and 25th for a 60-year-old individual (both with < high school education), indicating an impaired score for the former but not the latter, despite an age gap of only one year. In this case, the artificial age group effect can be avoided by using the regression-based norms calculation.
The strengths of our study include the relatively comprehensive battery of neuropsychological tests that covers most cognitive domains, the translation and adaptation of a verbal memory measure, and generation of both discrete and regression-based norms. However, there are several limitations to consider. The first issue concerns the lack of representation of people with lower educational attainment, largely explained by the data collection taking place in a heavily urbanized city in Vietnam. About 42.0% of the study cohort were college graduates, compared to a percentage of 11.1% of adults having a bachelor’s degree based on a 2020 National Census Bureau Survey, with the majority of these individuals living in either the capital Hanoi (44.8%) or Ho Chi Minh city (38.7%) (Vietnam Department of Population and Labor Statistics, 2019). Similarly, individuals aged 80 – 89 years only accounted for 5.9% of our cohort (n = 23). Given that advanced age is associated with a faster rate of cognitive decline and substantial variability in neuropsychological ability, normative values of the 80+ age group may be less accurate compared to those of younger adults and should be interpreted with caution (Arampatzi et al., Reference Arampatzi, Margioti, Messinis, Yannakoulia, Hadjigeorgiou, Dardiotis, Sakka, Scarmeas and Kosmidis2024). Second, neuropsychological measures of participants were collected at a single visit, thus individuals at preclinical stages of neurodegenerative disorders (e.g., dementia) might have been included in the study. As a result, normative standards may be less robust and could overlook the earliest signs of cognitive impairment (Arampatzi et al., Reference Arampatzi, Margioti, Messinis, Yannakoulia, Hadjigeorgiou, Dardiotis, Sakka, Scarmeas and Kosmidis2024; Kapoor et al., Reference Kapoor, Ho, Jang and Nation2024). Third, participants were not screened for depressive symptoms. Estimates of the prevalence of depression and anxiety disorders among Vietnamese adults vary across studies, e.g., from 2.6% (Vuong et al., Reference Vuong, Van Ginneken, Morris, Ha and Busse2011) to 14.2% (Nguyen et al., Reference Nguyen, Tran, Tran, Tran and Fisher2019), warranting assessment of these comorbidities in future norming efforts. Finally, the proportion of variance explained in the regression models averaged only 14.6%, ranging from 7% to 29% for different cognitive tests. This suggests that other than age, sex, and level of education, a variety of demographic (e.g., quality of education and socioeconomic status) and non-demographic factors (e.g., testing environment, participants’ attitude and motivation) could play significant roles in predicting test performance (Kiselica et al., Reference Kiselica, Karr, Mikula, Ranum, Benge, Medina and Woods2024).
In conclusion, we report a relatively comprehensive and sufficiently powered set of normative data for commonly used neuropsychological tests in Vietnamese adults. Administration of the battery can be fully done within 30 minutes, well-suited for clinical appointments. Demographic variables including age and education were associated with performance on all or most measures, while some sex effects consistent with the literature were found. The present study is part of a larger effort to strengthen research capacity and inform policy making in Alzheimer’s disease and related dementia in Vietnam (Vu et al., Reference Vu, Nguyen, Nguyen, Nguyen, Tran, Nguyen, Hinton and Pham2024). Along with many Asian counterparts, Vietnam is experiencing dramatic demographic changes, i.e., a rapidly aging population, leading to a substantial increase in dementia prevalence (Asian Development Bank, 2022; Kosowicz et al., Reference Kosowicz, Tran, Brodaty, Roughead, Esterman, Hinton, Kim, Kurrle, Dang, Crotty, Gilbert, Tan and Nguyen2023). We specifically hope to improve the interpretation of neuropsychological reports and research, and that testing services will become more widespread throughout the country. To improve the sensitivity and utilization of the battery, further refinements will be necessary, including (i) increasing representation of people with more advanced age and/or lower educational attainment, (ii) incorporating longitudinal assessment to allow for more robust norming, and (iii) applying neuropsychological measures that could be more culturally and linguistically suitable for Vietnamese people.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1355617725101100.
Acknowledgements
We are grateful to Drs. Huyen Thi Thanh Vu, Tuan Anh Nguyen, Huong Nguyen, Ladson Hinton, and David K. Johnson for their mentoring and support. We acknowledge Dr Gerda Fillenbaum for her permission to translate the CERAD Word List Tasks into Vietnamese. We wish to thank Drs. Mai Trang Tong, Lam Nguyet Doan Nguyen, My Linh Thi Nguyen, Nga Thi Dang, Tam Thi Hoa, and Ha Thi Tran for their assistance with participant recruitment and data collection. We also wish to acknowledge Dr. Duong Thi Thu Hang and Dr. Phan Thi Luu for their support in project administration.
Funding statement
Research reported in this publication was supported by the National Institute on Aging (NIA) of the National Institutes of Health (NIH) under award number R01AG064688 (Hinton/Nguyen MPI) (awarded to Dr Cong-Thang Tran). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIA and the NIH. Additional funding (awarded to Dr Ingo Kilimann): Hospital Partnership Funding from the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ, i.e., German Corporation for International Cooperation) between Hospital 30/4 and the DZNE to modify the existing neuropsychological testing at Hospital 30/4 to the version used for normative data.
Competing interests
The authors report there are no competing interests to declare.
Open access
