Statement of Research Significance
Research Question(s) or Topic(s): Is the Hong Kong version of the Penn Computerized Neurocognitive Battery a valid and feasible tool for assessing multi-domain neurocognitive function in Hong Kong Chinese children with autism spectrum disorder?. Main Findings: The Hong Kong version of the Penn Computerized Neurocognitive Battery demonstrates high feasibility in assessing cognitive abilities across multiple domains in Hong Kong children with autism spectrum disorder. It enables comparisons of cognitive profiles between children with and without autism spectrum disorder. Study Contributions: Validation of the Hong Kong version of the Penn Computerized Neurocognitive Battery provides a culturally tailored, standardized tool for assessing neurocognitive function in Chinese children with autism spectrum disorder. This enables future research to explore cognitive mechanisms underlying autism spectrum disorder, inform targeted interventions, and facilitate cross-cultural comparisons of neurocognitive profiles, advancing both clinical practice and research in neurodevelopmental disorders.
Introduction
Autism spectrum disorder (ASD) is a heterogeneous neurodevelopmental condition (Lombardo et al., Reference Lombardo, Lai and Baron-Cohen2019) and little is known about the drivers of its diverse presentation. As genes associated with ASD are involved in neurocognitive development (Grove et al., Reference Grove, Ripke, Als, Mattheisen, Walters, Won, Pallesen, Agerbo, Andreassen, Anney, Awashti, Belliveau, Bettella, Buxbaum, Bybjerg-Grauholm, Bækvad-Hansen, Cerrato, Chambert, Christensen and Børglum2019), cognitive functioning may explain the heterogeneity of ASD. Within cognition, there is a range of domains, such as memory, language, and executive function. Notably, uneven cognitive development is well-described in ASD and executive function has prognostic implications (Joseph et al., Reference Joseph, Tager-Flusberg and Lord2002; Kenny et al., Reference Kenny, Cribb and Pellicano2019). Hence, assessing the full spectrum of cognition is essential to examine its role in phenotypic variability (Bassett & Sporns, Reference Bassett and Sporns2017; Menon & D’Esposito, Reference Menon and D’Esposito2021). While the NIH Toolbox Cognition Battery (NIH-TCB) is a feasible option (Jones et al., Reference Jones, Dallman, Harrop, Whitten, Pritchett, Lecavalier, Bodfish and Boyd2022), its psychometric properties were not examined in the ASD population. Furthermore, social cognition tests, core to ASD, are rarely included in standardized batteries.
The Penn Computerized Neurocognitive Battery (CNB) is a multi-domain cognitive assessment tool developed at the Brain Behavior Laboratory at the University of Pennsylvania. The tasks of the CNB measure accuracy, speed, and efficiency (a composite of accuracy and speed), with their construct validity established against traditional paper-and-pencil tasks (Gur et al., Reference Gur, Ragland, Moberg, Turner, Bilker, Kohler, Siegel and Gur2001). Its factor structure has been analyzed using correlated-factor models and bi-factor models, yielding both domain-specific factors and general factors (g) for accuracy, speed, and efficiency (Gur et al., Reference Gur, Richard, Hughett, Calkins, Macy, Bilker, Brensinger and Gur2010; Moore et al., Reference Moore, Reise, Gur, Hakonarson and Gur2015). For accuracy, three domain-specific factors were identified: episodic memory, social cognition, and executive/complex cognition. For speed, three factors were identified: episodic memory, deliberative (a trade-off between speed and accuracy, often termed “slow thinking”(Kahneman, Reference Kahneman2011)), and vigilance (requiring rapid response, often termed “fast thinking”). For efficiency, four factors were identified: episodic memory, social cognition, complex cognition, and executive function. These approaches provide varied CNB performance summaries. Its automated, low-literacy design suits large-scale studies, as exemplified by the Philadelphia Neurodevelopmental Cohort (PNC) (n = 9,498; aged 8–21 years) (Calkins et al., Reference Calkins, Merikangas, Moore, Burstein, Behr, Satterthwaite, Ruparel, Wolf, Roalf, Mentch, Qiu, Chiavacci, Connolly, Sleiman, Gur, Hakonarson and Gur2015). Following the standard translation procedures, the CNB has been translated into over 25 languages, where the factor structure has been replicated in various psychiatric disorders, across the lifespan and cultures (Bhatia et al., Reference Bhatia, Mazumdar, Wood, He, Gur, Gur, Nimgaonkar and Deshpande2017; Izgi et al., Reference Izgi, Moore, Yalcinay-Inan, Port, Kuscu, Gur and Eser2022; da Motta et al., Reference da Motta, Carvalho, Castilho and Pato2021; Pelt et al., Reference Van Pelt, Moore, Scott, Phoi, Mbakile-Mahlanza, Morales, Gur, Rampa, Matshaba and Lowenthal2022; Scott et al., Reference Scott, Moore, Stein, Pretorius, Zingela, Nagdee, Ngqengelele, Campbell, Sibeko, King, McClellan, Port, Jackson, Ruparel, Susser and Gur2021; Service et al., Reference Service, Vargas Upegui, Castaño Ramírez, Port, Moore, Munoz Umanes, Agudelo Arango, Díaz-Zuluaga, Melo Espejo, López, Palacio, Ruiz Sánchez, Valencia, Teshiba, Espinoza, Olde Loohuis, De la Hoz Gomez, Brodey, Sabatti and Freimer2020). However, the CNB has not been translated into either Mandarin or Cantonese and validated in the Chinese population. In the PNC, the CNB was successfully administered to 218 children and adolescents with ASD (aged 8–21 years), revealing that they performed worse across all neurocognitive domains compared to the group without ASD (Wong et al., Reference Wong, Barzilay, Lam, Chan, Calkins, Gur and Gur2023). In the present study, we examined the factor structure of the Hong Kong version of the CNB (CNB-HK), a Cantonese-translated version of the CNB, in children with and without ASD. We hypothesized that 1. the factor structures of the CNB-HK would be consistent with that of the original CNB, 2. CNB-HK performance correlates with estimated IQ in participants with ASD, 3. the measurement model of the CNB-HK is invariant to the ASD diagnosis, and 4. the group with ASD performs worse in the CNB compared to peers without ASD.
Method
Study population
Participants included 1,048 Chinese children with (n = 636) and without (n = 412) ASD aged 6–12 years. Children with ASD were diagnosed by child psychiatrists based on the DSM-5 criteria and recruited from a university-affiliated psychiatric center in Hong Kong (Su et al., Reference Su, Wong, Lu, Wan, Zhang, Xu, Li, Liu, Cheung, Ching, Cheong, Leung, Chan, Leung, Chan and Ng2024; Wan et al., Reference Wan, Wong, Tun, Su, Xu, Tang, Ma, Chan, Chan and Ng2024). Children without ASD were recruited from a territory-wide child and adolescent psychiatric epidemiological study (Chan et al., Reference Chan, Wong, Hussain, Tsoi, Ma, Chau, Ma, Lai, Chu, Lo, Ho, Leung, Yiu, So, Sham, Hung and Leung2025), in which participants were assessed for the presence of 31 DSM-5-defined psychiatric disorders with the Diagnostic Interview Schedule for Children Version 5 (DISC-5) (Shaffer et al., Reference Shaffer, Fisher, Lucas, Dulcan and Schwab-Stone2000) and screened for ASD using the Hong Kong version of the 10-item Autism Spectrum Quotient (AQ-10-HK) (Leung et al., Reference Leung, Leung, Chan and Leung2023; Wong et al., Reference Wong, Chan, Chau, Chu, Ho, Ho, Hung, Hussain, Lai, Lam, Lo, Ma, Ma, Mo, Sham, Shea, So, Tsoi and Leung2025). Participants who were studying in mainstream school, without history of intellectual disability, and with negative results in the DISC-5 and AQ-10-HK were recruited. Exclusion criteria for both groups included the presence of intellectual disability, neurological disorders, severe mental illness (e.g., psychosis), and major medical illnesses. Common co-occurring psychiatric conditions such as ADHD and anxiety were not considered as exclusion criteria for the ASD group. Parents or legal guardians provided written informed consent. The study protocol followed the Declaration of Helsinki and was approved by the Joint CUHK-NTEC CREC (Ref.: 2021.550).
Demographic data and IQ assessment
Sociodemographic data were obtained from caregivers. Full-scale Intelligence Quotient (FSIQ) for the children with ASD was estimated using the short form of the Hong Kong Wechsler Intelligence Scale for Children, 4th Edition [WISC-IV (HK) Short Form] (Wechsler, Reference Wechsler2010), which included the subtests of similarities, digit span, symbol coding, and matrix reasoning.
Adaptation and pilot testing of the CNB-HK
After examination by the authors, 12 out of the original 13 tasks were deemed suitable for translation into Cantonese. The Penn Verbal Reasoning Test, a complex cognition task with verbal analogy problems, was not translated due to cultural and language specificity in the original English analogies. The Penn Conditional Exclusion Test (PCET) was originally developed as an executive function task. However, analyses of its properties revealed that it was better categorized in the domain of complex cognition (Moore et al., Reference Moore, Reise, Gur, Hakonarson and Gur2015). Therefore, we considered the PCET a task of complex cognition in the present study. To ensure there would be three tasks as indicators for each cognitive domain for the factor analyses, a Go/No-Go Task (GNG) was incorporated as an executive function task. The details of the 13 tasks are summarized in Table 1 and described elsewhere (Gur et al., Reference Gur, Richard, Hughett, Calkins, Macy, Bilker, Brensinger and Gur2010). We followed guidelines for the translation process (Van Widenfelt et al., Reference Van Widenfelt, Treffers, De Beurs, Siebelink and Koudijs2005): The content of the 13 tasks was first translated into Cantonese by the first author, a bilingual research psychiatrist with over 10 years of experience in conducting neurocognitive examinations. Subsequently, an independent bilingual postdoctoral psychology researcher performed the back-translation. The back-translation was then compared to the original English version. The process was iterated as necessary, and any discrepancies were resolved by consensus.
Table 1. Summary of the tasks and their respective cognitive domain in the CNB-HK
Note: CNB-HK: Hong Kong version of the Penn Computerized Neurocognitive Battery.
For the Word Memory Task (CPW), the original version comprised 36 words for memorization. These words were chosen for easy comprehension to ensure that performance was not confounded by literacy levels. The translation of the 36 words adhered to two criteria: 1. Each translated word was expressed with two Chinese characters to prevent participants from relying on the number of characters for memorization, and 2. the translated Chinese word was checked against a Chinese word database by the Hong Kong Education Bureau (www.edbchinese.hk/lexlist_ch/), to ensure that the translated Chinese words were culturally common and understood by six-year-olds. Among the 36 words, all but one could be expressed using two Chinese characters. Furthermore, eight translated words were not commonly learned by six-year-olds, or not culturally common in Hong Kong. Hence, these nine words were replaced in consultation with experts from the Department of Educational Psychology of the University. The substituted words belonged to the same semantic categories as the original English words, while fulfilling the aforementioned criteria. Pilot testing was conducted with volunteers (children and adults) and the translation was refined based on feedback.
Administration, scoring of the CNB-HK and establishment of local normative data
The CNB-HK was administered by research staff and undergraduate medical students in a quiet office. Proctors received training, including sessions of supervised administration led by the first author. A standardized manual of the CNB was followed: written instructions for each task were read aloud, and understanding of the task was ensured before each task began. The 13 tasks were administered in a prefixed order, and the proctors assigned a validity score for each task on data completeness and validity. To standardize validity scoring, a written guideline outlined task-specific criteria, as invalid data vary by task. For example, an attention lapse during the registration phase of episodic memory tasks critically impacts recognition performance, whereas lapses in continuous performance tasks may reflect actual attention ability. Proctors documented scenarios potentially affecting data validity, such as participant’s distractions, lack of effort, technical errors, or assessor deviations. The first author regularly reviewed these reports with proctors to ensure accurate scoring. Only complete data without significant validity concerns were used for analyses.
Except for the MP task, which is scored solely on speed, the other 12 tasks are evaluated based on both accuracy (number of correct responses) and speed (time for correct responses in milliseconds). To establish local normative data for z-transformation, we computed means and standard deviations (SD) of task accuracy and speed for each age group (age 6–12, divided by year into seven age groups, balancing males and females) among children without ASD who provided valid and complete data. After assessing the normality of the distributions, the speed scores of all tasks except for SCPT and GNG were log10-transformed before z-transformation. To reflect faster performance as higher scores, z-transformed speed scores were multiplied by –1. An efficiency score for each task was computed by summing the z-transformed accuracy and inverted speed scores, with higher scores indicating better performance. The details of the normative data are summarized in Supplementary Table S1.
Statistical analysis
Statistical analyses were performed using R version 4.4.1. Comparisons of demographics, clinical, and task-level performance of CNB-HK were examined using χ2 tests for categorical data and independent t or Mann–Whitney’s U tests for continuous data. The data structure of the CNB-HK was examined using the R package “lavaan” (Rosseel, Reference Rosseel2012) for confirmatory factor analysis (CFA) and “psych” (Revelle, Reference Revelle2020) for exploratory factor analysis (EFA). CFAs were conducted using maximum likelihood estimation, and EFAs used least-squares extraction with oblimin rotation. A direct implementation of CFA was first conducted, followed by EFAs to detect subtle factor structure anomalies, such as cross-loadings specific to the study population, to aid fine-tuning the CFA model (if necessary). Although unconventional, this CFA-first approach was chosen because the factor structures of the CNB-HK were theory-driven (Moore et al., Reference Moore, Reise, Gur, Hakonarson and Gur2015). The original Moore et al. (Reference Moore, Reise, Gur, Hakonarson and Gur2015) study used a CFA-first approach, as did a previous translational adaptation of the CNB (Pelt et al., Reference Pelt, Scott, Morales, Matshaba, Gur, Tshume, Thuto, Lowenthal and Moore2022). Model fit indices included the root mean square error of approximation (RMSEA) (Steiger, Reference Steiger1990), the comparative fit index (CFI) (Bentler, Reference Bentler1990), and the standardized root mean square residual (SRMR): RMSEA ⩽ 0.05 indicates good fit (⩽0.08 reasonable); CFI ⩾ 0.90 suggests reasonable fit (⩾0.95 preferable); and SRMR ⩽ 0.08 indicates good fit (Hu & Bentler, Reference Hu and Bentler1999). To test construct validity, correlations of the CNB-HK performance and estimated FSIQ were examined with Spearman’s rho. Finally, ANCOVA and MANCOVA were used to compare CNB-HK performance between children with and without ASD. A statistical threshold of alpha ⩽ 0.05 was considered significant.
Results
Clinical and demographic data
The sample included 1,048 children (636 with ASD, 412 without ASD). The ASD group had more males (87.1% vs. 55.1%; χ2 = 120, df = 1, p < 0.001) and was slightly younger (mean ages 8.37 vs. 8.59; U = 121166, p = 0.037). Parental education levels were comparable between the two groups. Among the children with ASD, 42.8% had co-occurring attention-deficit/hyperactivity disorder (ADHD). The mean estimated IQ for the ASD group was 97.5 with a standard deviation of 16.1, approximating the general population’s IQ distribution (Table 2).
Table 2. Demographics of participants and comparisons between children with and without ASD
Note: *p < 0.05, ***p < 0.001. ASD = Autism Spectrum Disorder, ADHD = Attention-deficit/hyperactivity disorder.
Feasibility of CNB-HK and comparison of task-level scores
The CNB-HK was well-tolerated and could be completed by most participants in one hour. Table 3 summarizes the performance and proportion of invalid data at the task level. Children with ASD had lower sensorimotor speed (MP), performed more poorly in word memory (CPW), emotion differentiation (MEDF), emotion recognition (ER), set-shifting (PCET), and inhibition (GNG). Although the ASD group had more invalid data, the proportion was only in the range of 1.10% to 6.40% across different tasks. Most invalid data resulted from participants’ inability to understand the task or lack of motivation. In the ASD group, 72.7% provided valid data for the entire CNB-HK battery (12 tasks), while 91.1% provided valid data for at least 11 out of 12 tasks. The proportion of full valid data varied by age, increasing from 53.2% for 6-year-olds to 94.1% for 12-year-olds. Among 6-year-olds with ASD, 70.3% provided valid data for 11 tasks and 80.2% for 10 tasks. The three executive function tasks (SLNB, GNG, CPT) had the highest rates of invalid data (18.4–19.3%) in this specific group (Supplementary Table S2). These missing data were handled using maximum likelihood estimation in the subsequent factor analyses.
Table 3. Comparisons on invalid data rate and standardized scores of the 13 CNB-HK tasks between children with and without ASD
Note: **p < 0.01, ***p < 0.001. All scores are z-transformed efficiency scores, except for the Motor Praxis Test, which is a z-transformed speed score multiplied by −1 to align the direction of better performance with other scores. MP = Motor Praxis Test, SVOLT = Visual Object Learning Test, CPW = Word Memory Task, CPF = Face Memory Task, MEDF = Emotional Differentiation Test, ADT = Age Differentiation Test, ER = Emotion Identification Test, PMAT = Matrix Reasoning Test, PCET = Penn Conditional Exclusion Test, VSPLOT = Line Orientation Test, SCPT = Penn Continuous Performance Test, SLNB = Letter N-Back Task, GNG = Go/No-Go Task, ASD = Autism Spectrum Disorder.
Confirmatory factor analysis 1: correlated-factor models
The theory-driven correlated 4-factor CFA model of efficiency scores on the entire sample of 1,048 children with and without ASD showed good model fit, with CFI = 0.947, SRMR = 0.036, and RMSEA = 0.047. The fit indices for all factor models are summarized in Table S3. All factor loadings exceeded 0.5, except for ADT (0.43). Correlations between the four cognitive domains ranged from 0.43 to 0.81 (Figure 1). A moderate correlation (0.68) between complex cognition and executive function was expected due to overlapping constructs, while social cognition and episodic memory also showed a strong correlation (0.81). The correlated 3-factor models for speed and accuracy also had acceptable fit indices (Table S3), with all the tasks loading significantly onto their corresponding factor (Supplementary Figures S1 and S2).
Figure 1. Correlated four-factor model for the efficiency scores of the CNB-HK.
Confirmatory factor analysis 2: bi-factor models
Based on the established cognitive theory and the original data of the CNB (Moore et al., Reference Moore, Reise, Gur, Hakonarson and Gur2015) which indicate the presence of a general (g) factor, bi-factor models (Reise, Reference Reise2012) for efficiency, accuracy, and speed scores were specified, with the cognitive domains and g factor being orthogonal to each other. The model of the efficiency scores fit acceptably, with a CFI = 0.947, SRMR = 0.039, and RMSEA = 0.051. While all 12 tasks loaded significantly on the g factor, tasks from episodic memory no longer loaded significantly onto their specific domain (Figure 2). A similar pattern, where certain tasks no longer loaded onto their specific domain beyond g, was observed in the bi-factor models of accuracy and speed (Supplementary Figures S3 and S4).
Figure 2. Bi-factor model for the efficiency scores of the CNB-HK.
Exploratory factor analyses
EFAs on the entire sample were conducted and scree plot, parallel analysis, model fit indices, and theoretical interpretability were used to determine the optimal number of factors. Scree plot and parallel analysis indicated that a four-factor solution was optimal for the efficiency scores (Figure 3). We compared factor structures from a unidimensional model to a four-factor solution (Table 4). In the unidimensional model, while the loadings of all tasks ranged from 0.370 to 0.645, its fit indices were not satisfactory (Supplementary Table S3). From the 2-factor solution onward, the fit indices improved progressively, and tasks of complex cognition and executive function began to form their distinct factors. This remained true in the 4-factor solution with the fit indices CFI = 0.986, SRMR = 0.021, and RMSEA = 0.037. However, the episodic memory and social cognition tasks did not separate in the 3-factor solution, and there was a mixing of tasks loading onto each of these two factors in the 4-factor solution (Table 4). We examined whether this factor structure was specific to ASD by re-running the EFA separately for the children with and without ASD. While the 4-factor solution of the ASD group closely resembled that of the entire study population, the without ASD group had a 4-factor solution that replicated the exact theory-based factor structure of the CNB (Moore et al., Reference Moore, Reise, Gur, Hakonarson and Gur2015), with all 12 tasks loading primarily on the intended cognitive domain (Supplementary Tables S4 and S5).
Figure 3. Scree plot and parallel analysis of the 12 CNB-HK tasks efficiency scores.
Table 4. Exploratory factor analyses on the CNB-HK efficiency scores from unidimensional to four-factor solutions
Note: Loadings < 0.20 removed. * = significant at 1% level. SVOLT = Visual Object Learning Test, CPW = Word Memory Task, CPF = Face Memory Task, MEDF = Emotional Differentiation Test, ADT = Age Differentiation Test, ER = Emotion Identification Test, PMAT = Matrix Reasoning Test, PCET = Penn Conditional Exclusion Test, VSPLOT = Line Orientation Test, SCPT = Penn Continuous Performance Test, SLNB = Letter N-Back Task, GNG = Go/No-Go Task.
The exploratory solutions for accuracy and speed are presented in Supplementary Tables S6 and S7. Three-factor solutions were optimal for both metrics, similarly aligning with the original study (Moore et al., Reference Moore, Reise, Gur, Hakonarson and Gur2015).
Factor scores generation and measurement invariance
To generate factor scores, we examined the factor determinacy (FD) (Grice, Reference Grice2001), where a FD value above 0.8 is considered adequate for estimation of factor scores for each cognitive domain (Gorsuch, Reference Gorsuch1983). Across the models for efficiency, accuracy, and speed, the g factors had an FD of >0.8 in the bi-factor models, and the specific cognitive domains achieved FD values >0.8 only in the correlated-factor models (Supplementary Table S8). Hence, in the subsequent analyses, the g factor scores were generated from the bi-factor models, while the domain scores were from the correlated-factor models. Note that these scores come from different models and hence could not be used simultaneously in downstream analyses due to redundant variance in “g” and the correlated factors.
To compare the factor scores between the two populations, measurement invariance of the CNB-HK was examined. As sole use of Δχ2 as a criterion for measurement invariance would be oversensitive given our sample size, we considered primarily the fit indices’ changes at each level of configural, metric, and scalar invariance. The threshold of < −0.01 change in CFI, and <0.015 change in SRMR and RMSEA was adopted (Putnick & Bornstein, Reference Putnick and Bornstein2016). Configural and metric invariance for both the bi-factor and correlated four-factor models were established. While the Δχ2 was significant for scalar invariance of the bi-factor model, fit indices changes (ΔCFI = −0.006; ΔSRMR = 0.002; ΔRMSEA = 0.001; Δχ2 = 19.16, Δdf = 7, p = 0.003) remained below thresholds, confirming scalar invariance. However, scalar invariance was violated in the correlated four-factor model with ΔCFI = −0.057; ΔSRMR = 0.012; ΔRMSEA = 0.019; Δχ2 = 125.90, Δdf = 8, p < 0.001. Univariate tests revealed that ADT, CPW, PCET, and SCPT in the correlated four-factor models significantly impacted the model fit. The scalar model was adjusted with these tasks “released” for free estimation in both groups. Subsequently, partial scalar invariance was established, where ΔCFI = −0.006; ΔSRMR = 0.001; ΔRMSEA = 0.002, despite Δχ2 being significant (Δχ2 = 15.65, Δdf = 4, p = 0.003). The results of measurement invariance analysis are summarized in Supplementary Table S3.
Correlations of factor scores with IQ and between-group comparisons
The correlations between factor scores of the CNB-HK and estimated FSIQ of the ASD population are summarized in Table 5. Generally, all factor scores correlated positively with FSIQ, with moderate strength of correlations in efficiency and accuracy factor scores and very weak correlations with speed factor scores. The Complex Cognition/Executive Function accuracy factor score had the strongest correlation with FSIQ (ρ = 0.52, df = 612, p < 0.001).
Table 5. Spearman’s rho correlation matrix for age, FSIQ, and CNB-HK factor scores (Efficiency, accuracy, speed) in the ASD group
Note. *p < 0.05, **p < 0.01, ***p < 0.001, FSIQ = Full-Scale Intelligence Quotient, ASD = Autism Spectrum Disorder.
Comparison of the efficiency factor scores between children with and without ASD was conducted with a MANCOVA for the four cognitive domains and an ANCOVA for g. Both analyses were adjusted for age and sex, with the factor scores from the CFA reflected and log-transformed before analysis. Overall, the ASD group had worse cognitive performance in the four cognitive domains (Pillai’s Trace = 0.08, F (5, 994) = 18.04, p < 0.001, ηp2 = 0.08), as well as g (F = 19.06, p < 0.001, ηp2 = 0.019). Univariate tests showed that although the ASD group had worse cognitive performance in the four specific domains, the effect sizes were different, with ηp2 = 0.017 and 0.020 respectively for episodic memory and social cognition, while complex cognition and executive function had ηp2 < 0.01 (Table 6, and Figure 4). As only partial scalar measurement invariance was established for the correlated-factor model, a reduced fully invariant correlated-factor model, with ADT, CPW, PCET, and SCPT removed (i.e. the tasks “released” for partial scalar measurement invariance), was built (Supplementary Table S9). The MANCOVA analysis was re-run and showed comparable results (Supplementary Table S10).
Figure 4. Comparison of the domain-specific factor efficiency scores of the CNB-HK between children with and without ASD.
Note: Error bars represent 95% confidence intervals.
Table 6. MANCOVA on cognitive domain factor efficiency scores between children with and without ASD
Note. Results were controlled for age and sex. *p < 0.05, **p < 0.01, ***p < 0.001. ASD = Autism Spectrum Disorder.
Discussion
In the present study, we demonstrated the feasibility of administering the CNB-HK to a cohort of children with and without ASD aged 6–12 years. Factor structure of the CNB-HK replicated that of the original CNB, and task performance could be summarized into factor scores that correlated positively with FSIQ. Thus, the CNB-HK is a valid instrument for assessing multi-domain cognitive function in children with ASD. Compared to children without ASD, children with ASD performed more poorly across all cognitive domains.
Feasibility of the CNB-HK as a multi-domain cognitive assessment for children with ASD
The merits of the CNB-HK include its automated scoring, ease of administration, and ability to assess major neurocognitive domains efficiently in one hour. Additionally, the CNB-HK assesses cognitive domains critical to ASD: social cognition, a domain linked to the social communication deficit; processing speed, which is often impaired in ASD (Zapparrata et al., Reference Zapparrata, Brooks and Ober2023); and executive function, which is associated with prognosis (Kenny et al., Reference Kenny, Cribb and Pellicano2019). These features support the CNB-HK’s ecological validity for ASD research.
Most children with ASD completed the battery, with 72.7% providing valid data for all 12 tasks and 91.1% for at least 11 tasks, indicating broad applicability for school-age ASD. However, a notable limitation was the higher invalid data rate in 6-year-old ASD, with only 53.2% completing all 12 tasks in this age group. Despite this, 70.3% and 80.2% of 6-year-olds could provide respectively valid data for at least 11 and 10 tasks. With three tasks per cognitive domain, the CNB-HK allows missing data from a single task to be managed using imputations. While this supports applicability for 6-year-olds, imputation may introduce assumptions affecting result precision. The invalid data among 6-year-olds was particularly notable for executive function tasks (SLNB, GNG, CPT), aligning with findings from the NIH-TCB, where only 57% of participants with ASD, including preschoolers (as young as 3 years) and individuals with intellectual disability (ID), completed the four tasks that assessed executive function (Jones et al., Reference Jones, Dallman, Harrop, Whitten, Pritchett, Lecavalier, Bodfish and Boyd2022). These findings caution the applicability of existing executive function tasks, including those in the CNB-HK, to populations with younger mental ages. Adaptations such as using a gamified platform, shortening, and adjusting the difficulty of the executive tasks may be necessary for preschoolers to children around 6 years old (Moore et al., Reference Moore, Di Sandro, Scott, Lopez, Ruparel, Njokweni, Santra, Conway, Port, D’Errico, Rush, Wolf, Calkins, Gur and Gur2023). In summary, we recommend a minimum age of 6 years for CNB-HK administration, with the caveat that executive function tasks may require tailoring for younger children.
Factor structure of the CNB-HK and factor scores generation
With the necessary task-level changes for adaptation of the CNB to the Chinese population, the 4-factor CFA models (bi-factor and correlated-factor models) fit well with the data when all tasks were assigned to their respective designed factors, consistent with other CNB adaptations for youth with HIV in Botswana (Pelt et al., Reference Pelt, Scott, Morales, Matshaba, Gur, Tshume, Thuto, Lowenthal and Moore2022), Xhosa-speaking adults with schizophrenia in South Africa (Scott et al., Reference Scott, Moore, Stein, Pretorius, Zingela, Nagdee, Ngqengelele, Campbell, Sibeko, King, McClellan, Port, Jackson, Ruparel, Susser and Gur2021), and healthy individuals in Portugal (da Motta et al., Reference da Motta, Carvalho, Castilho and Pato2021). This demonstrates that the factor structure of the CNB is stable cross-culturally for trans-diagnostic comparisons. Curiously, in the EFA, tasks within episodic memory and social cognition did not separate in the ASD group, and the two domains showed a high correlation. At first glance, a 3-factor solution may appear sufficient in the population with ASD. Yet, the observation can be explained by the core deficits of ASD leading to similar task performances in the two domains, while considering task impurity and shared task demand in cognitive assessment. For instance, word memory (CPW) captures language impairment which is causally linked to social deficits (Constantino et al., Reference Constantino, Gruber, Davis, Hayes, Passanante and Przybeck2004), while face memory (CPF) and social cognition tasks (ADT, ER, MEDF) all require facial stimuli processing, a key component of social cognition (Haxby et al., Reference Haxby, Hoffman and Gobbini2002). Additionally, this correlation may be related to shared neural mechanisms in the temporo-limbic system, with evidence suggesting that social interaction and memory are supported by the hippocampus in ASD (Banker et al., Reference Banker, Gu, Schiller and Foss-Feig2021; Roesler et al., Reference Roesler, Parent, LaLumiere and McIntyre2021). These biological mechanisms warrant further investigation to clarify their role in the observed correlation. The EFA results did not invalidate the ability of the tasks to capture the performance of their intended domains, given that in the correlated-factor CFA model, all the tasks still loaded significantly onto their intended domain.
When generating factor scores from factor analysis, the loadings and number of the indicators (i.e. tasks) for a factor are critical to whether the score is well-determined. In the bi-factor CFA model, when all 12 tasks contributed as indicators of the g factor, little variance was left for the specific cognitive domains. Consequently, while the g factor had adequate factor determinacy, the domain-specific factors in the bi-factor model did not; they were sufficiently determined only in the correlated-factor model. Although the orthogonality of the factors within the bi-factor model can be advantageous for examining specific cognitive effect, achieving determinacy for domain-specific factors in the bi-factor model would require substantially increasing the number of tasks per domain, which would prolong and compromise the feasibility of the battery. Conversely, even though the domain-specific factor scores in the correlated-factor CFA model were well-determined, their specificity was compromised due to shared variance among the scores. The phenomenon of low factor loadings for specific domains after accounting for the g factor, as well as the trade-off between specificity and determinacy were also observed in the original study of the CNB (Moore et al., Reference Moore, Reise, Gur, Hakonarson and Gur2015). In optimal circumstances, a bi-factor model provides well-determined general and specific factor scores, allowing one to study all factors simultaneously – e.g., the general and specific factor scores can be included in a regression model together – without the cross-contamination between the specific factors and the general factor (as in the correlated-traits model). Here, however, specific factors of the bi-factor model had such weak loadings that they were too poorly determined to generate reliable scores. We therefore had to use both models, each for a different purpose: the correlated-factor model was our only option for examining domain-specific cognitive abilities, such as targeting specific deficits in ASD (e.g., episodic memory or social cognition), due to its adequate factor determinacy. In contrast, the bi-factor model does provide better estimates of general cognitive performance (g factor) than, for example, simply taking the average of the correlated-factor scores; we therefore used the bi-factor model for overall “g” scores.
Correlation of the CNB-HK factor scores with FSIQ
The generalized positive correlation between all factor scores (g and domain-specific accuracy, speed, and efficiency scores) and estimated FSIQ provided construct validity for the CNB-HK as a cognitive assessment tool for children with ASD. As both the WISC-IV (HK) Short Form IQ subtests include complex cognition and executive tasks, FSIQ exhibited the strongest correlation with the accuracy factor score of these two domains. Except for the digit symbol substitution task, the subtests of the WISC-IV did not consider the speed of response for scoring, which may explain the weak correlations between FSIQ and the speed factor scores.
Measurement invariance and comparison between children with and without ASD
Measurement invariance is an important aspect to ensuring comparability of the factor scores between the groups. Briefly, configural invariance confirms that the same factor structure (e.g., cognitive domains) holds across groups, while metric invariance ensures that factor loadings are equivalent. Scalar invariance verifies that item intercepts are equivalent to enable valid comparisons of group means (Putnick & Bornstein, Reference Putnick and Bornstein2016). In our study, even though configural and metric invariance were both established, we could only achieve partial scalar invariance of the correlated-factor CFA model. Although partial scalar invariance may introduce estimation bias in between-group comparisons, the extent to which such violations preclude valid comparisons remains under investigation (Davidov et al., Reference Davidov, Meuleman, Cieciuch, Schmidt and Billiet2014; Putnick & Bornstein, Reference Putnick and Bornstein2016). To examine spurious results from such violations, a sensitivity analysis was done with a reduced and fully invariant model. As similar results were observed, the impact of the partial scalar invariance may not be significant.
Although the children with ASD in this study were recruited from a specialist clinic, they represented a spectrum of clinical severity (with and without ADHD), and varying intelligence levels (a mean FSIQ of 97.5 and SD of 16.10). The MANCOVA showed that, at the factor level, the ASD group had a generalized cognitive impairment across all cognitive domains and the g factor. However, the effect sizes of the cognitive impairments vary. Social cognition, a core deficit of ASD, exhibited the largest effect size, followed by memory, while effect sizes for complex cognition and executive function were smaller, indicating less pronounced impairments in these areas. This pattern of varying effect sizes was consistent with the findings from the ASD group in the PNC (Wong et al., Reference Wong, Barzilay, Lam, Chan, Calkins, Gur and Gur2023). Although executive dysfunction is common in individuals with ASD (Sadozai et al., Reference Sadozai, Sun, Demetriou, Lampit, Munro, Perry, Boulton and Guastella2024), the small effect size for executive function differences between children with and without ASD (ηp2 = 9.00E-03) suggests some overlap in performance distributions. This indicates that some children with ASD likely possess intact executive function, despite group-level impairments. Well-developed executive function has been conceptualized as protective in neurodevelopmental conditions owing to its postulated compensatory mechanism against other deficits (Johnson, Reference Johnson2012). It is also a major determining factor for co-occurring psychopathology and functional level in youth with ASD (Wong et al., Reference Wong, Barzilay, Lam, Chan, Calkins, Gur and Gur2023), and their longitudinal outcomes (Kenny et al., Reference Kenny, Cribb and Pellicano2019). Given the variability of executive function and clinical profile within our cohort, further exploration of their interplay may shed light on the linkage of cognitive function and clinical heterogeneity of ASD. Previous studies have also identified clinical correlates of social cognition, memory, and complex cognition in ASD (Bishop-Fitzpatrick et al., Reference Bishop-Fitzpatrick, Mazefsky, Eack and Minshew2017; Edirisooriya et al., Reference Edirisooriya, Dykiert and Auyeung2021; Toichi, Reference Toichi2008). To this end, the CNB-HK allows concurrent examination of multiple cognitive domains to pinpoint specific correlates and predictors of clinical outcomes.
Limitations
There are several limitations in the present study. First, since the ID and younger populations were not included, the results from this study may not be generalizable to these populations. As discussed, the CNB-HK is feasible for individuals with a mental age of six years or above. Further adaptation of the battery for the younger and ID population is needed. Second, the factor determinacy was inadequate for domain-specific factor scores from the bi-factor model, such that only the scores from the correlated-factor model could be used. This limits the specificity of the analysis using the scores with shared variance. Third, the CNB-HK was not fully invariant at the scalar level, and the effect of this non-invariance on the subsequent between-group comparisons is not fully understood. Fourth, although the CNB-HK factor scores exhibited a generalized positive correlation with estimated IQ, they have not been validated against established neuropsychological measures of the respective cognitive domains. Finally, females were relatively underrepresented among children with ASD, and the intrinsic sex differences in cognitive ability should be addressed for in future studies.
Conclusion
Our study showed that the CNB-HK is a feasible and valid multi-domain cognitive assessment battery for normal IQ school-age individuals with and without ASD. The stable and replicable factor structure enables cross-cultural and trans-diagnostic comparison of cognitive functions in ASD with other conditions. Future research should examine clinical correlates of cognitive profiles to guide understanding of the heterogeneity of ASD.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1355617725101653.
Acknowledgments
None.
Funding statement
This work was support by The D.H. Chen Foundation.
Competing interests
The authors declare no conflict of interest.
Open access
