2.1. Participants

Participants were bilingual (n = 88) and trilingual (n = 29) children growing up in France (n = 15), the Netherlands (n = 56) or the United Kingdom (n = 46), aged between 5 and 9 years old (mean = 83 months, sd = 12). This dataset excludes children who did not have a full set of measures as required by our analyses. Recruitment targeted schools in deprived vs privileged areas and relied on existing participant databases and contacts from outreach activities. In France, assistants helped the parents fill in the questionnaire if they had no computer equipment or limited literacy. The original sample included children recruited via speech and language therapy clinics in France. These children were excluded from the present study. In the Netherlands, children with a known history of speech and language therapy (SLT) were excluded from recruitment. In the UK, no exclusion criterion was applied (i.e., the SLT status of children was unknown).

Children’s language experience background was documented using the full version of the Q-BEx questionnaire. The only exception was that in France, the language mixing module was not implemented. Just under half of the participants (n = 54) had been exposed to the Societal Language (SL: either French, Dutch, or English) from birth. For the rest (n = 63), SL exposure started between 1 and 84 months of age.

As shown the left pane of Figure 2, the amount of cumulative experience to the SL varies substantially, both within the sequential and the simultaneous bilinguals. Relatedly, in the simultaneous bilinguals, the correlation between (biological) age and cumulative SL exposure is relatively weak (r = 0.36, p = 0.01).

Figure 2. Correlation between onset of exposure and cumulative exposure in each language (left: SL; right: HL).

All children were exposed to a Heritage Language (HL) from birth. In this study, we only consider children’s strongest HL in the case of trilinguals. We use the term ‘bilingual’ to refer to both bilingual and trilingual children, and the term HL to designate the child’s strongest or unique HL. Across our sample, these HLs were: Afrikaans, Arabic, Bangla, Bulgarian, Chinese, Creole, Czech, English, French, Frisian, German, Greek, Gujarati, Hindi, Hindko, Italian, Japanese, Kurdish, Latvian, Lithuanian, Lingala, Manouche, Mirpuri, Polish, Portuguese, Punjabi, Romanian, Russian, Spanish, Telugu, Turkish, Ukrainian and Urdu. The number of participants per HL per country is given in Table 6 in the Supplementary Materials. Another aspect of the diversity of our sample can be seen in the right pane of Figure 2, which shows the correlation between onset of SL exposure and cumulative experience in the (main) HL.

Two thirds (66%) of participants grew up in homes where the highest level of caregiver education was university level. A sixth (15%) did not have post-secondary school education.

In sum, our sample features substantial variability in terms of age of onset of bilingualism, languages experienced in the home and in the society, and the amount of experience in the HL and the SL. There is also variability in the highest education level achieved by caregivers across their languages.

2.2. The Q-BEx questionnaire

We asked children’s caregivers to complete the full version of Q-BEx, including all components from all modules (i.e., background information, risk factors, language exposure and use, language proficiency, richness, attitudes, language mixing).Footnote ¹ It took between 15 and 130 minutes to complete (35 minutes on average).Footnote ²

In this study, we focus on the Q-BEx module with the highest impact on questionnaire completion time, that is, language exposure and use.Footnote ³ However, we also consider information from other Q-BEx modules, when they are relevant covariates alongside language exposure, in models predicting language outcomes (as explained in the section about language outcome measures). Table 7 in the Supplementary Materials lists these covariates and identifies the Q-BEx module from which the relevant information was elicited.

2.3. Alternative estimates of language exposure

Table 1 lists the alternative estimates of language exposure considered in this study. For each, it specifies how the estimate is calculated and on the basis of what information from the questionnaire. Whenever they apply to both languages, the estimates are calculated in the same way for both the SL and the HL. However, some estimates are only considered with respect to the HL: we assume that exposure in the home and the number of interlocutors are likely to be reliable predictors for the HL but not for the SL. Indeed, for many heritage bilinguals, the home is where most of their HL experience takes place, and the number of HL interlocutors can be very small. HL exposure during the holidays seems to be particularly important for HL acquisition and maintenance (as shown, for example, by Kubota & Rothman, Reference Kubota and Rothman2024). It is currently unclear whether the same is true of the SL.

Table 1. Questionnaire detail and calculation for each of the alternative measures of language exposure

Both for current exposure estimates and for cumulative estimates, the order of presentation in the table (from top to bottom) broadly aligns with a decrease in the level of questionnaire detail required. This corresponds to the ‘cost’ of collecting that information in terms of respondent time. This cost can be interpreted as the ‘complexity’ of the measure, in terms of information quantity. Although we administered the longest version of the questionnaire, the alternative measures we compare are proxies for what would be obtained with a shorter version of the questionnaire.

In summary, we compared six estimates of SL exposure (as alternative predictors of morphosyntax and vocabulary outcomes) and eight estimates of HL exposure (as alternative predictors of HL outcomes). Based on questionnaire completion time, the most costly estimate for each language was current exposure adjusted (i.e., weighted) for the amount of time the child spends with each (group of) interlocutor(s) in each context. We considered four alternative estimates of current exposure: (i) the weighted estimates (defined above), (ii) their nonweighted counterpart, (iii) a global estimate (calculated as an average by context)Footnote ⁴ and (iv) a global estimate of exposure during holidays. In the case of the HL, we additionally considered (v) a global estimate of current exposure in the home. Furthermore, for both languages, we included (vi) an estimate of cumulative exposure (consisting in the sum of global estimates for each respondent-defined period in the child’s life). We also considered (vii) the age of SL onset as a coarse estimate of cumulative exposure to each language (indicating the length of SL exposure, or the length of any monolingual exposure to the HL). Finally, (viii) the number of HL interlocutors was considered as a coarse approximation for HL exposure.

2.4. Language outcome measures

Objective measures of proficiency in the SL (French, Dutch or English) were obtained for morphosyntax and vocabularyFootnote ⁵, as there is ample evidence that they are affected by language exposure (e.g., Hoff, Reference Hoff2003; Thordardottir, Reference Thordardottir2011; Verhagen et al., Reference Verhagen, Boom, Thieme, Kuiken, Keydeniers, Aalberse and Andringa2024).

Morphosyntactic abilities in the SL were measured using the LITMUS Sentence Repetition Task (SRT; Marinis & Armon-Lotem, Reference Marinis, Armon-Lotem, Armon-Lotem, de Jong and Meir2015). The reliability of this methodology is well established (see, e.g., the scoping review by Rujas et al., Reference Rujas, Mariscal, Murillo and Lázaro2021). Repeating a sentence taps into underlying linguistic knowledge in order to parse the stimulus sentence and then reconstruct its structure and meaning. If the morphosyntactic properties featured in the sentence have not been acquired or are too complex to compute, performance is negatively affected (Marinis & Armon-Lotem, Reference Marinis, Armon-Lotem, Armon-Lotem, de Jong and Meir2015; Polišenská et al., Reference Polišenská, Chiat and Roy2015).

In the French version, the SRT consisted of 16 sentences varying in complexity, from less (short sentences in present simple) to more (object relative clauses) complex. In the English and the Dutch versions, the SRT consisted of 30 sentences. They were ordered in two blocks (of 16 and 14 items, respectively) of identical complexity, so that the first block would be as close and equivalent to the French version as possible. No significant difference in accuracy was observed between the two blocks (see Prévost et al. Reference Prévost, Gusnanto, Kašćelan, Serratrice, Tuller, Unsworth and De Catunder review). In all three versions, the sentences were presented auditorily using headphones in a fixed order. Response accuracy was scored in three different ways (eliciting 3 scores), as defined in (3):

This yielded three outcome measures, which we transformed into percentages (i.e., the proportion of accurate repetitions out of the total number of items).

Vocabulary breadth, which corresponds to a receptive measure of vocabulary size, was assessed using the receptive Peabody Picture Vocabulary Task: BPVS-3 for English (Lloyd Dunn et al., Reference Dunn, Dunn, Sewell, Styles, Brzyska, Shamsan and Burge2009), EVIP for French (Dunn et al., Reference Dunn, Thoriault-Whalen and Dunn1993) and PPVT-III-NL for Dutch (Dunn et al., Reference Dunn, Dunn and Schlichting2005). In this task, children are presented with four pictures, they hear a single word and are asked to point to the corresponding picture. Administration followed the instructions in the manual, starting at the age-appropriate starting set and moving up (and if necessary, down) until the required number of errors was met. The outcome measure is the total number of correct responses (i.e., the raw score).Footnote ⁶

Receptive vocabulary depth, which corresponds to how well words are known, was assessed using the Word Classes sub-test of the CELF-5 in English (Semel et al., Reference Semel, Wiig and Secord2017) and French (Wiig et al., Reference Wiig, Semel and Wayne2019), and CELF-4 for Dutch (Kort et al., Reference Kort, Schittekatte and Compaan2008). In this task, children hear words and are asked to indicate which words belong together. As the task progresses, the number of words from which children need to make a selection increase from three to four and visual support in the form of pictures is removed. Administration followed the instructions in the manual until children reached the end or failed to provide a correct response to the required number of consecutive items (four in English, five in Dutch and French). The proportion of correct responses out of the total number of items answered was included as dependent variable in the analyses.

Subjective measures of proficiency outcomes in the HL were derived from parental estimates (based on questions from the Q-BEX module on language proficiency). The questions asked how well the child speaks and how well they understand the HL for their age. We did not include reading and writing abilities, as many children had not learned to read yet. Answer options included hardly at all/not very well/pretty well/very well.

2.5. Cognitive measures

To include as control variables in the models predicting language outcomes, we collected measures of non-verbal intelligence, short-term memory and working memory. Non-verbal intelligence was assessed with the Matrices task from the WISC–V (Wechsler, Reference Wechsler2014) or the WPPSI (Wechsler, Reference Wechsler2013) (the latter for children under 6 years of age).

Short-term memory was assessed through Forward Digit Recall (FDR); working memory was assessed through Backward Digit Recall (BDR). Both tasks were administered through Psychopy, according to the protocol described in Hill et al. (Reference Hill, Shire, Allen, Crossley, Wood, Mason and Waterman2021). Children were presented auditorily with sequences of numbers (through headphones) and asked to repeat these numbers either in the same order (in the FDR task) or in reverse order (in the BDR task). The length of the sequence increased by one digit after 4 trials, starting with 3 digits in the first block of the FDR task, and 2 digits in the first block of the BDR task. The maximum sequence length was 6 digits in the FDR task and 5 digits in the BDR task.

2.6. Modelling procedures

We adopted a predictive modelling approach throughout this study. This determines the baseline assumptions for the two sets of analyses reported below. Predictive modelling aims to identify the smallest set of predictor variables required for a model to achieve the highest level of generalisability in relation to an outcome variable. It seeks to maximise the variance explained by the model, while avoiding over-fitting the data. It does not aim to interpret the individual effect of a particular variable. Our quest for the optimal estimate of language exposure (as a predictor of language proficiency) is therefore situated in the context of a previously identified optimal set of predictor variables. In other words, the control variables included in each model were determined by the optimal predictive model we identified in previous work for each of the outcome variables (in the same group of children): We refer the reader to Prévost et al. (Reference Prévost, Gusnanto, Kašćelan, Serratrice, Tuller, Unsworth and De Catunder review) for SL morphosyntax outcomes, Serratrice et al. (Reference Serratrice, Gusnanto, Kašćelan, Prévost, Tuller, Unsworth and De Catunder review) for SL vocabulary outcomes, and De Cat et al. (Reference De Cat, Gusnanto, Kašćelan, Prévost, Serratrice, Tuller and Unsworthin preparation) for outcomes of oral proficiency in the HL. The optimal set of control variables retained in each model is listed in the Supplementary Materials. Across models, this included the following aspects: SL (coinciding with country of residence), parental education, cognitive measures, phonological competence, attitudes towards the SL, early language development milestones and concerns, diversity and richness of the language environment and experience. The type of regression depended on the distribution of the outcome variable. We used beta regressions for percent scores (SRT and vocabulary depth), linear regression for numeric scores (vocabulary breadth) and cumulative link models for ordinal scores (parental estimates of HL proficiency).

For the first set of analyses, we use the information-theoretic approach to model selection proposed by Burnham and Anderson (Reference Burnham and Anderson2003). Rather than selecting a single ‘best’ model, this approach considers a set of models, and makes inferences to determine the relative support for each model in the set. It thereby acknowledges the unavoidable uncertainty in model selection. Akaike weights, derived from the AIC, are calculated for each model, balancing model fit against model complexity to identify the most parsimonious model. The weights add up to 1 (i.e., 100%) across the models in the set. A model’s weight can be interpreted as the probability (expressed as a percentage) that a given model is the best in the set, as will be explained in detail in the Results section. This method provides a rigorous framework to compare alternative predictors within an otherwise identical model (including the same set of covariates), by focusing on the strength of the evidence rather than arbitrary significance thresholds. In that framework, differences in model parsimony can be interpreted as differences in the parsimony of the alternative predictors (as everything else is held constant).

For each language outcome measure, a series of models are fitted treating that measure as outcome variable, and considering in each model one of the alternative measures of language exposure as predictor, together with a set of control variables. The control variables included in each model were determined by the optimal predictive model we identified in previous work for the outcome variable in question (in the same group of children), as explained under Modelling procedures.

For the second set of analyses, we use cross-validation (Desmarais & Harden, Reference Desmarais and Harden2014) to assesses the true predictive performance of each alternative measure of language exposure (in relation to language outcomes) and estimate the risk of over-fitting. The data are split (by participants) into an estimation set and a validation set (with the same response variable and predictor variables in both sets). We employ k = 5-fold cross-validation, where observations are randomly split into 5 different groups and 4 of them (80%) are assigned as estimation set and the other (20%) as validation set. Model fitting is then performed (5 times) on the estimation set to obtain parameter estimates for each of the language experience measure. The parameter estimates obtained from the estimation set are then used to make ‘new sample’ prediction in the validation set using the same predictors.

In the validation set, therefore, we have a ‘new sample’ prediction on the outcome variable, in addition to the observed outcome variable itself. We calculate the average of squared differences between the two as the root mean square error (RMSE) of prediction. RMSE is interpreted as the predictive ability of the model. The lower the RMSE, the better the predictive ability of the model. This allows us to assess the extent to which the analysis of the validation set could generalise to an independent dataset including different children.

For each language outcome measure, cross-validations are performed on models using, in turn, each of the alternative measures of language experience as well as the same control variables as in the first set of analyses. In addition to RMSE, we calculate the range of RMSE across the five different folds to assess whether the calculation of RMSE is stable across different folds. Furthermore, to measure the uncertainty in the calculation of RMSE, we calculate the confidence interval of RMSE using the bootstrap method (Efron & Tibshirani, Reference Efron and Tibshirani1986, Reference Efron and Tibshirani1994). The bootstrap method is implemented by resampling with replacement observations in the estimation set to obtain bootstrapped RMSEs in the validation set. We can therefore calculate the 95% confidence interval from the 2.5 and 97.5 percentiles of the distribution of bootstrapped RMSEs.

According to the cross-validation analysis, the optimal model (i.e., the one containing the optimal estimate of language exposure) is the one that has the lowest RMSE and that shows the greatest stability of RMSE across folds. Furthermore, the confidence intervals indicate RMSE precision and allow comparison between models.Footnote ⁷ Overlapping CIs across models would suggest that improvements are modest.

When evaluating cost/benefit overall (across the two sets of analyses), the optimal measure of language experience (as predictor of language outcome) should be the one that requires the least questionnaire detail while yielding the best predictive performance, i.e., the smallest error size across folds.