This paper responds to Al-Hoorie, Hiver, and In’nami’s (2024) critique of the second language (L2) Motivational Self System (L2MSS) by advocating for an immediate cessation of its use in the absence of substantial revision and validation. We revisit foundational studies in the tradition, exposing critical methodological flaws that we feel undermine empirical support for the model. Further, we examine systemic factors that contributed to the largely uncritical acceptance of the model. Drawing on our own experiences, we reflect on how these dynamics have obstructed the adoption of more robust motivational theories available in psychology and education. We further caution that without a stronger emphasis on validating measurement instruments, similar distractions may continue to hinder progress in the field.

This is a reaction to Borsboom’s (2006) discussion paper on the issue that psychology takes so little notice of the modern developments in psychometrics, in particular, latent variable methods. Contrary to Borsboom, it is argued that latent variables are summaries of interesting data properties, that construct validation should involve studying nomological networks, that psychological research slowly but definitely will incorporate latent variable methods, and that the role of psychometrics in psychology is that of partner, not role model.

This paper analyzes the theoretical, pragmatic, and substantive factors that have hampered the integration between psychology and psychometrics. Theoretical factors include the operationalist mode of thinking which is common throughout psychology, the dominance of classical test theory, and the use of “construct validity” as a catch-all category for a range of challenging psychometric problems. Pragmatic factors include the lack of interest in mathematically precise thinking in psychology, inadequate representation of psychometric modeling in major statistics programs, and insufficient mathematical training in the psychological curriculum. Substantive factors relate to the absence of psychological theories that are sufficiently strong to motivate the structure of psychometric models. Following the identification of these problems, a number of promising recent developments are discussed, and suggestions are made to further the integration of psychology and psychometrics.

Borsboom (2006) attacks psychologists for failing to incorporate psychometric advances in their work, discusses factors that contribute to this regrettable situation, and offers suggestions for ameliorating it. This commentary applauds Borsboom for calling the field to task on this issue and notes additional problems in the field regarding measurement that he could add to his critique. It also chastises Borsboom for occasionally being unnecessarily perjorative in his critique, noting that negative rhetoric is unlikely to make converts of offenders. Finally, it exhorts psychometricians to make their work more accessible and points to Borsboom, Mellenbergh, and Van Heerden (2003) as an excellent example of how this can be done.

A new model of confirmatory factor analysis (CFA) for multitrait-multimethod (MTMM) data sets is presented. It is shown that this model can be defined by only three assumptions in the framework of classical psychometric test theory (CTT). All other properties of the model, particularly the uncorrelatedness of the trait with the method factors are logical consequences of the definition of the model. In the model proposed there are as many trait factors as different traits considered, but the number of method factors is one fewer than the number of methods included in an MTMM study. The covariance structure implied by this model is derived, and it is shown that this model is identified even under conditions under which other CFA-MTMM models are not. The model is illustrated by two empirical applications. Furthermore, its advantages and limitations are discussed with respect to previously developed CFA models for MTMM data.

Established results on latent variable models are applied to the study of the validity of a psychological test. When the test predicts a criterion by measuring a unidimensional latent construct, not only must the total score predict the criterion, but the joint distribution of criterion scores and item responses must exhibit a certain pattern. The presence of this population pattern may be tested with sample data using the stratified Wilcoxon rank sum test. Often, criterion information is available only for selected examinees, for instance, those who are admitted or hired. Three cases are discussed: (i) selection at random, (ii) selection based on the current test, and (iii) selection based on other measures of the latent construct. Discriminant validity is also discussed.

Sound general and sports nutrition knowledge in athletes is essential for making appropriate dietary choices. Assessment of nutrition knowledge enables evaluation and tailoring of nutrition education. However, few well-validated tools are available to assess nutrition knowledge in athletes. The objective of the present study was to establish the validity of the Platform to Evaluate Athlete Knowledge Sports – Nutrition Questionnaire (PEAKS-NQ) for use in the United Kingdom and Irish (UK-I) athletes. To confirm content validity, twenty-three sports nutritionists (SNs) from elite, UK-I sports institutes provided feedback on the PEAKS-NQ via a modified Delphi method. After minor changes, the UK-I version of the PEAKS-NQ was administered to UK-I SN from the British Dietetic Association Sport and Exercise Nutrition Register, and elite athletes (EA) training at elite sports institutes in the UK and Ireland. Independent samples t-test and independent samples median tests were used to compare PEAKS-NQ total and subsection scores between EA and SN (to assess construct validity). Cronbach's alpha (good ≥ 0⋅7) was used to establish internal consistency. The SN achieved greater overall [SN (n 23) 92⋅3 (9⋅3) v. EA (n 154): 71⋅4 (10⋅0)%; P < 0⋅001] and individual section scores (P < 0⋅001) except Section B, Identification of Food Groups (P = 0⋅07). Largest knowledge differences between SN and EA were in Section D, Applied Sports Nutrition [SN: 88⋅5 (8⋅9) v. EA: 56⋅7 (14⋅5)%; P < 0⋅00]. Overall ES was large (2⋅1), with subsections ranging from 0⋅6 to 2⋅3. Cronbach's alpha was good (0⋅83). The PEAKS-NQ had good content and construct validity, supporting its use to assess nutrition knowledge of UK-I athletes.

Psychopathologists have failed to make significant progress toward understanding the causes of psychopathology. Despite the foundational importance of construct validity and measurement to our field, insufficient attention is paid to these concerns in the assessment of psychopathology vulnerabilities prior to their implementation in causal models. I review the current state of construct validity and measurement in psychopathology research, highlighting the lack of consensus regarding how we should define and measure vulnerability constructs. The limited capacity of open science practices to address these definitional and measurement challenges is discussed. Recommendations for progress are made, including the need for consensus agreement on (1) working definitions and (2) measures of vulnerability constructs. Other recommendations include (3) the need to incentivize ‘pre-clinical’ descriptive work focused on measurement development, (4) the formation of open-access databases designed to facilitate measurement evaluation and development, and (5) increased exploration of the use of novel technologies to facilitate the collection of high-quality measures of vulnerability.

In this paper, we evaluate the factorial validity of the Spanish short version of the Utrecht Work Engagement Scale (UWES–9) and assess its predictive validity with respect to self-assessed work performance. A total of 229 employees from educational institutions in Ecuador participated. Using a model comparison analysis, the unidimensional model exhibited an excellent goodness of fit, χ2 = 26.176 (24), p = .344; CFI =1.000; TLI = 1.000; RMSEA = .020; SRMR = .034; it was not improved by more complex models, Three-factor model: χ2 = 22.148 (21), p = .391; CFI =1.000; TLI = 1.000; RMSEA = .016; SRMR = .033. Two-factor model: χ2 = 26.080 (23), p = .297; CFI = 1.000; TLI = 1.000; RMSEA = .025; SRMR = .034). Therefore, it is justified as a unidimensional instrument of work engagement. However, upon analyzing the correlation patterns of the overall score and the work engagement dimensions in relation to the task performance, contextual performance, and counterproductive behaviors, we conclude that, while the unidimensional model exhibits a good fit, the three-factor theoretical approach is substantively superior in that it maintains differential predictive validity for each theoretical dimension.

Delay discounting paradigms have gained widespread popularity across clinical research. Given the prevalence in the field, researchers have set lofty expectations for the importance of delay discounting as a key transdiagnostic process and a ‘core’ process underlying specific domains of dysfunction (e.g. addiction). We believe delay discounting has been prematurely reified as, in and of itself, a core process underlying psychological dysfunction, despite significant concerns with the construct validity of discounting rates. Specifically, high delay discounting rates are only modestly related to measures of psychological dysfunction and therefore are not ‘core’ to these more complex behavioral problems. Furthermore, discounting rates do not appear to be specifically related to any disorder(s) or dimension(s) of psychopathology. This raises fundamental concerns about the utility of discounting, if the measure is only loosely associated with most forms of psychopathology. This stands in striking contrast to claims that discounting can serve as a ‘marker’ for specific disorders, despite never demonstrating adequate sensitivity or specificity for any disorder that we are aware of. Finally, empirical evidence does not support the generalizability of discounting rates to other decisions made either in the lab or in the real-world, and therefore discounting rates cannot and should not serve as a summary measure of an individual's decision-making patterns. We provide recommendations for improving future delay discounting research, but also strongly encourage researchers to consider whether the empirical evidence supports the field's hyper-focus on discounting.

Reliable and valid assessment of sports nutrition knowledge can inform athlete nutrition education to address knowledge gaps. This study aimed to test the reliability and validity of an electronically administered sports nutrition knowledge tool – Platform to Evaluate Athlete Knowledge of Sports Nutrition Questionnaire (PEAKS-NQ). A 94-item PEAKS-NQ was piloted to 149 developmental athletes (DA) in New Zealand, with a subset invited to complete the PEAKS-NQ again to assess reliability. Reliability was evaluated using sign test, intraclass correlation and Cronbach’s α. Accredited sports dietitians (ASD; n 255) completed the PEAKS-NQ to establish construct validity via known-groups methodology and provided relevance scores to determine the scale content validity index (S-CVI). Rasch analysis was conducted to identify potentially problematic items and test reliability. Score differences between DA and ASD were analysed using independent t or non-parametric tests. DA (n 88) were 17·8 (sd 1·4) years, 61·4 % female and mostly in high school (94·3 %). ASD (n 45) were 37·8 (sd 7·6) years, 82·2 % female, with >5 years of dietetic experience (59·1 %). ASD scored higher than DA in all sections and overall (91·5 (sd 3·4) v. 67·1 (sd 10·5) %) (P < 0·001). There were no differences between retests (n 18; P = 0·14). Cronbach’s α was 0·86. S-CVI indicated good content validity (0·88). Rasch analysis resulted in a fifty-item PEAKS-NQ with high item (0·91) and person (0·92) reliability. The PEAKS-NQ is reliable and valid for assessing sports nutrition knowledge which could assist practitioners effectively tailor and evaluate nutrition education.