Hostname: page-component-669899f699-cf6xr Total loading time: 0 Render date: 2025-04-27T07:12:48.764Z Has data issue: false hasContentIssue false
  • English
  • Français

Validation of the NIH Toolbox-Cognition Battery against legacy neurocognitive measures in adults with cognitive impairments: An exploratory analysis

Published online by Cambridge University Press:  05 September 2022

Emily A. Kringle Open the ORCID record for Emily A. Kringle [Opens in a new window] ,
Enrico M. Novelli ,
Meryl A. Butters  and
Elizabeth R. Skidmore
Emily A. Kringle*
Affiliation:
University of Illinois at Chicago, Department of Medicine, Chicago, USA
Enrico M. Novelli
Affiliation:
University of Pittsburgh, Department of Medicine, Pittsburgh, USA
Meryl A. Butters
Affiliation:
University of Pittsburgh, Department of Psychiatry, Pittsburgh, USA
Elizabeth R. Skidmore
Affiliation:
University of Pittsburgh, Department of Occupational Therapy, Pittsburgh, USA
*
Corresponding author: Emily A. Kringle, email: kringle@uic.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Objective:

The purpose of this exploratory study was to describe associations between NIH Toolbox-Cognition Battery subtests and legacy measures of neurocognitive function in two samples with neurological conditions (stroke and sickle cell disease (SCD)).

Method:

This exploratory secondary analysis uses data from two studies that assessed cognition at one time point using the NIH Toolbox-Cognition Battery, the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), and subtests from the Delis-Kaplan Executive Functions System (DKEFS). People with stroke (n = 26) and SCD (n = 64) were included. Associations between the NIH Toolbox-Cognition Battery subtests and corresponding legacy measures were examined using linear correlations, Bland–Altman analysis, and Lin’s Concordance Correlation Coefficient.

Results:

Linear correlations and Lin’s Concordance Correlation Coefficient were poor to strong in both samples on NIH Toolbox-CB subtests: Flanker Inhibitory Control and Attention (r = .35 to .48, Lin CCC = .27 to .37), Pattern Comparison Processing Speed (r = .40 to .65, Lin CCC = .37 to .62), Picture Sequence Memory (r = .19 to .55, Lin CCC = .18 to .48), Dimensional Change Card Sort (r = .39 to .77, Lin CCC = .38 to .63), Fluid Cognition Composite (r = .88 to .90, Lin CCC = .60 to .79), and Total Cognition Composite (r = .64 to .83, Lin CCC = .60 to .78). Bland–Altman analyses demonstrated wide limits of agreement across all subtests (–3.17 to 3.78).

Conclusions:

The NIH Toolbox-Cognition Battery subtests may behave similarly to legacy measures as an overall assessment of cognition across samples at risk for neurological impairment. Findings should be replicated across additional clinical samples.

Keywords

cognitionmemoryperceptionlanguagecomprehensionexecutive functionanemiasickle cellstroke
Type
Research Article
Information
Journal of the International Neuropsychological Society , Volume 29 , Issue 5 , June 2023 , pp. 472 - 479
Copyright
Copyright © INS. Published by Cambridge University Press, 2022

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Akoglu, H. (2018). User’s guide to correlation coefficients. Turkish Journal of Emergency Medicine, 18, 91–93.CrossRefGoogle ScholarPubMed
Altman, D. (1991). Practical statistics for medical research. Chapman and Hall.Google Scholar
Ballas, S. K. (2012). More definitions in sickle cell disease: Steady state versus base line data. American Journal of Hematology, 87, 338–338.CrossRefGoogle Scholar
Barnhart, H. X., Haber, M. J., & Lin, L. I. (2007). An overview on assessing agreement with continuous measurements. Journal of Biopharmaceutical Statistics, 17, 529–569.CrossRefGoogle ScholarPubMed
Carlozzi, N., Beaumont, J. L., Tulsky, D. S., & Gershon, R. C. (2015). The NIH toolbox pattern comparison processing speed test: Normative data. Archives of Clinical Neuropsychology, 30, 359–368.CrossRefGoogle ScholarPubMed
Carlozzi, N., Goodnight, S., Casaletto, K., Goldsmith, A., Heaton, R., Wong, A., Baum, C. M., Gershon, R., & Tulsky, D. (2017a). Validation of the NIH toolbox in individuals with neurologic disorders. Archives of Clinical Neuropsychology, 32, 555–573.CrossRefGoogle ScholarPubMed
Carlozzi, N., Tulsky, D. S., Wolf, T. J., Goodnight, S., Heaton, R. K., Casaletto, K. B., Wong, A. W. K., Baum, C. M., Gershon, R. C., & Heinemann, A. W. (2017b). Construct validity of the NIH Toolbox Cognition Battery in individuals with stroke. Rehabilitation Psychology, 62, 443.CrossRefGoogle ScholarPubMed
Casaletto, K. B., Umlauf, A., Beaumont, J., Gershon, R., Slotkin, J., Akshoomoff, N., & Heaton, R. K. (2015). Demographically corrected normative standards for the English version of the NIH Toolbox Cognition Battery. Journal of the International Neuropsychological Society, 21, 378–391.CrossRefGoogle ScholarPubMed
Chakravorty, S., & Williams, T. N. (2015). Sickle cell disease: A neglected chronic disease of increasing global health importance. Archives of Disease in Childhood, 100, 48–53.CrossRefGoogle ScholarPubMed
Ciafone, J., Little, B., Thomas, A. J., & Gallagher, P. (2020). The neuropsychological profile of mild cognitive impairment in Lewy body dementias. Journal of the International Neuropsychological Society, 26, 210–225.CrossRefGoogle ScholarPubMed
Delis, D. C., Jacobson, M., Bondi, M. W., Hamilton, J. M., & Salmon, D. P. (2003). The myth of testing construct validity using factor analysis or correlations with normal or mixed clinical populations: Lessons from memory assessment; Testing construct validity; DC Delis et al. Journal of the International Neuropsychological Society: JINS, 9, 936.CrossRefGoogle ScholarPubMed
Delis, D. C., Kaplan, E., & Kramer, J. H. (2001). Delis-Kaplan executive function system. Psychological Corporation.Google Scholar
Dikmen, S. S., Bauer, P. J., Weintraub, S., Mungas, D., Slotkin, J., Beaumont, J. L., Gershon, R., Temkin, R. K., & Heaton, R. K. (2014). Measuring episodic memory across the lifespan: NIH toolbox picture sequence memory test. Journal of the International Neuropsychological Society, 20, 611–619.CrossRefGoogle ScholarPubMed
Duncan, F., Wu, S., & Mead, G. E. (2012). Frequency and natural history of fatigue after stroke: A systematic review of longitudinal studies. Journal of Psychosomatic Research, 73, 18–27.CrossRefGoogle ScholarPubMed
Edwards, C. L., Raynor, R. D., Feliu, M., McDougald, C., Johnson, S., Schmechel, D., Wood, M., Bennett, G. G., Saurona, P., Bonner, M., Wellington, C., DeCastro, L. M., Whitworth, E., Abrams, M., Logue, P., Edwards, L., Martinez, S., & Whitfied, K. (2007). Neuropsychological assessment, neuroimaging, and neuropsychiatric evaluation in pediatric and adult patients with sickle cell disease (SCD). Neuropsychiatric Disease and Treatment, 3, 705.CrossRefGoogle ScholarPubMed
Gershon, R. C., Cook, K. F., Mungas, D., Manly, J. J., Slotkin, J., Beaumont, J. L., & Weintraub, S. (2014). Language measures of the NIH Toolbox Cognition Battery. Journal of the International Neuropsychological Society, 20, 642–651.CrossRefGoogle ScholarPubMed
Giavarina, D. (2015). Understanding bland Altman analysis. Biochemia Medica: Biochemia Medica, 25, 141–151.CrossRefGoogle ScholarPubMed
Hackett, K., Krikorian, R., Giovannetti, T., Melendez-Cabrero, J., Rahman, A., Caesar, E. E., Chen, J. L., Hristov, H., Seifan, A., Mosconi, L., & Isaacson, R. S. (2018). Utility of the NIH Toolbox for assessment of prodromal Alzheimer’s disease and dementia. Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring, 10, 764–772.Google ScholarPubMed
Heaton, R. K., Akshoomoff, N., Tulsky, D., Mungas, D., Weintraub, S., Dikmen, S., Beaumont, J., Casaletto, K. B., Conway, K., Slotkin, J., & Gershon, R. (2014). Reliability and validity of composite scores from the NIH Toolbox Cognition Battery in adults. Journal of the International Neuropsychological Society, 20, 588–598.CrossRefGoogle ScholarPubMed
IBM Corp. (2020). IBM SPSS Statistics for Windows, Version 127.0. Armonk, NY: IBM Corp.Google Scholar
Jean, K. R., Lindbergh, C. A., Mewborn, C. M., Robinson, T. L., Gogniat, M. A., & Miller, L. S. (2019). Education differentially buffers cognitive performance in black and white older adults. The Journals of Gerontology: Series B, 74, 1366–1375.CrossRefGoogle ScholarPubMed
Jorgensen, D. R., Metti, A., Butters, M. A., Mettenburg, J. M., Rosano, C., & Novelli, E. M. (2017). Disease severity and slower psychomotor speed in adults with sickle cell disease. Blood Advances, 1, 1790–1795.CrossRefGoogle ScholarPubMed
Karr, J. E., Hofer, S. M., Iverson, G. L., & Garcia-Barrera, M. A. (2019). Examining the latent structure of the Delis–Kaplan executive function system. Archives of Clinical Neuropsychology, 34, 381–394.CrossRefGoogle ScholarPubMed
Kassim, A. A., Pruthi, S., Day, M., Rodegheier, M., Gindville, M. C., Brodsky, M. A., DeBaun, M. R., & Jordan, L. C. (2016). Silent cerebral infarcts and cerebral aneurysms are prevalent in adults with sickle cell anemia. Blood, 127, 2038–2040.CrossRefGoogle ScholarPubMed
Kirkham, F. J., & Datta, A. K. (2006). Hypoxic adaptation during development: Relation to pattern of neurological presentation and cognitive disability. Developmental Science, 9, 411–427.CrossRefGoogle ScholarPubMed
Kroenke, K., Spitzer, R. L., & Williams, J. B. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16, 606–613.CrossRefGoogle ScholarPubMed
Lawrence, I., & Lin, K. (1989). A concordance correlation coefficient to evaluate reproducibility. Biometrics, 45, 255–268.Google Scholar
Mackin, R. S., Insel, P., Truran, D., Vichinsky, E. P., Neumayr, L. D., Armstrong, F. D., Gold, J. I., Kesler, K., Brewer, J., Weiner, M. W., and the Neuropsychological Dysfunction and Neuroimaging Adult Sickle Cell Anemia Study Group. (2014). Neuroimaging abnormalities in adults with sickle cell anemia: Associations with cognition. Neurology, 82, 835–841.CrossRefGoogle ScholarPubMed
McFarland, D. J. (2020). Factor-analytic evidence for the complexity of the Delis–Kaplan executive function system (D-KEFS). Assessment, 27, 1645–1656.CrossRefGoogle ScholarPubMed
Mole, J. A., & Demeyere, N. (2018). The relationship between early post-stroke cognition and longer term activities and participation: A systematic review. Neuropsychological Rehabilitation, 30, 346–370.CrossRefGoogle ScholarPubMed
Mungas, D., Heaton, R., Tulsky, D., Zelazo, P. D., Slotkin, J., Blitz, D., Lai, J.-S., & Gershon, R. (2014). Factor structure, convergent validity, and discriminant validity of the NIH Toolbox Cognitive Health Battery (NIHTB-CHB) in adults. Journal of the International Neuropsychological Society, 20, 579–587.CrossRefGoogle ScholarPubMed
Pugh, E., Robinson, A., De Vito, A. N., Bernstein, J. P., & Calamia, M. (2021). Representation of US Black Americans in neuropsychology research: How well do our reporting practices show that Black lives matter? The Clinical Neuropsychologist, 36, 214–226.CrossRefGoogle ScholarPubMed
Randolph, C., Tierney, M. C., Mohr, E., & Chase, T. N. (1998). The repeatable battery for the assessment of neuropsychological status (RBANS): Preliminary clinical validity. Journal of Clinical and Experimental Neuropsychology, 20, 310–319.CrossRefGoogle ScholarPubMed
Sanger, M., Jordan, L., Pruthi, S., Day, M., Covert, B., Merriweather, B., Rodeghier, M., DeBaun, M., & Kassim, A. (2016). Cognitive deficits are associated with unemployment in adults with sickle cell anemia. Journal of Clinical and Experimental Neuropsychology, 38, 661–671.CrossRefGoogle ScholarPubMed
Sexton, E., McLoughlin, A., Williams, D. J., Merriman, N. A., Donnelly, N., Rohde, D., Hickey, A., Wren, M.-A., & Bennett, K. (2019). Systematic review and meta-analysis of the prevalence of cognitive impairment no dementia in the first year post-stroke. European Stroke Journal, 4, 160–171.CrossRefGoogle ScholarPubMed
Spearman, C. (1904). “General intelligence,” objectively determined and measured. American Journal of Psychology, 15, 201–292.CrossRefGoogle Scholar
Spitzer, R. L., Williams, J. B., Kroenke, K., Linzer, M., Verloin deGruy, F., Hahn, S. R., Brody, D., & Johnson, J. G. (1994). Utility of a new procedure for diagnosing mental disorders in primary care: The PRIME-MD 1000 study. JAMA, 272, 1749–1756.CrossRefGoogle ScholarPubMed
Tulsky, D. S., Carlozzi, N., Chiaravalloti, N. D., Beaumont, J. L., Kisala, P. A., Mungas, D., Conway, K., & Gershon, R. (2014). NIH Toolbox Cognition Battery (NIHTB-CB): List sorting test to measure working memory. Journal of the International Neuropsychological Society, 20, 599–610.CrossRefGoogle ScholarPubMed
Vichinsky, E. P., Neumayr, L. D., Gold, J. I., Weiner, M. W., Rule, R. R., Truran, D., Kasten, J., Eggleston, B., Kesler, K., McMahon, L., Orringer, E. P., Harrington, T., Kalinyak, K., De Castro, L. M., Kutlar, A., Rutherford, C. J., Johnson, C., Bessman, J. D., Jordan, L. B., Armstrong, F. D., & the Neurospychological Dysfunction and Neuroimaging Adult Sickle Cell Anemia Study Group. (2010). Neuropsychological dysfunction and neuroimaging abnormalities in neurologically intact adults with sickle cell anemia. JAMA, 303, 1823–1831.CrossRefGoogle ScholarPubMed
Watson, P. F. & Petrie, A. (2010). Method agreement analysis: A review of correct methodology. Therogenology, 73, 1167–1179.CrossRefGoogle ScholarPubMed
Weintraub, S., Dikmen, S. S., Heaton, R. K., Tulsky, D. S., Zelazo, P. D., Bauer, P. J., Carlozzi, N. E., Slotkin, J., Blitz, D., Wallner-Allen, K., Fox, N. A., Beaumont, J. L., Mungas, D., Nowinski, C. J., Richler, J., Deocampo, J. A., Anderson, J. E., Manly, J. J., Borosh, B., Havlik, R., Conway, K., Edwards, E., Freund, L., King, J. W., Moy, C., Witt, E., & Gershon, R. C. (2013). Cognition assessment using the NIH Toolbox. Neurology, 80, S54–S64.CrossRefGoogle ScholarPubMed
Wilkinson, G., & Robertson, G. (2006). Wide range achievement test (WRAT4). Psychological Assessment Resources.Google Scholar
Zelazo, P. D., Anderson, J. E., Richler, J., Wallner-Allen, K., Beaumont, J. L., Conway, K. P., Gershon, R., & Weintraub, S. (2014). NIH Toolbox Cognition Battery (CB): Validation of executive function measures in adults. Journal of the International Neuropsychological Society, 20, 620–629.CrossRefGoogle ScholarPubMed
Zhao, L., Biesbroek, J. M., Shi, L., Liu, W., Kuijf, H. J., Chu, W. W., Abrigo, J. M., Lee, R. K. L., Leung, T. W. H., Lau, A. Y., Biessels, G. J., Mok, V., & Wong, A. (2018). Strategic infarct location for post-stroke cognitive impairment: A multivariate lesion-symptom mapping study. Journal of Cerebral Blood Flow & Metabolism, 38, 1299–1311.CrossRefGoogle ScholarPubMed
Supplementary material: PDF

Kringle et al. supplementary material

Kringle et al. supplementary material 1

Download Kringle et al. supplementary material(PDF)
PDF 11.9 KB
Supplementary material: PDF

Kringle et al. supplementary material

Kringle et al. supplementary material 2

Download Kringle et al. supplementary material(PDF)
PDF 149.1 KB