The NIH Toolbox, for the assessment of neurological and behavioral function, is a multidimensional set of brief royalty-free measures that researchers and clinicians can use to assess cognitive, sensory, motor and emotional function in people ages 3–85. This suite of measures can be administered to study participants in two hours or less, in a variety of settings, with a particular emphasis on measuring outcomes in longitudinal epidemiologic studies and prevention or intervention trials. The battery has been normed and validated across the lifespan in subjects age 3-85 and its use ensures that assessment methods and results can be used for comparisons across existing and future studies. The NIH Toolbox is capable of monitoring neurological and behavioral function over time, and measuring key constructs across developmental stages.[1][2]
In 2004, the 15 Institutes, Centers and Offices at the National Institutes of Health which support neuroscience research formed a coalition, the NIH Blueprint for Neuroscience Research, whose goal is to develop new tools, resources, and training opportunities to accelerate the pace of discovery in neuroscience research. Because the research community had long sought the development of standard instruments to measure cognitive and emotional health, in 2006 the NIH Blueprint awarded a contract to develop an innovative approach to meet this need. Under the leadership of principal investigator Richard C. Gershon, a team of more than 300 scientists from nearly 100 academic institutions were charged with developing a set of tools to enhance data collection in large cohort studies and to advance the neurobehavioral research enterprise.[3][4]
The NIH Toolbox divides tests into four aspects of neural function, called "domain batteries":
Prior to the NIH Toolbox, there were many studies that collected information on aspects of neural function with little uniformity among the measures used to capture these constructs.[5] Moreover, capturing information on all four domains within a study would be costly in terms of time and subject burden. Custom measures could not easily be compared across studies, and assessments were typically limited to looking at cognitive variables. Expensive equipment and per-subject royalty fees were often required. Time-consuming measures usually required highly trained administrators.
With the NIH Toolbox, researchers can assess function using a common metric and can “crosswalk” among measures, supporting the pooling and sharing of large data sets. The NIH Toolbox will support scientific discovery by bringing a common language to research questions – both with respect to the primary study aims and to those arising from secondary data analyses. The four batteries provide researchers with measures that have minimal subject burden and cost.[3] The NIH Toolbox battery of measures will be used by The Human Connectome Project (HCP) to understand the relationship between brain connectivity and behavior,[6] Standardized measures are easily compared across studies. Measures are validated against “gold standard” instruments and easily incorporate multiple areas of neurological functioning. NIH Toolbox requires inexpensive equipment, no royalties, low per-subject costs (per-subject costs limited to taste and olfaction assessments). NIH Toolbox offers brief, psychometrically sound measures that can be administered with minimal expertise.
Initial literature and database reviews and a Request for Information of NIH-funded researchers identified the sub-domains for inclusion in the NIH Toolbox, existing measures relevant to the project goals, and criteria for instrument selection. NIH Project Team members, external content experts, and contract scientists met at a follow-up consensus meeting to discuss potential sub-domains along with the criteria affecting instrument selection, creation, and norming. Additional expert interviews were undertaken to gather more detailed information from clinical and scientific experts to help further refine the list of possible sub-domains. A second consensus group meeting was held and results directed the selection of the sub-domains within each core domain area to be measured in the final NIH Toolbox.
More than 1,400 existing measures were identified and evaluated for inclusion in the NIH Toolbox. The selection criteria included a measure’s applicability across the life span, psychometric soundness, brevity, ease of use, applicability in diverse settings and with different groups, and lack of intellectual property constraints. There was also a preference for instruments that were already validated and normed for use with individuals between 3 and 85 years old. Results of the instrument selection process greatly facilitated the drafting of plans to develop the NIH Toolbox measures.
Validation studies were conducted for all NIH Toolbox measures, to assure that these important tools for research met rigorous scientific standards. Studies were conducted across the entire age range, typically included 450-500 subjects, and were statistically compared against “gold standard” measures wherever available.[7][8] For tests using item response theory approaches to scoring, calibration samples generally included several thousand participants, ensuring robust models. In total, data was collected from more than 16,000 subjects as part of field-test, calibration and validation activities.[9]
NIH Toolbox conducted a national standardization study in both English and Spanish languages to allow for normative comparisons on each assessment. A sample of 4,859 participants, ages 3–85 – representative of the U.S. population based on gender, race/ethnicity, and socioeconomic status – was administered all of the NIH Toolbox measures at sites around the country. NIH Toolbox normative scores are now available for each year of age from 3 through 17, as well as for ages 18–29, 30-39, 40-49, 50-59, 60-69, 70-79, and 80-85, allowing for targeted, accurate comparisons for any research study participant groups against the U.S. population.
The NIH Toolbox measures utilize several advanced approaches in item development, test construction, and scoring. Two of these are item response theory and computer adaptive testing (CAT).[10][11] Item Response Theory allows tests to be brief, yet still precise and valid.[12] Using IRT methodology, sets of items are calibrated along a continuum that covers the full range of the construct to be measured. This calibrated set of items enables the creation of CAT, a specialized type of computer-based testing that enables frequent assessments and immediate feedback with minimal burden on participants and precise evaluation at the individual level.[10]
NIH Toolbox measure development focused special attention on assessing young children,[13] to ensure that all tests given are developmentally appropriate for ages 3–7. A special team of early childhood assessment consultants was engaged to provide testing guidelines for the very young, to offer input on measure development, and to review all NIH Toolbox measures to ensure they fit the needs of young children.[14][15]
NIH Toolbox measures are administered using Assessment Center, a free, browser-based research management software application where users can access, practice, and then administer NIH Toolbox measures. Assessment Center enables researchers to create study-specific websites for capturing participant data securely. Studies can include measures within the Assessment Center library as well as custom measures created or entered by the researcher.[11][16][17]
As Assessment Center is no longer available, the toolbox has transitioned into an iPad app, available via the App Store. The iPad version of the toolbox requires a years subscription.