A review of Australian mental health services identified a gap in routine outcome measures addressing social, emotional and behavioural domains for pre-schoolers and infants. A Child and Adolescent Mental Health Information Development Expert Advisory Panel working group developed the Health of the Nation Outcome Scales for Infants (HoNOSI), a clinician-reported routine outcome measure for infants 0–47 months. Prior face validity testing showed that the HoNOSI was considered useful in measuring mental health outcomes.

To examine the concurrent validity of the HoNOSI.

Mental health clinicians providing assessment and treatment to infants in routine clinical practice participated in the study. The mental health status of 108 infants were rated by a minimum of 26 clinicians with the HoNOSI, the Parent-Infant Relationship Global Assessment Scale (PIR-GAS) and measures of symptom severity and distress.

The HoNOSI was statistically significantly correlated with the PIR–;GAS, rs = −0.73; Clinical Worry, rs = 0.77; and Severity Judgement ratings, rs = 0.85; P < 0.001. A good level of internal consistency was found. Using the COsensus-based Standards for the selection of health Measurement INstruments (COSMIN) criteria for judging instrument acceptability, the HoNOSI meets the standard for both concurrent validity and internal consistency.

There has been a clear need for a routine outcome measure for use with infants. This study provides positive evidence of aspects of validity. These findings, along with those from the prior face validity study, support a controlled release of the HoNOSI accompanied by further research and development.

A review of Australian mental health services identified a gap in routine outcome measures addressing social, emotional and behavioural domains for pre-schoolers and infants. The Child and Adolescent Mental Health Information Development Expert Advisory Panel Working Group developed the Health of the Nation Outcome Scales for Infants (HoNOSI), a clinician-reported routine outcome measure for use with those aged under 4 years. Prior psychometric testing showed that the HoNOSI was considered to show face validity, and that it met the standards for concurrent validity and internal consistency.

We aimed to investigate the interrater reliability of the HoNOSI.

Forty-five infant mental health clinicians completed HoNOSI ratings on a set of five case vignettes.

Quadratic weighted kappa interrater reliability estimates showed the HoNOSI to have Almost Perfect interrater reliability for the HoNOSI total score. Of the 15 scales, one had Moderate, seven had Substantial and seven had Almost Perfect interrater reliability. Ten of the fifteen scales and the total score exceeded the COnsensus-based Standards for the Selection of Health Measurement INstruments criteria for interrater reliability (κw ≥ 0.7).

There has been a clear need for a routine outcome measure for use with infants and pre-schoolers. This study provides evidence of interrater reliability. The current findings, combined with the face and concurrent validity studies, support further examination of HoNOSI in real-world settings.