There is emerging evidence that the performance of risk assessment instruments is weaker when used for clinical decision‐making than for research purposes. For instance, research has found lower agreement between evaluators when the risk assessments are conducted during routine practice. We examined the field interrater reliability of the Short‐Term Assessment of Risk and Treatability: Adolescent Version (START:AV). Clinicians in a Dutch secure youth care facility completed START:AV assessments as part of the treatment routine. Consistent with previous literature, interrater reliability of the items and total scores was lower than previously reported in non‐field studies. Nevertheless, moderate to good interrater reliability was found for final risk judgments on most adverse outcomes. Field studies provide insights into the actual performance of structured risk assessment in real‐world settings, exposing factors that affect reliability. This information is relevant for those who wish to implement structured risk assessment with a level of reliability that is defensible considering the high stakes.
The Kennedy Axis V is a routine outcome measurement instrument which can assist the assessment of the short-term risk for violence and other adverse patient outcomes. The purpose of this study was to evaluate the interrater reliability and clinical utility of the instrument when used by mental health nurses in daily care of patients with mental illness. This cross-sectional study was conducted in inpatient and outpatient adult psychiatric care units and in one adolescent inpatient unit at a university hospital in the Netherlands. Interrater reliability was measured based on the independent scores of two different nurses for the same patients. The clinical utility of the instrument was evaluated by means of a clinical utility questionnaire. To gain a deeper understanding of rating difficulties at the adolescent unit, additional data were collected in two focus group interviews. The overall results revealed a substantial level of agreement between nurses (intraclass correlation coefficient and Pearson 0.79). Some rating challenges were identified, including difficulties with scoring the instrument and using tailor-made interventions related to the scores. These challenges can be resolved using refined training and implementation strategies. When the Kennedy Axis V is accompanied by a solid implementation strategy in adult mental health care, the instrument can be used for short-term risk assessment and thereby contribute in efforts to reduce violence, suicide, self-harm, severe selfneglect, and enhanced objectivity in clinical decision-making.
