There is emerging evidence that the performance of risk assessment instruments is weaker when used for clinical decision‐making than for research purposes. For instance, research has found lower agreement between evaluators when the risk assessments are conducted during routine practice. We examined the field interrater reliability of the Short‐Term Assessment of Risk and Treatability: Adolescent Version (START:AV). Clinicians in a Dutch secure youth care facility completed START:AV assessments as part of the treatment routine. Consistent with previous literature, interrater reliability of the items and total scores was lower than previously reported in non‐field studies. Nevertheless, moderate to good interrater reliability was found for final risk judgments on most adverse outcomes. Field studies provide insights into the actual performance of structured risk assessment in real‐world settings, exposing factors that affect reliability. This information is relevant for those who wish to implement structured risk assessment with a level of reliability that is defensible considering the high stakes.
DOCUMENT
Risk assessment plays an important role in forensic mental health care. The way the conclusions of those risk assessments are communicated varies considerably across instruments. In an effort to make them more comparable, Hanson, R. K., Bourgon, G., McGrath, R., Kroner, D. D., Amora, D. A., Thomas, S. S., & Tavarez, L. P. [2017. A five-level risk and needs system: Maximizing assessment results in corrections through the development of a common language. The Council of State Governments Justice Center. https:// csgjusticecenter.org/wp-content/uploads/2017/01/A-Five-Level-Risk-and-Needs-system_Report.pdf] developed the Five-Level Risk and Needs System, placing the conclusions of different instruments along five theoretically meaningful levels. The current study explores a Five-Level Risk and Needs system for violent recidivism to which the numerical codings of the HCR-20 Version 2 and its successor, the HCR-20V3 are calibrated, using a combined sample from six previous studies for the HCR-20 Version 2 (n = 411 males with a violent index offence) and a pilot sample for the HCR-20V3 (n = 66 males with a violent index offence). Baselines for the five levels were defined by a combination of theoretical (e.g. expert meetings) and empirical (e.g. literature review) considerations. The calibration of the HCR-20 Version 2 was able to detect four levels, from a combined level I/II to an adjusted level V. The provisional calibration of the HCR-20V3 showed a substantial overlap with the HCR-20 Version 2, with each level boundary having a 2-point difference. Implications for practice and future research are discussed.
DOCUMENT
Risk assessment instruments are widely used to predict risk of adverse outcomes, such as violence or victimization, and to allocate resources for managing these risks among individuals involved in criminal justice and forensic mental health services. For risk assessment instruments to reach their full potential, they must be implemented with fidelity. A lack of information on administration fidelity hinders transparency about the implementation quality, as well as the interpretation of negative or inconclusive findings from predictive validity studies. The present study focuses on adherence, a dimension of fidelity. Adherence denotes the extent to which the risk assessment is completed according to the instrument’s guidelines. We developed an adherence measure, tailored to the ShortTerm Assessment of Risk and Treatability: Adolescent Version (START:AV), an evidence-based risk assessment instrument for adolescents. With the START:AV Adherence Rating Scale, we explored the degree to which 11 key features of the instrument were adhered to in 306 START:AVs forms, completed by 17 different evaluators in a Dutch residential youth care facility over a two-year period. Good to excellent interrater reliability was found for all adherence items. We identified differences in adherence scores on the various START:AV features, as well as significant improvement in adherence for those who attended a START:AV refresher workshop. Outcomes of risk assessment instruments potentially impact decision-making, for example, whether a youth’s secure placement should be extended. Therefore, we recommend fidelity monitoring to ensure the risk assessment practice was delivered as intended.
DOCUMENT
