There is emerging evidence that the performance of risk assessment instruments is weaker when used for clinical decision‐making than for research purposes. For instance, research has found lower agreement between evaluators when the risk assessments are conducted during routine practice. We examined the field interrater reliability of the Short‐Term Assessment of Risk and Treatability: Adolescent Version (START:AV). Clinicians in a Dutch secure youth care facility completed START:AV assessments as part of the treatment routine. Consistent with previous literature, interrater reliability of the items and total scores was lower than previously reported in non‐field studies. Nevertheless, moderate to good interrater reliability was found for final risk judgments on most adverse outcomes. Field studies provide insights into the actual performance of structured risk assessment in real‐world settings, exposing factors that affect reliability. This information is relevant for those who wish to implement structured risk assessment with a level of reliability that is defensible considering the high stakes.
DOCUMENT
Caregivers of persons with profound intellectual and multiple disabilities (PIMD) often describe the quality of the daily movements of these persons in terms of flexibility or stiffness. Objective outcome measures for flexibility and stiffness are muscle tone or level of spasticity. Two instruments used to grade muscle tone and spasticity are the Modified Ashworth Scale (MAS) and the Modified Tardieu Scale (MTS). To date, however, no research has been performed to determine the psychometric properties of the MAS and MTS in persons with PIMD. Therefore, the purpose of this study was to determine the feasibility, test-retest reliability, and interrater reliability of the MAS and MTS in persons with PIMD. We assessed 35 participants on the MAS and MTS twice, first for the test and second a week later for the retest. Two observers performed the measurements. Feasibility was assessed based on the percentage of successful measurements. Test-retest and interrater reliability were determined by using the Wilcoxon signed rank test, intraclass correlation coefficients (ICC), Spearman's correlation, and either limits of agreement (LOA) or quadratically weighted kappa. The feasibility of the measurements was good, because an acceptable percentage of successful measurements were performed. MAS measurements had substantial to almost perfect quadratically weighted kappa (>0.8) and an acceptable ICC (>0.8) for both inter- and intrarater reliability. However, MTS measurements had insufficient ICCs, Spearman's correlations, and LOAs for both inter- and interrater reliability. Our data indicated that the feasibility of the MAS and MTS for measuring muscle tone in persons with PIMD was good. The MAS had sufficient test-retest and interrater reliability; however, the MTS had an insufficient test-retest and interrater reliability in persons with PIMD. Thus, the MAS may be a good method for evaluating the quality of daily movements in persons with PIMD. Providing test administrators with training and clear instructions will improve test reliability.
DOCUMENT
The Nociception Coma Scale (NCS) is a pain observation tool, developed for patients with disorders of consciousness (DOC) due to acquired brain injury (ABI). The aim of this study was to assess the interrater reliability of the NCS and NCS-R among nurses for the assessment of pain in ABI patients with DOC. A secondary aim was further validation of both scales by assessing its discriminating abilities for the presence or absence of pain. Hospitalized patients with ABI (n = 10) were recorded on film during three conditions: baseline, after tactile stimulation, and after noxious stimulation. All stimulations were part of daily treatment for these patients. The 30 recordings were assessed with the NCS and NCS-R by 27 nurses from three university hospitals in the Netherlands. Each nurse viewed 9 to 12 recordings, totaling 270 assessments. Interrater reliability of the NCS/NCS-R items and total scores were estimated by intraclass correlations (ICC), which showed excellent and equal average measures reliability for the NCS and NCR-R total scores (ICC 0.95), and item scores (range 0.87-0.95). Secondary analysis was performed to assess differences in ICCs among nurses' education and experience and to assess the scales discriminating properties for the presence of pain. The NCS and NCS-R are valid and reproducible scales that can be used by nurses with an associate (of science) in nursing degree or baccalaureate (of science) in nursing degree. It seems that more experience with ABI patients is not a predictor for good agreement in the assessment of the NCS(-R).
DOCUMENT
