BACKGROUND: Although the importance of evaluating implementation fidelity is acknowledged, little is known about heterogeneity in fidelity over time. This study aims to generate insight into the heterogeneity in implementation fidelity trajectories of a health promotion program in multidisciplinary settings and the relationship with changes in patients' health behavior.METHODS: This study used longitudinal data from the nationwide implementation of an evidence-informed physical activity promotion program in Dutch rehabilitation care. Fidelity scores were calculated based on annual surveys filled in by involved professionals (n = ± 70). Higher fidelity scores indicate a more complete implementation of the program's core components. A hierarchical cluster analysis was conducted on the implementation fidelity scores of 17 organizations at three different time points. Quantitative and qualitative data were used to explore organizational and professional differences between identified trajectories. Regression analyses were conducted to determine differences in patient outcomes.RESULTS: Three trajectories were identified as the following: 'stable high fidelity' (n = 9), 'moderate and improving fidelity' (n = 6), and 'unstable fidelity' (n = 2). The stable high fidelity organizations were generally smaller, started earlier, and implemented the program in a more structured way compared to moderate and improving fidelity organizations. At the implementation period's start and end, support from physicians and physiotherapists, professionals' appreciation, and program compatibility were rated more positively by professionals working in stable high fidelity organizations as compared to the moderate and improving fidelity organizations (p < .05). Qualitative data showed that the stable high fidelity organizations had often an explicit vision and strategy about the implementation of the program. Intriguingly, the trajectories were not associated with patients' self-reported physical activity outcomes (adjusted model β = - 651.6, t(613) = - 1032, p = .303).CONCLUSIONS: Differences in organizational-level implementation fidelity trajectories did not result in outcome differences at patient-level. This suggests that an effective implementation fidelity trajectory is contingent on the local organization's conditions. More specifically, achieving stable high implementation fidelity required the management of tensions: realizing a localized change vision, while safeguarding the program's standardized core components and engaging the scarce physicians throughout the process. When scaling up evidence-informed health promotion programs, we propose to tailor the management of implementation tensions to local organizations' starting position, size, and circumstances.TRIAL REGISTRATION: The Netherlands National Trial Register NTR3961 . Registered 18 April 2013.
LINK
Risk assessment instruments are widely used to predict risk of adverse outcomes, such as violence or victimization, and to allocate resources for managing these risks among individuals involved in criminal justice and forensic mental health services. For risk assessment instruments to reach their full potential, they must be implemented with fidelity. A lack of information on administration fidelity hinders transparency about the implementation quality, as well as the interpretation of negative or inconclusive findings from predictive validity studies. The present study focuses on adherence, a dimension of fidelity. Adherence denotes the extent to which the risk assessment is completed according to the instrument’s guidelines. We developed an adherence measure, tailored to the ShortTerm Assessment of Risk and Treatability: Adolescent Version (START:AV), an evidence-based risk assessment instrument for adolescents. With the START:AV Adherence Rating Scale, we explored the degree to which 11 key features of the instrument were adhered to in 306 START:AVs forms, completed by 17 different evaluators in a Dutch residential youth care facility over a two-year period. Good to excellent interrater reliability was found for all adherence items. We identified differences in adherence scores on the various START:AV features, as well as significant improvement in adherence for those who attended a START:AV refresher workshop. Outcomes of risk assessment instruments potentially impact decision-making, for example, whether a youth’s secure placement should be extended. Therefore, we recommend fidelity monitoring to ensure the risk assessment practice was delivered as intended.
DOCUMENT
This article describes a measure developed to assess fidelity of working with the Boston University approach to Psychiatric Rehabilitation (BPR) in Dutch mental health care. The instrument is intended to measure and improve BPR adherence and clinician competence on an individual level and within individual rehabilitation processes. https://www.ncbi.nlm.nih.gov/pubmed/28771017
DOCUMENT
The TOP program is a fully implemented responsive parenting intervention for very preterm born infants. Fidelity monitoring of interventions is important for preserving program adherence, impact outcomes and to make evidence-based adaptations. The aim of this study was to develop a fidelity tool for the TOP program following an iterative and co-creative process and subsequently evaluate the reliability of the tool. Three consecutive phases were carried out. Phase I: Initial development and pilot testing two methods namely self-report and video based observation. Phase II: Adaptations and refinements. Phase III: Evaluation of the psychometric properties of the tool based on 20 intervention videos rated by three experts.The interrater reliability of the adherence and competence subscales was good (ICC.81 to .84) and varied from moderate to excellent for specific items (ICC between .51 and .98). The FITT displayed a high correlation (Spearman’s rho.79 to.82) between the subscales and total impression item. The co-creative and iterative process resulted in a clinical useful and reliable tool for evaluating fidelity in the TOP program. This study offers insights in the practical steps in the development of a fidelity assessment tool which can be used by other intervention developers.
MULTIFILE
This systematic review evaluates the implementation of treatment integrity procedures in outcome studies of youth interventions targeting behavioral problems. The Implementation of Treatment Integrity Procedures Scale (ITIPS), developed by Perepletchikova, Treat, and Kazdin (2007), was adapted (ITIPS-A) and used to evaluate 32 outcome studies of evidence-based interventions for youths with externalizing behavioral problems. Integrity measures were found to be still rare in these studies. Of the studies that took integrity into account, 80% approached adequacy in implementing procedures for treatment integrity. The ITIPS-A is recommended as an instrument to guide development of integrity instruments and the implementation of treatment integrity procedures in youth care.
DOCUMENT
Background: Multimodal prehabilitation programs are effective at reducing complications after colorectal surgery in patients with a high risk of postoperative complications due to low aerobic capacity and/or malnutrition. However, high implementation fidelity is needed to achieve these effects in real-life practice. This study aimed to investigate the implementation fidelity of an evidence-based prehabilitation program in the real-life context of a Dutch regional hospital.Methods: In this observational cohort study with multiple case analyses, all patients who underwent colorectal surgery from January 2023 to June 2023 were enrolled. Patients meeting the criteria for low aerobic capacity or malnutrition were advised to participate in a prehabilitation program. According to recent scientific insights and the local care context, this program consisted of four exercise modalities and three nutrition modalities. Implementation fidelity was investigated by evaluating: (1) coverage (participation rate), (2) duration (number of days between the start of prehabilitation and surgery), (3) content (delivery of prescribed intervention modalities), and (4) frequency (attendance of sessions and compliance with prescribed parameters). An aggregated percentage of content and frequency was calculated to determine overall adherence.Results: Fifty-eight patients intended to follow the prehabilitation care pathway, of which 41 performed a preoperative risk assessment (coverage 80%). Ten patients (24%) were identified as high-risk and participated in the prehabilitation program (duration of 33-84 days). Adherence was high (84-100%) in five and moderate (72-73%) in two patients. Adherence was remarkably low (25%, 53%, 54%) in three patients who struggled to execute the prehabilitation program due to multiple physical and cognitive impairments.Conclusion: Implementation fidelity of an evidence-based multimodal prehabilitation program for high-risk patients preparing for colorectal surgery in real-life practice was moderate because adherence was high for most patients, but low for some patients. Patients with low adherence had multiple impairments, with consequences for their preparation for surgery. For healthcare professionals, it is recommended to pay attention to high-risk patients with multiple impairments and further personalize the prehabilitation program. More knowledge about identifying and treating high-risk patients is needed to provide evidence-based recommendations and to obtain higher effectiveness.
LINK
Abstract Summary: Program fidelity instruments are a key ingredient for clinical supervision and implementation as well as effectiveness studies. This study examines the factor structure of the Functional Family Parole services Global Rating Measure (FFP-GRM); the program fidelity instrument of Functional Family Parole services for case management in youth parole, child protection and child welfare services. Between October 2012 and February 2015, program fidelity was measured with the FFP-GRM by Functional Family Parole supervisors. Confirmatory factor analysis was performed on 380 cases and internal consistency reliability coefficients were calculated. Findings: Confirmatory factor analyses showed that the 33-item and four-factor model of the FFP-GRM achieved a good fit to the data. Internal validity testing results showed that subscale Cronbach’s a ranged between .82 and .90. Applications: Findings affirm a good fit to the data and a good-to-excellent internal consistency of the FFP-GRM, which is considered sufficient to justify its use. The results are discussed with regard to the use of fidelity instruments for both clinical and research purposes.
DOCUMENT
Background: Interprofessional collaboration (IPC) among health and social care providers is crucial to effectively implement community-based fall prevention. Several factors hinder successful and sustainable IPC, highlighting the need to both design and evaluate context-specific implementation strategies. However, there remains a fundamental gap in the detailed description and evaluation of such strategies. Therefore, this study aims to (1) monitor the implementation process over time and (2) evaluate the impact of a multifaceted implementation strategy aimed at improving interprofessional collaboration among health and social care professionals in community-based fall prevention. Methods: This study was conducted in two districts and one municipality in the Netherlands. We conducted a longitudinal mixed-methods study with a convergent design, emphasizing qualitative methodology. Over 24 months, qualitative (focus groups and regular meetings) and quantitative (questionnaires) data were collected semi-annually from three working groups of health and social care professionals (HSCPs). Qualitative and quantitative data were initially analyzed separately, followed by an integrated analysis for comprehensive insights on themes influencing the implementation process and the impact of the strategy on IPC and implementation outcomes. Results: In total, 32 HSCPs originating from three communities participated in this study. Monitoring and evaluation of the multifaceted implementation strategy revealed four overarching themes: (1) “Network building”, including aspects and activities that contribute to network building; (2) “Team dynamics”, referring to interactions within the working groups; (3) “Coordination”, addressing the coordination of implementation and establishment of protocols and work flows; and (4) “Implementation dynamics” highlighting aspects that influence the implementation process and outcomes. Conclusions This study identified four key themes influencing the implementation process and impact of a multifaceted implementation strategy aimed at improving IPC among HSCPs in community-based fall prevention: network building, team dynamics, coordination and implementation dynamics. Monitoring and evaluation are crucial.
MULTIFILE
Abstract Background: Several interventions have been developed to improve physical health and lifestyle behaviour of people with a severe mental illness (SMI). Recently, we conducted a pragmatic cluster-randomised controlled trial which evaluated the efects of the one-year Severe Mental Illness Lifestyle Evaluation (SMILE) lifestyle intervention compared with usual care in clients with SMI. The SMILE intervention is a 12-month group-based lifestyle intervention with a focus on increased physical activity and healthy food intake. The aim of the current study was to explore the experiences of people with SMI and healthcare professionals (HCPs) regarding implementation feasibility of the SMILE intervention and the fdelity to the SMILE intervention. Methods: A process evaluation was conducted alongside the pragmatic randomized controlled trial. The experiences of clients and HCPs in the lifestyle intervention group were studied. First, descriptive data on the implementation of the intervention were collected. Next, semi-structured interviews with clients (n=15) and HCPs (n=13) were performed. Interviews were audiotaped and transcribed verbatim. A thematic analysis of the interview data was performed using MAXQDA software. In addition, observations of group sessions were performed to determine the fdelity to the SMILE intervention using a standardised form. Results: Ten out of 26 HCPs who conducted the group sessions discontinued their involvement with the intervention, primarily due to changing jobs. 98% of all planned group sessions were performed. Four main themes emerged from the interviews: 1) Positive appraisal of the SMILE intervention, 2) Suggestions for improvement of the SMILE intervention 3) Facilitators of implementation and 4) Barriers of implementation. Both clients and HCPs had positive experiences regarding the SMILE intervention. Clients found the intervention useful and informative. The intervention was found suitable and interesting for all people with SMI, though HCPs sometimes had to tailor the intervention to individual characteristics of patients (e.g., with respect to cognitive functioning). The handbook of the SMILE intervention was perceived as user-friendly and helpful by HCPs. Combining SMILE with daily tasks, no support from other team members, and lack of staf and time were experienced as barriers for the delivery of the intervention Conclusion: The SMILE intervention was feasible and well-perceived by clients and HCPs. However, we also identifed some aspects that may have hindered efective implementation and needs to be considered when implementing the SMILE intervention in daily practice
DOCUMENT
Objective: To evaluate the implementation of a multicomponent lifestyle intervention at two different worksites. Methods: Data on eight process components were collected by means of questionnaires and interviews. Data on the effectiveness were collected using questionnaires. Results: The program was implemented partly as planned, and 84.0% (max 25) and 85.7% (max 14) of all planned interventions were delivered at the university and hospital, respectively. Employees showed high reach (96.6%) and overall participation (75.1%) but moderate overall satisfaction rates (6.8 ± 1.1). Significant intervention effects were found for days of fruit consumption (β = 0.44 days/week, 95% CI: 0.02 to 0.85) in favor of the intervention group. Conclusions: The study showed successful reach, dose, and maintenance but moderate fidelity and satisfaction. Mainly relatively simple and easily implemented interventions were chosen, which were effective only in improving employees’ days of fruit consumption.
DOCUMENT
