Introduction: In March 2014, the New South Wales (NSW) Government (Australia) announced the NSW Integrated Care Strategy. In response, a family-centred, population-based, integrated care initiative for vulnerable families and their children in Sydney, Australia was developed. The initiative was called Healthy Homes and Neighbourhoods. A realist translational social epidemiology programme of research and collaborative design is at the foundation of its evaluation. Theory and Method: The UK Medical Research Council (MRC) Framework for evaluating complex health interventions was adapted. This has four components, namely 1) development, 2) feasibility/piloting, 3) evaluation and 4) implementation. We adapted the Framework to include: critical realist, theory driven, and continuous improvement approaches. The modified Framework underpins this research and evaluation protocol for Healthy Homes and Neighbourhoods. Discussion: The NSW Health Monitoring and Evaluation Framework did not make provisions for assessment of the programme layers of context, or the effect of programme mechanism at each level. We therefore developed a multilevel approach that uses mixed-method research to examine not only outcomes, but also what is working for whom and why.
LINK
This study explores the evaluation of research pathways of self-management health innovations from discovery to implementation in the context of practice-based research. The aim is to understand how a new process model for evaluating practice-based research provides insights into the implementation success of innovations. Data were collected from nine research projects in the Netherlands. Through document analysis and semi-structured interviews, we analysed how the projects start, evolve, and contribute to the healthcare practice. Building on previous researchevaluation approaches to monitor knowledge utilization, we developed a Research Pathway Model. The model’s process character enables us to include and evaluate the incremental work required throughout the lifespan of an innovation project and it helps to foreground that innovation continues during implementation in real-life settings. We found that in each researchproject, pathways are followed that include activities to explore a new solution, deliver a prototype and contribute to theory. Only three projects explored the solution in real life and included activities to create the necessary changes for the solutions to be adopted. These three projects were associated with successful implementation. The exploration of the solution in a real-life environment in which users test a prototype in their own context seems to be a necessaryresearch activity for the successful implementation of self-management health innovations.
LINK
This paper presents a method for Universities of Applied Sciences (UAS) to account for the impact of research. The 36 UAS in The Netherlands aim to contribute to global challenges and pressing social issues through practice-based research. Given this aim UAS have a strong responsibility to account for the impact of their research and to show that the public research money is well spent. This paper shows that none of the existing methods for assessing the impact of research are suitable for the research conducted at Dutch UAS. It offers an alternative approach based on narratives supported by empirical evidence.
DOCUMENT
The author presents the methodological backgrounds and underlying research design of an on-going scientific research project concerned with the scientific evaluation of serious games and/or computer-based simulation-games (SG) for advanced learning. The main questions of this research project are: 1. what are the requirements and design principles for a comprehensive social-scientific methodology for the evaluation of SG? 2. To what extend does SG contribute to advanced learning? 3. What factors contribute to, or determine this learning? 4. To what extend and under what conditions can SGbased learning be transferred to the real world (RW)? Between 2004 and 2012, several hundreds of SG-sessions in the Netherlands with twelve different SG were evaluated systematically, uniformly and quantitatively to give a data-set of 2100 respondents in higher education and in work-organizations. The author presents the research model, the quasi-experimental design and evaluation instruments. This focus in this article is on methodology and data-set to establish a proper foundation for forthcoming publications on empirical results.
DOCUMENT
In this article, we describe a study on the impact of an ethics program aimed at strengthening the ethical agency of 15 social workers of three welfare organizations. The goal of the study was to make an inventory of the impact of the program, and to evaluate the relevance of this impact with the help of several stakeholders. The most significant change (MSC) approach was used as a research strategy, though some changes to the approach were made with a view to our research goal. We explain the MSC approach and how we used it in our study design. Further, we describe the research process, answering the question whether our adaptation of the MSC was helpful to inventory the impact of our ethics program and the evaluation of its relevance. The implications of MSC's focus on "most significant" changes and the need for a thorough feedback of the results of the evaluation process in the participating organizations are discussed.
MULTIFILE
This chapter provides insights into the complex and adaptive nature of systems and illustrates key characteristics of such systems. These contribute to an understanding of the challenges in health promotion and imply a need for more context-specific research to evaluate the health promotion interventions. CARA can address this need as it can be used to evaluate and support change in complex adaptive systems. To support and inspire other health promotion researchers who want to adopt CARA as their research approach, we have discussed our experiences and provided some guiding principles. Overall, complexity thinking can help to understand the challenges in health promotion, whereby CARA provides a possible strategy for health promotion researchers when dealing with the challenges of evaluating health promotion interventions in complex adaptive systems.
LINK
Aim: To evaluate healthcare professionals' performance and treatment fidelity in the Cardiac Care Bridge (CCB) nurse-coordinated transitional care intervention in older cardiac patients to understand and interpret the study results. Design: A mixed-methods process evaluation based on the Medical Research Council Process Evaluation framework. Methods: Quantitative data on intervention key elements were collected from 153 logbooks of all intervention patients. Qualitative data were collected using semi-structured interviews with 19 CCB professionals (cardiac nurses, community nurses and primary care physical therapists), from June 2017 until October 2018. Qualitative data-analysis is based on thematic analysis and integrated with quantitative key element outcomes. The analysis was blinded to trial outcomes. Fidelity was defined as the level of intervention adherence. Results: The overall intervention fidelity was 67%, ranging from severely low fidelity in the consultation of in-hospital geriatric teams (17%) to maximum fidelity in the comprehensive geriatric assessment (100%). Main themes of influence in the intervention performance that emerged from the interviews are interdisciplinary collaboration, organizational preconditions, confidence in the programme, time management and patient characteristics. In addition to practical issues, the patient's frailty status and limited motivation were barriers to the intervention. Conclusion: Although involved healthcare professionals expressed their confidence in the intervention, the fidelity rate was suboptimal. This could have influenced the non-significant effect of the CCB intervention on the primary composite outcome of readmission and mortality 6 months after randomization. Feasibility of intervention key elements should be reconsidered in relation to experienced barriers and the population. Impact: In addition to insight in effectiveness, insight in intervention fidelity and performance is necessary to understand the mechanism of impact. This study demonstrates that the suboptimal fidelity was subject to a complex interplay of organizational, professionals' and patients' issues. The results support intervention redesign and inform future development of transitional care interventions in older cardiac patients.
DOCUMENT
As most Universities around the world the Amsterdam University of Applied Sciences conduct surveys (student evaluation monitor: STEM) among their students to evaluate the different courses and their teachers. At the Department of Media, Information and Communication the response by students tend to decline in the course of the year. In 2011-2012 with a limited enrolment of 900 first year students, 70% responded to the first survey conducted after the first exams in October and dropped to 26% in the last survey at the end of the first year (July 2012). In 2012-2013 (with the same amount of students) the response was respectively 75% and 30%. This might be due to several factors, such as the length of the questionnaire, the way the survey is spread (via e-mail to the students University account), the time of spreading the surveys (after the courses and exams) or simple due to lack of interest. Another problem of the surveys is found in the quest to limit the length of the questionnaires. Hereby, some relevant aspects to apprehend the success of students (or the return of the department) and the quality of the courses and teachers aren’t measured, such as: coherence between the courses, the students opinion about the form of education and exams, the connection between the evaluation and the exam results or other influential factors of student’s success. Given these difficulties and the fact that insight in all of the above mentioned aspects are crucial for both students and teachers and not in the least for the management, a new approach for evaluating is needed. An evaluating system that can uncover crucial information, for example to pinpoint the characteristics of dropout or long-term students in order to limit these, and/or improve the education/course. This paper will describe a pilot study wherein a first step towards a new way of evaluating is taken by separating the course- and teacher evaluation from the rest of the surveys by using an app/QR or website. Furthermore, the literature about in- or outside class surveys and student success will serve as a theoretical base for the discussion this pilot and is part of a broader PhD research.
LINK
While traditional crime rates are decreasing, cybercrime is on the rise. As a result, the criminal justice system is increasingly dealing with criminals committing cyber-dependent crimes. However, to date there are no effective interventions to prevent recidivism in this type of offenders. Dutch authorities have developed an intervention program, called Hack_Right. Hack_Right is an alternative criminal justice program for young first-offenders of cyber-dependent crimes. In order to prevent recidivism, this program places participants in organizations where they are taught about ethical hacking, complete (technical) assignments and reflect on their offense. In this study, we have evaluated the Hack_Right program and the pilot interventions carried out thus far. By examining the program theory (program evaluation) and implementation of the intervention (process evaluation), the study adds to the scarce literature about cybercrime interventions. During the study, two qualitative research methods have been applied: 1) document analysis and 2) interviews with intervention developers, imposers, implementers and participants. In addition to the observation that the scientific basis for linking specific criminogenic factors to cybercriminals is still fragile, the article concludes that the theoretical base and program integrity of Hack_Right need to be further developed in order to adhere to principles of effective interventions.
DOCUMENT
Posterpresentatie die een introductie geeft in het onderzoek: "Hoe kan ontwikkelingsgerichte evaluatie een bijdrage leveren aan verduurzaming van DBE waarbij recht gedaan wordt aan kenmerkende aspecten van DBE als onderwijsconcept."
DOCUMENT
