Introduction: In March 2014, the New South Wales (NSW) Government (Australia) announced the NSW Integrated Care Strategy. In response, a family-centred, population-based, integrated care initiative for vulnerable families and their children in Sydney, Australia was developed. The initiative was called Healthy Homes and Neighbourhoods. A realist translational social epidemiology programme of research and collaborative design is at the foundation of its evaluation. Theory and Method: The UK Medical Research Council (MRC) Framework for evaluating complex health interventions was adapted. This has four components, namely 1) development, 2) feasibility/piloting, 3) evaluation and 4) implementation. We adapted the Framework to include: critical realist, theory driven, and continuous improvement approaches. The modified Framework underpins this research and evaluation protocol for Healthy Homes and Neighbourhoods. Discussion: The NSW Health Monitoring and Evaluation Framework did not make provisions for assessment of the programme layers of context, or the effect of programme mechanism at each level. We therefore developed a multilevel approach that uses mixed-method research to examine not only outcomes, but also what is working for whom and why.
LINK
Dit artikel voor Tijdschrift voor Veiligheid behandelt de invloed van digitale buurtpreventie op criminaliteit. Het vaak veronderstelde positieve effect op het aantal aanhoudingen laat zich niet zien. Tóch is het aannemelijk dat digitale buurtpreventie criminaliteit vermindert. Door middel van sociale controle wordt de gelegenheid voor criminaliteit beperkt en worden criminele handelingen verstoord. Dit blijkt uit recente literatuur en bevindingen uit eigen onderzoek: een ‘realist evaluation’ waarin professionals en deelnemers van digitale buurtpreventiegroepen in Rotterdam is gevraagd naar hun perceptie van de werkzaamheid van dit middel. Relevant voor het werkveld: in het artikel geven we aanknopingspunten waarmee de werkzaamheid kan worden vergroot.
MULTIFILE
Research into interprofessional collaboration (IPC) has predominantly focused on health care and special-ized care settings, but there is an increasing interest in interprofesssional ‘teams around the child’ in community-based settings. We conducted a realist synthesis of empirical studies into IPC between youth professionals, often in regular community settings, to explore barriers and facilitators of IPC. Included studies were coded with an elaborated scheme to chart the focus of studies and to identify moderators and context-mechanism-outcome configurations of IPC. Professional and normative integration was the main focus of the included studies. Most studies emphasized the challenges of IPC in practice, like unclear roles of self and others, lack of trust and inadequate communication. Other perceived barriers are exclud-ing others in the planning of interventions, taking ownership of plans (vs. sharing) and different modes of communication. Interprofessional education, co-location of staff, acting as a mediator in the team, organising formal and informal meetings, conflict resolutions, self-sacrifice, and conceptualizing practice were perceived as facilitators of IPC. Future IPC research into community-based settings should include all professional stakeholders and the children and their families to evaluate outcomes at both interprofes-sional and clinical level.
DOCUMENT
Background: Collaboration between parents and speech and language therapists (SLTs) is seen as a key element in family-centred models. Collaboration can have positive impacts on parental and children’s outcomes. However, collaborative practice has not been well described and researched in speech and language therapy for children and may not be easy to achieve. It is important that we gain a deeper understanding of collaborative practice with parents, how it can be achieved and how it can impact on outcomes. This understanding could support practitioners in daily practice with regard to achieving collaborative practice with parents in different contexts. Aims: To set a research agenda on collaborative practice between parents and SLTs in order to generate evidence regarding what works, how, for whom, in what circumstances and to what extent. Methods & Procedures: A realist evaluation approach was used to make explicit what collaborative practice with parents entails. The steps suggested by the RAMESES II project were used to draft a preliminary programme theory about collaborative practice between parents and SLTs. This process generates explicit hypotheses which form a potential research agenda. Discussion & Conclusions: A preliminary programme theory of collaborative practice with parents was drafted using a realist approach. Potential contextual factors (C), mechanisms (M) and outcomes (O) were presented which could be configured into causal mechanisms to help explain what works for whom in what circumstances. CMO configurations were drafted, based on the relevant literature, which serve as exemplars to illustrate how this methodology could be used. In order to debate, test and expand our hypothesized programme theory for collaborative practice with parents, further testing against a broader literature is required alongside research to explore the functionality of the configurations across contexts. This paper highlights the importance of further research on collaborative practice with parents and the potential value of realist evaluation methodology
DOCUMENT
Living labs are complex multi-stakeholder collaborations that often employ a usercentred and design-driven methodology to foster innovation. Conventional management tools fall short in evaluating them. However, some methods and tools dedicated to living labs' special characteristics and goals have already been developed. Most of them are still in their testing phase. Those tools are not easily accessible and can only be found in extensive research reports, which are difficult to dissect. Therefore, this paper reviews seven evaluation methods and tools specially developed for living labs. Each section of this paper is structured in the following manner: tool’s introduction (1), who uses the tool (2), and how it should be used (3). While the first set of tools, namely “ENoLL 20 Indicators”, “SISCODE Self-assessment”, and “SCIROCCO Exchange Tool” assess a living lab as an organisation and are diving deeper into the organisational activities and the complex context, the second set of methods and tools, “FormIT” and “Living Lab Markers”, evaluate living labs’ methodologies: the process they use to come to innovations. The paper's final section presents “CheRRIes Monitoring and Evaluation Tool” and “TALIA Indicator for Benchmarking Service for Regions”, which assess the regional impact made by living labs. As every living lab is different regarding its maturity (as an organisation and in its methodology) and the scope of impact it wants to make, the most crucial decision when evaluating is to determine the focus of the assessment. This overview allows for a first orientation on worked-out methods and on possible indicators to use. It also concludes that the existing tools are quite managerial in their method and aesthetics and calls for designers and social scientists to develop more playful, engaging and (possibly) learning-oriented tools to evaluate living labs in the future. LinkedIn: https://www.linkedin.com/in/overdiek12345/ https://www.linkedin.com/in/mari-genova-17a727196/?originalSubdomain=nl
DOCUMENT
While traditional crime rates are decreasing, cybercrime is on the rise. As a result, the criminal justice system is increasingly dealing with criminals committing cyber-dependent crimes. However, to date there are no effective interventions to prevent recidivism in this type of offenders. Dutch authorities have developed an intervention program, called Hack_Right. Hack_Right is an alternative criminal justice program for young first-offenders of cyber-dependent crimes. In order to prevent recidivism, this program places participants in organizations where they are taught about ethical hacking, complete (technical) assignments and reflect on their offense. In this study, we have evaluated the Hack_Right program and the pilot interventions carried out thus far. By examining the program theory (program evaluation) and implementation of the intervention (process evaluation), the study adds to the scarce literature about cybercrime interventions. During the study, two qualitative research methods have been applied: 1) document analysis and 2) interviews with intervention developers, imposers, implementers and participants. In addition to the observation that the scientific basis for linking specific criminogenic factors to cybercriminals is still fragile, the article concludes that the theoretical base and program integrity of Hack_Right need to be further developed in order to adhere to principles of effective interventions.
DOCUMENT
In these uncertain times, politicians and society have been voicinghigh expectations from teacher education asking for evidence ofadded value and impact on the student. In this study a longitudinalmixed-methods approach was used to determine theimpact of in-service Master of Education programmes on teachers(N = 1,917) and their work environment in the Netherlands. Theresults of online surveys were explained by means of realist evaluationusing focus-group and in-depth interviews. The studyshows that Master of Education programmes had an impact onprofessional and pedagogical skills of teachers, and in some casesalso on their work environment. A two-tier mechanism is proposed:the Master’s programmes provide teachers with more indepthknowledge about teaching and learning and a more criticalstance through inquiry and research. Consequently, some teacherscontribute to a culture of inquiry in their schools, provided thatthe schools facilitate them in new roles.
DOCUMENT
Explainable Artificial Intelligence (XAI) aims to provide insights into the inner workings and the outputs of AI systems. Recently, there’s been growing recognition that explainability is inherently human-centric, tied to how people perceive explanations. Despite this, there is no consensus in the research community on whether user evaluation is crucial in XAI, and if so, what exactly needs to be evaluated and how. This systematic literature review addresses this gap by providing a detailed overview of the current state of affairs in human-centered XAI evaluation. We reviewed 73 papers across various domains where XAI was evaluated with users. These studies assessed what makes an explanation “good” from a user’s perspective, i.e., what makes an explanation meaningful to a user of an AI system. We identified 30 components of meaningful explanations that were evaluated in the reviewed papers and categorized them into a taxonomy of human-centered XAI evaluation, based on: (a) the contextualized quality of the explanation, (b) the contribution of the explanation to human-AI interaction, and (c) the contribution of the explanation to human- AI performance. Our analysis also revealed a lack of standardization in the methodologies applied in XAI user studies, with only 19 of the 73 papers applying an evaluation framework used by at least one other study in the sample. These inconsistencies hinder cross-study comparisons and broader insights. Our findings contribute to understanding what makes explanations meaningful to users and how to measure this, guiding the XAI community toward a more unified approach in human-centered explainability.
MULTIFILE
Background: The Nurses in the Lead (NitL) programme consists of a systematic approach and training to 1) empower community nurses in implementing evidence, targeted at encouraging functional activities of older adults, and 2) train community nurses in enabling team members to change their practice. This article aims to describe the process evaluation of NitL. Methods: A mixed-methods formative process evaluation with a predominantly qualitative approach was conducted. Qualitative data were collected by interviews with community nurses (n = 7), focus groups with team members (n = 31), and reviewing seven implementation plans and 28 patient records. Quantitative data were collected among community nurses and team members (N = 90) using a questionnaire to assess barriers in encouraging functional activities and attendance lists. Data analysis was carried out through descriptive statistics and content analysis. Results: NitL was largely executed according to plan. Points of attention were the use and value of the background theory within the training, completion of implementation plans, and reporting in patient records by community nurses. Inhibiting factors for showing leadership and encouraging functional activities were a lack of time and a high complexity of care; facilitating factors were structure and clear communication within teams. Nurses considered the systematic approach useful and the training educational for their role. Most team members considered NitL practical and were satisfied with the coaching provided by community nurses. To optimise NitL, community nurses recommended providing the training first and extending the training. The team members recommended continuing clinical lessons, which were an implementation strategy from the community nurses. Conclusions: NitL was largely executed as planned, and appears worthy of further application in community care practice. However, adaptations are recommended to make NitL more promising in practice in empowering community nurse leadership in implementing evidence.
DOCUMENT
In dit rapport wordt verslag gedaan van de resultaten van een onderzoek naar ontwikkelwerkplaatsen die zijn geïnitieerd en uitgevoerd. Het onderzoek vond plaats in de periode januari 2019 - juni 2020 door onderzoekers van de lectoraten: Innovatieve Maatschappelijke Dienstverlening, Participatie Zorg en Ondersteuning en Jeugd van het Kenniscentrum Sociale Innovatie en met ondersteuning van het Lectoraat Methodologie van Praktijkgericht Onderzoek van de Hogeschool Utrecht. Het onderzoek is door al deze lectoraten uit eigen middelen gefinancierd. De aanleiding voor dit onderzoek was dat het moment zich ervoor leent samen met facilitators en onderzoekers van deze werkplaatsen terug te blikken en vooruit te kijken. De bedoeling ervan was niet alleen een evaluatie uit te voeren maar tevens te leren van deze ervaringen en bij te dragen aan de verdere ontwikkeling en onderbouwing van de ontwikkelwerkplaatsen als een succesformule voor praktijkonderzoek. Over het algemeen is namelijk de indruk dat de ontwikkelwerkplaatsen een krachtige stimulans vormen voor praktijkontwikkeling, maar het is nog niet zo eenvoudig aan te geven waardoor dit het geval is. Vragen die hiermee samenhangen zijn welke doelen worden nagestreefd en welke resultaten worden bereikt, wat werkzame elementen zijn en waarom ze in de ene situatie wel goed uitpakken en in een andere iets minder (zie Metz, 2020 voor een actueel en verhelderend overzicht van de stand van zaken).
DOCUMENT
