DOCUMENT
The current set of research methods on ictresearchmethods.nl contains only one research method that refers to machine learning: the “Data analytics” method in the “Lab” strategy. This does not reflect the way of working in ML projects, where Data Analytics is not a method to answer one question but the main goal of the project. For ML projects, the Data Analytics method should be divided in several smaller steps, each becoming a method of its own. In other words, we should treat the Data Analytics (or more appropriate ML engineering) process in the same way the software engineering process is treated in the framework. In the remainder of this post I will briefly discuss each of the existing research methods and how they apply to ML projects. The methods are organized by strategy. In the discussion I will give pointers to relevant tools or literature for ML projects.
LINK
Living labs are complex multi-stakeholder collaborations that often employ a usercentred and design-driven methodology to foster innovation. Conventional management tools fall short in evaluating them. However, some methods and tools dedicated to living labs' special characteristics and goals have already been developed. Most of them are still in their testing phase. Those tools are not easily accessible and can only be found in extensive research reports, which are difficult to dissect. Therefore, this paper reviews seven evaluation methods and tools specially developed for living labs. Each section of this paper is structured in the following manner: tool’s introduction (1), who uses the tool (2), and how it should be used (3). While the first set of tools, namely “ENoLL 20 Indicators”, “SISCODE Self-assessment”, and “SCIROCCO Exchange Tool” assess a living lab as an organisation and are diving deeper into the organisational activities and the complex context, the second set of methods and tools, “FormIT” and “Living Lab Markers”, evaluate living labs’ methodologies: the process they use to come to innovations. The paper's final section presents “CheRRIes Monitoring and Evaluation Tool” and “TALIA Indicator for Benchmarking Service for Regions”, which assess the regional impact made by living labs. As every living lab is different regarding its maturity (as an organisation and in its methodology) and the scope of impact it wants to make, the most crucial decision when evaluating is to determine the focus of the assessment. This overview allows for a first orientation on worked-out methods and on possible indicators to use. It also concludes that the existing tools are quite managerial in their method and aesthetics and calls for designers and social scientists to develop more playful, engaging and (possibly) learning-oriented tools to evaluate living labs in the future. LinkedIn: https://www.linkedin.com/in/overdiek12345/ https://www.linkedin.com/in/mari-genova-17a727196/?originalSubdomain=nl
DOCUMENT
Blended learning environments (BLEs) have become an indispensable part of higher education and an essential part of course delivery. Although teachers need to be active agents in facilitating students’ self-regulation and interaction, little is known to what extent such support is provided. This study investigated the use of self-regulation strategies (SRS) and interactional methods of teachers and students in BLEs. In a cross-sectional design, 171 teachers and 331 students completed a questionnaire on the use of SRS and the application of human and non-human interactional methods. Results showed that, on average, teachers and students pay little attention to SRS and do not or hardly use interactional methods. Results also showed that experienced and inexperienced teachers did not differ in their attention to SRS, although a significant difference was found between teachers with and without online teaching experience. Teachers with more online teaching experience pay more specific attention to metacognitive and management strategies. A positive relationship was also found between the extent to which teachers use both human and non-human interactional methods and the extent to which they pay attention to SRS in the online component of BLEs. Finally, there was a positive relationship between the extent to which students utilize both human and non-human interactional methods and the extent to which they apply SRS. Outcomes of this research provide insight into the design of BLEs and emphasize the importance of teachers' attention to students’ SRS and the use of interactional methods.
DOCUMENT
Aim: To evaluate healthcare professionals' performance and treatment fidelity in the Cardiac Care Bridge (CCB) nurse-coordinated transitional care intervention in older cardiac patients to understand and interpret the study results. Design: A mixed-methods process evaluation based on the Medical Research Council Process Evaluation framework. Methods: Quantitative data on intervention key elements were collected from 153 logbooks of all intervention patients. Qualitative data were collected using semi-structured interviews with 19 CCB professionals (cardiac nurses, community nurses and primary care physical therapists), from June 2017 until October 2018. Qualitative data-analysis is based on thematic analysis and integrated with quantitative key element outcomes. The analysis was blinded to trial outcomes. Fidelity was defined as the level of intervention adherence. Results: The overall intervention fidelity was 67%, ranging from severely low fidelity in the consultation of in-hospital geriatric teams (17%) to maximum fidelity in the comprehensive geriatric assessment (100%). Main themes of influence in the intervention performance that emerged from the interviews are interdisciplinary collaboration, organizational preconditions, confidence in the programme, time management and patient characteristics. In addition to practical issues, the patient's frailty status and limited motivation were barriers to the intervention. Conclusion: Although involved healthcare professionals expressed their confidence in the intervention, the fidelity rate was suboptimal. This could have influenced the non-significant effect of the CCB intervention on the primary composite outcome of readmission and mortality 6 months after randomization. Feasibility of intervention key elements should be reconsidered in relation to experienced barriers and the population. Impact: In addition to insight in effectiveness, insight in intervention fidelity and performance is necessary to understand the mechanism of impact. This study demonstrates that the suboptimal fidelity was subject to a complex interplay of organizational, professionals' and patients' issues. The results support intervention redesign and inform future development of transitional care interventions in older cardiac patients.
DOCUMENT
From the article : "In this paper the implications of different research approaches and methods are illustrated by using two projects of the authors. Both projects take place in the same context: exploring participatory innovation within Small-to-Medium sized Enterprizes (SMEs). The main aspects coming forward when comparing the research characteristics of both projects are the importance of time and momentum, the structural set up of the project, people or participants and the abilities of the people involved. The research goal and the background of the researcher are main determinants for the chosen research methods. We hope with this paper to make researchers aware of the implications of the research methods and approach on the results of the project."
DOCUMENT
Forensic and behavioural science are often seen as two different disciplines. However, there is a growing realization that the two disciplines should be more strongly integrated. Incorporating psychological theories on human behaviour in forensic science could help solving investigative problems, especially at the crime scene. At the crime scene it is not just about applying scientific methods to analyse traces; these traces must first be perceived and categorized as relevant. At the crime scene, the behavioural perspective of an investigative psychologist could play an important role. In this study, we examine to what extent (1) investigative psychologists detect deviant behavioural cues compared to forensic examiners when investigating a crime scene, (2) forensic examiners can find the relevant traces that can be associated with this behaviour and (3) the availability of a psychological report highlighting these behavioural cues helps forensic examiners in finding more relevant traces. To this end, a total of 14 investigative psychologists and 40 forensic examiners investigated a virtual 3D mock crime scene. The results of this study show that investigative psychologists see significantly more deviant behavioural cues than forensic examiners, and that forensic examiners who receive a psychological report on these cues recognize and collect significantly more traces that can be linked to deviant behaviour and have a high evidential value than examiners who did not receive this information. However, the study also demonstrates that behavioural information is likely to be ignored when it contradicts existing beliefs.
DOCUMENT
Citizens regularly search the Web to make informed decisions on daily life questions, like online purchases, but how they reason with the results is unknown. This reasoning involves engaging with data in ways that require statistical literacy, which is crucial for navigating contemporary data. However, many adults struggle to critically evaluate and interpret such data and make data-informed decisions. Existing literature provides limited insight into how citizens engage with web-sourced information. We investigated: How do adults reason statistically with web-search results to answer daily life questions? In this case study, we observed and interviewed three vocationally educated adults searching for products or mortgages. Unlike data producers, consumers handle pre-existing, often ambiguous data with unclear populations and no single dataset. Participants encountered unstructured (web links) and structured data (prices). We analysed their reasoning and the process of preparing data, which is part of data-ing. Key data-ing actions included judging relevance and trustworthiness of the data and using proxy variables when relevant data were missing (e.g., price for product quality). Participants’ statistical reasoning was mainly informal. For example, they reasoned about association but did not calculate a measure of it, nor assess underlying distributions. This study theoretically contributes to understanding data-ing and why contemporary data may necessitate updating the investigative cycle. As current education focuses mainly on producers’ tasks, we advocate including consumers’ tasks by using authentic contexts (e.g., music, environment, deferred payment) to promote data exploration, informal statistical reasoning, and critical web-search skills—including selecting and filtering information, identifying bias, and evaluating sources.
LINK
Understanding the unprecedented impact of COVID-19 on mental health and digital interactions is crucial, but also difficult to study in times of physical distancing. This paper contributes to the understanding of well-being in The Netherlands during the pandemic by employing mixed-remote methods. Sentiments of the Dutch public expressed on X (formally Twitter) are analyzed with AI techniques. Additionally, co-creative toolkits and probes, such as diaries, were used with older adults and students for detailed in-situ capturing. The AI approach provides general insights, while toolkit studies can address interpersonal variation and provide non-automated individual feedback. Findings indicate that (1) the pandemic has impacted the expressed emotional states of ‘loneliness’ and ‘happiness’, (2) this varied over time, for example related to pandemic announcements, (3) there are differences between groups (such as young and old), and (4) the toolkits provided contextual self-reflective insights and active inspiration in support of mental well-being.
LINK
This chapter deals with the study of how deaf and hearing signers, and others, understand sign languages by themselves and in relationship to other languages and modalities. By doing linguistic ethnography, it is possible to investigate these language attitudes and ideologies as they unfold in everyday practice, towards ideas such as the status of sign languages and particular varieties; discourses surrounding linguistic authority, authenticity and ownership; and the emergence (or development) of new sign languages and new subject-specific vocabulary. The methods discussed in this chapter are ethnographic research methods and visual methods: participant observation, ethnographic filmmaking, and language portraits. The main points of the chapter are illustrated by means of three case studies: (1) participant observation in multilingual tourist spaces in Bali, in which Indonesian Sign Language, International Sign, and American Sign Language are used; (2) ethnographic filmmaking within an international multi-sited research project focusing on International Sign; and (3) the use of language portraits with new signers and heritage signers in Flanders, who mostly use Flemish Sign Language and Dutch
LINK
