Purpose: The aims of this study were to investigate how a variety of research methods is commonly employed to study technology and practitioner cognition. User-interface issues with infusion pumps were selected as a case because of its relevance to patient safety. Methods: Starting from a Cognitive Systems Engineering perspective, we developed an Impact Flow Diagram showing the relationship of computer technology, cognition, practitioner behavior, and system failure in the area of medical infusion devices. We subsequently conducted a systematic literature review on user-interface issues with infusion pumps, categorized the studies in terms of methods employed, and noted the usability problems found with particular methods. Next, we assigned usability problems and related methods to the levels in the Impact Flow Diagram. Results: Most study methods used to find user interface issues with infusion pumps focused on observable behavior rather than on how artifacts shape cognition and collaboration. A concerted and theorydriven application of these methods when testing infusion pumps is lacking in the literature. Detailed analysis of one case study provided an illustration of how to apply the Impact Flow Diagram, as well as how the scope of analysis may be broadened to include organizational and regulatory factors. Conclusion: Research methods to uncover use problems with technology may be used in many ways, with many different foci. We advocate the adoption of an Impact Flow Diagram perspective rather than merely focusing on usability issues in isolation. Truly advancing patient safety requires the systematic adoption of a systems perspective viewing people and technology as an ensemble, also in the design of medical device technology.
Explainable Artificial Intelligence (XAI) aims to provide insights into the inner workings and the outputs of AI systems. Recently, there’s been growing recognition that explainability is inherently human-centric, tied to how people perceive explanations. Despite this, there is no consensus in the research community on whether user evaluation is crucial in XAI, and if so, what exactly needs to be evaluated and how. This systematic literature review addresses this gap by providing a detailed overview of the current state of affairs in human-centered XAI evaluation. We reviewed 73 papers across various domains where XAI was evaluated with users. These studies assessed what makes an explanation “good” from a user’s perspective, i.e., what makes an explanation meaningful to a user of an AI system. We identified 30 components of meaningful explanations that were evaluated in the reviewed papers and categorized them into a taxonomy of human-centered XAI evaluation, based on: (a) the contextualized quality of the explanation, (b) the contribution of the explanation to human-AI interaction, and (c) the contribution of the explanation to human- AI performance. Our analysis also revealed a lack of standardization in the methodologies applied in XAI user studies, with only 19 of the 73 papers applying an evaluation framework used by at least one other study in the sample. These inconsistencies hinder cross-study comparisons and broader insights. Our findings contribute to understanding what makes explanations meaningful to users and how to measure this, guiding the XAI community toward a more unified approach in human-centered explainability.
MULTIFILE
Op donderdag 15 oktober 2009 zijn bij Saxion in Enschede de lectoren Henk van Leeuwen, Piet Griffioen en Wouter Teeuw officieel geïnstalleerd. Met zijn drieën vormen zij het lectoraat ‘ambient intelligence’ van het Saxion Kenniscentrum Design en Technologie. In hun lectorale rede ter ere van deze installatie gaan zij in op ontwikkelingen en toepassingen van ambient intelligence. Met de term ambient intelligence wordt een toekomstvisie aangeduid. In deze visie zijn omgevingen zich bewust van de aanwezigheid van personen, hun gedrag of zelfs hun intenties. Slimme omgevingen kunnen daarop reageren. Denk bijvoorbeeld aan spiegels waarop tijdens het tanden poetsen de file informatie van die dag verschijnt. Of een tapijt dat beweging kan registreren, bijvoorbeeld om patiënten te monitoren in een verzorgingstehuis. In hun rede geeft het drietal lectoren antwoord op stellingen en vragen over de mogelijkheden van ambient intelligence. Kunnen we systemen bedenken die anticiperen op wat mensen willen en ons zo beter ondersteunen in onze activiteiten? Kan een omgeving slim worden en als het ware weten wat er speelt en daarop zo te reageren dat dit door ‘ons’ als gebruiker als natuurlijk wordt ervaren? De lectoren werken voor het lectoraat ambient intelligence binnen het Kenniscentrum Design en Technologie van Saxion. Het lectoraat richt zich op de werkomgeving met aandacht voor veilig, plezierig en gezond werken.
MULTIFILE
