Explainable Artificial Intelligence (XAI) aims to provide insights into the inner workings and the outputs of AI systems. Recently, there’s been growing recognition that explainability is inherently human-centric, tied to how people perceive explanations. Despite this, there is no consensus in the research community on whether user evaluation is crucial in XAI, and if so, what exactly needs to be evaluated and how. This systematic literature review addresses this gap by providing a detailed overview of the current state of affairs in human-centered XAI evaluation. We reviewed 73 papers across various domains where XAI was evaluated with users. These studies assessed what makes an explanation “good” from a user’s perspective, i.e., what makes an explanation meaningful to a user of an AI system. We identified 30 components of meaningful explanations that were evaluated in the reviewed papers and categorized them into a taxonomy of human-centered XAI evaluation, based on: (a) the contextualized quality of the explanation, (b) the contribution of the explanation to human-AI interaction, and (c) the contribution of the explanation to human- AI performance. Our analysis also revealed a lack of standardization in the methodologies applied in XAI user studies, with only 19 of the 73 papers applying an evaluation framework used by at least one other study in the sample. These inconsistencies hinder cross-study comparisons and broader insights. Our findings contribute to understanding what makes explanations meaningful to users and how to measure this, guiding the XAI community toward a more unified approach in human-centered explainability.
MULTIFILE
The study of human factors in forensic science informs our understanding of the interaction between humans and the systems they use. The Expert Working Group (EWG) on Human Factors in Forensic DNA Interpretation used a systems approach to conduct a scientific assessment of the effects of human factors on forensic DNA interpretation with the goal of recommending approaches to improve practice and reduce the likelihood and consequence of errors. This effort resulted in 44 recommendations. The EWG designed many of these recommendations to improve the production, interpretation, evaluation, documentation, and communication of DNA comparison results.
MULTIFILE
This review is the first step in a long-term research project exploring how social robotics and AI-generated content can contribute to the creative experiences of older adults, with a focus on collaborative drawing and painting. We systematically searched and selected literature on human-robot co-creativity, and analyzed articles to identify methods and strategies for researching co-creative robotics. We found that none of the studies involved older adults, which shows the gap in the literature for this often involved participant group in robotics research. The analyzed literature provides valuable insights into the design of human-robot co-creativity and informs a research agenda to further investigate the topic with older adults. We argue that future research should focus on ecological and developmental perspectives on creativity, on how system behavior can be aligned with the values of older adults, and on the system structures that support this best.
Het lopen van een marathon wordt steeds populairder. Naast de vele positieve gezondheidseffecten van duurinspanning, kan duurinspanning ook gepaard gaan met maagdarmklachten. Zo’n 30-90% van de hardlopers heeft last van maagdarmklachten tijdens of in de uren na het hardlopen. Het ontstaan van maagdarmklachten heeft waarschijnlijk te maken met de herverdeling van het bloedvolume, resulterend in minder bloedtoevoer naar het spijsverteringskanaal en een minder goed functionerende darmbarrière. Doordat de darmbarrière minder goed functioneert kunnen er ongewenste stoffen (endotoxinen) de bloedbaan intreden en voor ontstekingsreacties zorgen. De vele micro-organismen in onze darm, gezamenlijk onze darmmicrobiota genoemd, zijn van invloed op de voedselvertering, maar ook op het functioneren van de cellen die de darmwand bekleden en de verbindingen tussen deze cellen. Mogelijk hebben hardlopers met maagdarmklachten tijdens duurinspanning te maken met een afwijkende samenstelling van de darmmicrobiota en/of metabolieten ten opzichte van hardlopers zonder klachten, waardoor de darmbarrière minder goed functioneert en er problemen kunnen optreden. Vandaar dat het voornaamste doel van ons onderzoeksproject is om te onderzoeken of er een relatie bestaat tussen de samenstelling van de darmmicrobiota en/of metabolieten en het ontstaan van maagdarmklachten tijdens duurinspanning. De onderzoeksvragen die zullen worden bestudeerd zijn: 1) Verschilt de samenstelling van de darmmicrobiota en/of metabolieten van hardlopers die wel en niet last krijgen van maagdarmklachten tijdens het lopen van een marathon? En zo ja, hoe? 2) Kan de samenstelling van de darmmicrobiota en/of metabolieten van getrainde sporters die maagdarmklachten ervaren tijdens duurinspanning positief beïnvloed worden door probiotica-suppletie, zodat de kans op en/of intensiteit van maagdarmklachten tijdens duurinspanning wordt verminderd en de sportprestatie verbeterd? Het onderzoeksproject richt zich op de identificatie van sporters die last hebben van maagdarmklachten tijdens duurinspanning. We hopen met de beoogde resultaten bij te kunnen dragen aan op de persoon gerichte preventie van maagdarmklachten door het aanpassen van de darmmicrobiota.
Due to the existing pressure for a more rational use of the water, many public managers and industries have to re-think/adapt their processes towards a more circular approach. Such pressure is even more critical in the Rio Doce region, Minas Gerais, due to the large environmental accident occurred in 2015. Cenibra (pulp mill) is an example of such industries due to the fact that it is situated in the river basin and that it has a water demanding process. The current proposal is meant as an academic and engineering study to propose possible solutions to decrease the total water consumption of the mill and, thus, decrease the total stress on the Rio Doce basin. The work will be divided in three working packages, namely: (i) evaluation (modelling) of the mill process and water balance (ii) application and operation of a pilot scale wastewater treatment plant (iii) analysis of the impacts caused by the improvement of the process. The second work package will also be conducted (in parallel) with a lab scale setup in The Netherlands to allow fast adjustments and broaden evaluation of the setup/process performance. The actions will focus on reducing the mill total water consumption in 20%.
The focus of the research is 'Automated Analysis of Human Performance Data'. The three interconnected main components are (i)Human Performance (ii) Monitoring Human Performance and (iii) Automated Data Analysis . Human Performance is both the process and result of the person interacting with context to engage in tasks, whereas the performance range is determined by the interaction between the person and the context. Cheap and reliable wearable sensors allow for gathering large amounts of data, which is very useful for understanding, and possibly predicting, the performance of the user. Given the amount of data generated by such sensors, manual analysis becomes infeasible; tools should be devised for performing automated analysis looking for patterns, features, and anomalies. Such tools can help transform wearable sensors into reliable high resolution devices and help experts analyse wearable sensor data in the context of human performance, and use it for diagnosis and intervention purposes. Shyr and Spisic describe Automated Data Analysis as follows: Automated data analysis provides a systematic process of inspecting, cleaning, transforming, and modelling data with the goal of discovering useful information, suggesting conclusions and supporting decision making for further analysis. Their philosophy is to do the tedious part of the work automatically, and allow experts to focus on performing their research and applying their domain knowledge. However, automated data analysis means that the system has to teach itself to interpret interim results and do iterations. Knuth stated: Science is knowledge which we understand so well that we can teach it to a computer; and if we don't fully understand something, it is an art to deal with it.[Knuth, 1974]. The knowledge on Human Performance and its Monitoring is to be 'taught' to the system. To be able to construct automated analysis systems, an overview of the essential processes and components of these systems is needed.Knuth Since the notion of an algorithm or a computer program provides us with an extremely useful test for the depth of our knowledge about any given subject, the process of going from an art to a science means that we learn how to automate something.
