Explainable Artificial Intelligence (XAI) aims to provide insights into the inner workings and the outputs of AI systems. Recently, there’s been growing recognition that explainability is inherently human-centric, tied to how people perceive explanations. Despite this, there is no consensus in the research community on whether user evaluation is crucial in XAI, and if so, what exactly needs to be evaluated and how. This systematic literature review addresses this gap by providing a detailed overview of the current state of affairs in human-centered XAI evaluation. We reviewed 73 papers across various domains where XAI was evaluated with users. These studies assessed what makes an explanation “good” from a user’s perspective, i.e., what makes an explanation meaningful to a user of an AI system. We identified 30 components of meaningful explanations that were evaluated in the reviewed papers and categorized them into a taxonomy of human-centered XAI evaluation, based on: (a) the contextualized quality of the explanation, (b) the contribution of the explanation to human-AI interaction, and (c) the contribution of the explanation to human- AI performance. Our analysis also revealed a lack of standardization in the methodologies applied in XAI user studies, with only 19 of the 73 papers applying an evaluation framework used by at least one other study in the sample. These inconsistencies hinder cross-study comparisons and broader insights. Our findings contribute to understanding what makes explanations meaningful to users and how to measure this, guiding the XAI community toward a more unified approach in human-centered explainability.
MULTIFILE
In a recent official statement, Google highlighted the negative effects of fake reviews on review websites and specifically requested companies not to buy and users not to accept payments to provide fake reviews (Google, 2019). Also, governmental authorities started acting against organisations that show to have a high number of fake reviews on their apps (DigitalTrends, 2018; Gov UK, 2020; ACM, 2017). However, while the phenomenon of fake reviews is well-known in industries as online journalism and business and travel portals, it remains a difficult challenge in software engineering (Martens & Maalej, 2019). Fake reviews threaten the reputation of an organisation and lead to a disvalued source to determine the public opinion about brands. Negative fake reviews can lead to confusion for customers and a loss of sales. Positive fake reviews might also lead to wrong insights about real users’ needs and requirements. Although fake reviews have been studied for a while now, there are only a limited number of spam detection models available for companies to protect their corporate reputation. Especially in times with the coronavirus, organisations need to put extra focus on online presence and limit the amount of negative input that affects their competitive position which can even lead to business loss. Given state-of-the-art derived features that can be engineered from review texts, a spam detector based on supervised machine learning is derived in an experiment that performs quite well on the well-known Amazon Mechanical Turk dataset.
MULTIFILE
Standard SARS-CoV-2 testing protocols using nasopharyngeal/throat (NP/T) swabs are invasive and require trained medical staff for reliable sampling. In addition, it has been shown that PCR is more sensitive as compared to antigen-based tests. Here we describe the analytical and clinical evaluation of our in-house RNA extraction-free saliva-based molecular assay for the detection of SARS-CoV-2. Analytical sensitivity of the test was equal to the sensitivity obtained in other Dutch diagnostic laboratories that process NP/T swabs. In this study, 955 individuals participated and provided NP/T swabs for routine molecular analysis (with RNA extraction) and saliva for comparison. Our RT-qPCR resulted in a sensitivity of 82,86% and a specificity of 98,94% compared to the gold standard. A false-negative ratio of 1,9% was found. The SARS-CoV-2 detection workflow described here enables easy, economical, and reliable saliva processing, useful for repeated testing of individuals.
LINK
Prompt and timely response to incoming cyber-attacks and incidents is a core requirement for business continuity and safe operations for organizations operating at all levels (commercial, governmental, military). The effectiveness of these measures is significantly limited (and oftentimes defeated altogether) by the inefficiency of the attack identification and response process which is, effectively, a show-stopper for all attack prevention and reaction activities. The cognitive-intensive, human-driven alarm analysis procedures currently employed by Security Operation Centres are made ineffective (as opposed to only inefficient) by the sheer amount of alarm data produced, and the lack of mechanisms to automatically and soundly evaluate the arriving evidence to build operable risk-based metrics for incident response. This project will build foundational technologies to achieve Security Response Centres (SRC) based on three key components: (1) risk-based systems for alarm prioritization, (2) real-time, human-centric procedures for alarm operationalization, and (3) technology integration in response operations. In doing so, SeReNity will develop new techniques, methods, and systems at the intersection of the Design and Defence domains to deliver operable and accurate procedures for efficient incident response. To achieve this, this project will develop semantically and contextually rich alarm data to inform risk-based metrics on the mounting evidence of incoming cyber-attacks (as opposed to firing an alarm for each match of an IDS signature). SeReNity will achieve this by means of advanced techniques from machine learning and information mining and extraction, to identify attack patterns in the network traffic, and automatically identify threat types. Importantly, SeReNity will develop new mechanisms and interfaces to present the gathered evidence to SRC operators dynamically, and based on the specific threat (type) identified by the underlying technology. To achieve this, this project unifies Dutch excellence in intrusion detection, threat intelligence, and human-computer interaction with an industry-leading partner operating in the market of tailored solutions for Security Monitoring.
Electrohydrodynamic Atomization (EHDA), also known as Electrospray (ES), is a technology which uses strong electric fields to manipulate liquid atomization. Among many other areas, electrospray is used as an important tool for biomedical application (droplet encapsulation), water technology (thermal desalination and metal recovery) and material sciences (nanofibers and nano spheres fabrication, metal recovery, selective membranes and batteries). A complete review about the particularities of this tool and its application was recently published (2018), as an especial edition of the Journal of Aerosol Sciences. One of the main known bottlenecks of this technique, it is the fact that the necessary strong electric fields create a risk for electric discharges. Such discharges destabilize the process but can also be an explosion risk depending on the application. The goal of this project is to develop a reliable tool to prevent discharges in electrospray applications.
Post-earthquake structural damage shows that wall collapse is one of the most common failure mechanisms in unreinforced masonry buildings. It is expected to be a critical issue also in Groningen, located in the northern part of the Netherlands, where human-induced seismicity has become an uprising problem in recent years. The majority of the existing buildings in that area are composed of unreinforced masonry; they were not designed to withstand earthquakes since the area has never been affected by tectonic earthquakes. They are characterised by vulnerable structural elements such as slender walls, large openings and cavity walls. Hence, the assessment of unreinforced masonry buildings in the Groningen province has become of high relevance. The abovementioned issue motivates engineering companies in the region to research seismic assessments of the existing structures. One of the biggest challenges is to be able to monitor structures during events in order to provide a quick post-earthquake assessment hence to obtain progressive damage on structures. The research published in the literature shows that crack detection can be a very powerful tool as an assessment technique. In order to ensure an adequate measurement, state-of-art technologies can be used for crack detection, such as special sensors or deep learning techniques for pixel-level crack segmentation on masonry surfaces. In this project, a new experiment will be run on an in-plane test setup to systematically propagate cracks to be able to detect cracks by new crack detection tools, namely digital crack sensor and vision-based crack detection.
