Search results

Research bias in judgement bias studies

Valuation judgement bias has been a research topic for several years due to its proclaimed effect on valuation accuracy. However, little is known on the emphasis of literature on judgement bias, with regard to, for instance, research methodologies, research context and robustness of research evidence. A synthesis of available research will establish consistency in the current knowledge base on valuer judgement, identify future research opportunities and support decision-making policy by educational and regulatory stakeholders how to cope with judgement bias. This article therefore, provides a systematic review of empirical research on real estate valuer judgement over the last 30 years. Based on a number of inclusion and exclusion criteria, we have systematically analysed 32 relevant papers on valuation judgement bias. Although we find some consistency in evidence, we also find the underlying research to be biased; the methodology adopted is dominated by a quantitative approach; research context is skewed by timing and origination; and research evidence seems fragmented and needs replication. In order to obtain a deeper understanding of valuation judgement processes and thus extend the current knowledge base, we advocate more use of qualitative research methods and scholars to adopt an interpretative paradigm when studying judgement behaviour.

Uncovering Bias in ASR Systems

It is crucial that ASR systems can handle the wide range of variations in speech of speakers from different demographic groups, with different speaking styles, and of speakers with (dis)abilities. A potential quality-of-service harm arises when ASR systems do not perform equally well for everyone. ASR systems may exhibit bias against certain types of speech, such as non-native accents, different age groups and gender. In this study, we evaluate two widely-used neural network-based architectures: Wav2vec2 and Whisper on potential biases for Dutch speakers. We used the Dutch speech corpus JASMIN as a test set containing read and conversational speech in a human-machine interaction setting. The results reveal a significant bias against non-natives, children and elderly and some regional dialects. The ASR systems generally perform slightly better for women than for men.

Ai-bias vs. menselijke bias

Het woord ‘bias’ komt naar voren in zowel maatschappelijk als wetenschappelijk debat over de inzet van artificiële intelligentie (ai). Het verwijst doorgaans naar een vooroordeel dat iets of iemand vaak onbedoeld heeft. Wanneer dit vooroordeel leidt tot een afwijking in besluitvorming vergeleken met een situatie wanneer dit vooroordeel er niet zou zijn, dan is een bias doorgaans onwenselijk.

Metrics for Popularity Bias in Dynamic Recommender Systems

Albeit the widespread application of recommender systems (RecSys) in our daily lives, rather limited research has been done on quantifying unfairness and biases present in such systems. Prior work largely focuses on determining whether a RecSys is discriminating or not but does not compute the amount of bias present in these systems. Biased recommendations may lead to decisions that can potentially have adverse effects on individuals, sensitive user groups, and society. Hence, it is important to quantify these biases for fair and safe commercial applications of these systems. This paper focuses on quantifying popularity bias that stems directly from the output of RecSys models, leading to over recommendation of popular items that are likely to be misaligned with user preferences. Four metrics to quantify popularity bias in RescSys over time in dynamic setting across different sensitive user groups have been proposed. These metrics have been demonstrated for four collaborative filteri ng based RecSys algorithms trained on two commonly used benchmark datasets in the literature. Results obtained show that the metrics proposed provide a comprehensive understanding of growing disparities in treatment between sensitive groups over time when used conjointly.

Exploring Bias in Data and Models for Misinformation Detection from Text

With the proliferation of misinformation on the web, automatic misinformation detection methods are becoming an increasingly important subject of study. Large language models have produced the best results among content-based methods, which rely on the text of the article rather than the metadata or network features. However, finetuning such a model requires significant training data, which has led to the automatic creation of large-scale misinformation detection datasets. In these datasets, articles are not labelled directly. Rather, each news site is labelled for reliability by an established fact-checking organisation and every article is subsequently assigned the corresponding label based on the reliability score of the news source in question. A recent paper has explored the biases present in one such dataset, NELA-GT-2018, and shown that the models are at least partly learning the stylistic and other features of different news sources rather than the features of unreliable news. We confirm a part of their findings. Apart from studying the characteristics and potential biases of the datasets, we also find it important to examine in what way the model architecture influences the results. We therefore explore which text features or combinations of features are learned by models based on contextual word embeddings as opposed to basic bag-of-words models. To elucidate this, we perform extensive error analysis aided by the SHAP post-hoc explanation technique on a debiased portion of the dataset. We validate the explanation technique on our inherently interpretable baseline model.

Uncovering bias in the PlantVillage dataset.