Lecture in PhD Programme Life Science Education Research UMCU. Course Methods of Life Science Education Research. Utrecht, The Netherlands. abstract Audit trail procedures are applied as a way to check the validity of qualitative research designs, qualitative analyses, and the claims that are made. Audit trail procedures can be conducted based on the three criteria of visibility, comprehensibility, and acceptability (Akkerman et al., 2008). During an audit trail procedure, all documents and materials resulting from the data gathering and the data analysis are assessed by an auditor. In this presentation, we presented a summative audit trail procedure (Agricola, Prins, Van der Schaaf & Van Tartwijk, 2021), whereas in a second study we used a formative one (Agricola, Van der Schaaf, Prins & Van Tartwijk, 2022). For both studies, two different auditors were chosen. For the study presented in Agricola et al. (2021) the auditor was one of the PhD supervisors, while in that presented Agricola et al. (2022) was a junior researcher not involved in the project. The first auditor had a high level of expertise in the study’s topic and methodology. As a result, he was able to provide a professional and critical assessment report. Although the second auditor might be considered to be more objective than the first, as she was not involved in the project, more meetings were needed to explain the aim of the study and the aim of the audit trail procedure. There are many ideas about the criteria that qualitative studies should meet (De Kleijn en Van Leeuwen, 2018). I argue that procedures of checking for interrater agreement and understanding, the triangulation, and audit trail procedures can increase the internal validity of qualitative studies. Agricola, B. T., Prins, F. J., van der Schaaf, M. F., & van Tartwijk, J. (2021). Supervisor and Student Perspectives on Undergraduate Thesis Supervision in Higher Education. Scandinavian Journal of Educational Research, 65(5), 877-897. doi: https://doi.org/10.1080/00313831.2020.1775115 Agricola, B. T., van der Schaaf, M. F., Prins, F. J., & van Tartwijk, J. (2022). The development of research supervisors’ pedagogical content knowledge in a lesson study project. Educational Action Research. doi: https://doi.org/10.1080/09650792.2020.1832551 de Kleijn, R. A. M., & Van Leeuwen, A. (2018). Reflections and review on the audit procedure: Guidelines for more transparency. International Journal of Qualitative Methods, 17(1), 1-8. doi: https://doi.org/10.1177/1609406918763214 Akkerman, S., Admiraal, W., Brekelmans, M., & Oost, H. (2008). Auditing quality of research in social sciences. Quality & Quantity, 42(2), 257-274. doi: https://doi.org/10.1007/s11135-006-9044-4
DOCUMENT
The current standard in accounting practice is the double-entry approach. Basis of the double-entry approach is that every financial event brings two equal and offsetting entries. Since these financial events are not automatically confirmed by both parties, the accounting quality can be improved. The blockchain mechanism possibly offers a different take on accounting. Based on an experimentation approach, data was collected to compare the double-entry method with the blockchain-based triple-entry method. The results show that the main difference concerns determining the completeness of the financial statement items. In the situation of double-entry accounting, segregation of duties is applied to do so. In the blockchain situation, the underlying mechanism of the blockchain already ensures this.
DOCUMENT
Wanneer een kwaliteitsmanagementsysteem net is opgezet, is het zinvol om een interne audit te beperken tot het onderzoeken of datgene wat is beschreven ook in de praktijk zo wordt uitgevoerd. Naarmate een interne audit vaker wordt gehanteerd, dient het doel te verschuiven van non-conformiteit naar sturing. Pas dan kan de auditor nagaan of doelstellingen van processen en management daadwerkelijk worden behaald en ontstaan zinvolle verbeteracties. Wanneer men bij interne audits blijft steken in het toetsen van non-conformiteiten, bestaat het risico dat het kwaliteitsmanagementsysteem niet verder wordt ontwikkeld en verbeterd. Uit de resultaten van een pilot onderzoek in het Medisch Centrum Alkmaar blijkt dat een combinatie van verschillende procedures voor auditing aanbeveling verdient. Aldus worden resultaten verkregen op meerdere management niveaus en op meer terreinen dan alleen het werkproces als zodanig. Discrepanties tussen de praktijksituatie en de beschreven procedures komen snel aan het licht, evenals opties voor verbetering van de efficiëntie van processen en procedures. De informatie is een belangrijke impuls voor verbetering van werkprocessen en biedt tevens aanknopingspunten voor verbetering van belangrijke thema’s en speerpunten in de organisatie.
DOCUMENT
Background: Healthy urban environments require careful planning and a testing of environmental quality that goes beyond statutory requirements. Moreover, it requires the inclusion of resident views, perceptions and experiences that help deepen the understanding of local (public health) problems. To facilitate this, neighbourhoods should be mapped in a way that is relevant to them. One way to do this is participative neighbourhood auditing. This paper provides an insight into availability and characteristics of participatory neighbourhood audit instruments. Methods: A scoping review in scientific and grey literature, consisting of the following steps: literature search, identification and selection of relevant audit instruments, data extraction and data charting (including a work meeting to discuss outputs), reporting. Results: In total, 13 participatory instruments were identified. The role of residents in most instruments was as ‘data collectors’; only few instruments included residents in other audit activities like problem definition or analysis of data. The instruments identified focus mainly on physical, not social, neighbourhood characteristics. Paper forms containing closed-ended questions or scales were the most often applied registration method. Conclusions: The results show that neighbourhood auditing could be improved by including social aspects in the audit tools. They also show that the role of residents in neighbourhood auditing is limited; however, little is known about how their engagement takes place in practice. Developers of new instruments need to balance not only social and physical aspects, but also resident engagement and scientific robustness. Technologies like mobile applications pose new opportunities for participative approaches in neighbourhood auditing.
DOCUMENT
Objectives There is a broad call for change towards € new era' quality systems in healthcare, in which the focus lies on learning and improving. A promising way to establish this in general practice care is to combine audit and feedback with peer group discussion. However, it is not known what different stakeholders think of this type of quality improvement. The aim of this research was to explore the opinions of different stakeholders in general practice on peer discussion of audit and feedback and on its opportunities and risks. Second, their thoughts on transparency versus accountability, regarding this system, were studied. Design An exploratory qualitative study within a constructivist paradigm. Semistructured interviews and focus group discussions were held and coded using thematic analysis. Included stakeholders were general practitioners (GP), patients, professional organisations and insurance companies. Setting General practice in the Netherlands. Participants 22 participants were purposively sampled for eight interviews and two focus group discussions. Results Three main opportunities of peer discussion of audit and feedback were identified: deeper levels of reflection on data, adding context to numbers and more ownership; and three main risks: handling of unwilling colleagues, lacking a safe group and the necessity of patient involvement. An additional theme concerned disagreement on the amount of transparency to be offered: insurance companies and patients advocated for complete transparency on data and improvement of outcomes, while GPs and professional organisations urged to restrict transparency to giving insight into the process. Conclusions Peer discussion of audit and feedback could be part of a change movement, towards a quality system based on learning and trust, that is initiated by the profession. Creating a safe learning environment and involving patients is key herein. Caution is needed when complete transparency is asked, since it could jeopardise practitioners' reflection and learning in safety.
DOCUMENT
Multilevel models using logistic regression (MLogRM) and random forest models (RFM) are increasingly deployed in industry for the purpose of binary classification. The European Commission’s proposed Artificial Intelligence Act (AIA) necessitates, under certain conditions, that application of such models is fair, transparent, and ethical, which consequently implies technical assessment of these models. This paper proposes and demonstrates an audit framework for technical assessment of RFMs and MLogRMs by focussing on model-, discrimination-, and transparency & explainability-related aspects. To measure these aspects 20 KPIs are proposed, which are paired to a traffic light risk assessment method. An open-source dataset is used to train a RFM and a MLogRM model and these KPIs are computed and compared with the traffic lights. The performance of popular explainability methods such as kernel- and tree-SHAP are assessed. The framework is expected to assist regulatory bodies in performing conformity assessments of binary classifiers and also benefits providers and users deploying such AI-systems to comply with the AIA.
DOCUMENT
What is the ‘raison d’être’ of auditing? Does auditing only exist by the grace of the legislator? Or does auditing fulfill other needs in contemporary society? For many companies, auditing has been made mandatory. This is possibly one of the reasons why researchers to date have given limited attention regarding the drivers for the demand for audit. Auditing (seen as a social control mechanism) is part of an organizational order in society. Therefore, it is essential to reflect on the (changing) demands of society. As a lack of insight why society demands an audit, accommodates the risk of not meeting the needs and expectations of society, the added value of auditing may be called into question. This dissertation deals with the question: what are drivers for the demand for audit for Dutch SME companies which are not mandatory required to have their financial statements audited.
LINK
Spreadsheets are known to be error-prone. Over the last decade, research has been done to determine the causes of the high rate of errors in spreadsheets. This paper examines the added value of a spreadsheet tool (PerfectXL) that visualizes spreadsheet dependencies and determines possible errors in spreadsheets by defining risk areas based on previous work. This paper will firstly discuss the most common mistakes in spreadsheets. Then we will summarize research on spreadsheet tools, focussing on the PerfectXL tool. To determine the perceptions of the usefulness of a spreadsheet tool in general and the PerfectXL tool in particular, we have shown the functionality of PerfectXL to several auditors and have also interviewed them. The results of these interviews indicate that spreadsheet tools support a more effective and efficient audit of spreadsheets; the visualization feature in particular is mentioned by the auditors as being highly supportive for their audit task, whereas the risk feature was deemed of lesser value.
LINK
What you don’t know can’t hurt you: this seems to be the current approach for responding to disinformation by public regulators across the world. Nobody is able to say with any degree of certainty what is actually going on. This is in no small part because, at present, public regulators don’t have the slightest idea how disinformation actually works in practice. We believe that there are very good reasons for the current state of affairs, which stem from a lack of verifiable data available to public institutions. If an election board or a media regulator wants to know what types of digital content are being shared in their jurisdiction, they have no effective mechanisms for finding this data or ensuring its veracity. While there are many other reasons why governments would want access to this kind of data, the phenomenon of disinformation provides a particularly salient example of the consequences of a lack of access to this data for ensuring free and fair elections and informed democratic participation. This chapter will provide an overview of the main aspects of the problems associated with basing public regulatory decisions on unverified data, before sketching out some ideas of what a solution might look like. In order to do this, the chapter develops the concept of auditing intermediaries. After discussing which problems the concept of auditing intermediaries is designed to solve, it then discusses some of the main challenges associated with access to data, potential misuse of intermediaries, and the general lack of standards for the provision of data by large online platforms. In conclusion, the chapter suggests that there is an urgent need for an auditing mechanism to ensure the accuracy of transparency data provided by large online platform providers about the content on their services. Transparency data that have been audited would be considered verified data in this context. Without such a transparency verification mechanism, existing public debate is based merely on a whim, and digital dominance is likely to only become more pronounced.
MULTIFILE
