Objective: To annotate a corpus of randomized controlled trial (RCT) publications with the checklist items of CONSORT reporting guidelines and using the corpus to develop text mining methods for RCT appraisal. Methods: We annotated a corpus of 50 RCT articles at the sentence level using 37 fine-grained CONSORT checklist items. A subset (31 articles) was double-annotated and adjudicated, while 19 were annotated by a single annotator and reconciled by another. We calculated inter-annotator agreement at the article and section level using MASI (Measuring Agreement on Set-Valued Items) and at the CONSORT item level using Krippendorff's α. We experimented with two rule-based methods (phrase-based and section header-based) and two supervised learning approaches (support vector machine and BioBERT-based neural network classifiers), for recognizing 17 methodology-related items in the RCT Methods sections. Results: We created CONSORT-TM consisting of 10,709 sentences, 4,845 (45%) of which were annotated with 5,246 labels. A median of 28 CONSORT items (out of possible 37) were annotated per article. Agreement was moderate at the article and section levels (average MASI: 0.60 and 0.64, respectively). Agreement varied considerably among individual checklist items (Krippendorff's α= 0.06–0.96). The model based on BioBERT performed best overall for recognizing methodology-related items (micro-precision: 0.82, micro-recall: 0.63, micro-F1: 0.71). Combining models using majority vote and label aggregation further improved precision and recall, respectively. Conclusion: Our annotated corpus, CONSORT-TM, contains more fine-grained information than earlier RCT corpora. Low frequency of some CONSORT items made it difficult to train effective text mining models to recognize them. For the items commonly reported, CONSORT-TM can serve as a testbed for text mining methods that assess RCT transparency, rigor, and reliability, and support methods for peer review and authoring assistance. Minor modifications to the annotation scheme and a larger corpus could facilitate improved text mining models. CONSORT-TM is publicly available at https://github.com/kilicogluh/CONSORT-TM.
Calls have been made for improving transparency in conducting and reporting research, improving work climates, and preventing detrimental research practices. To assess attitudes and practices regarding these topics, we sent a survey to authors, reviewers, and editors. We received 3,659 (4.9%) responses out of 74,749 delivered emails. We found no significant differences between authors’, reviewers’, and editors’ attitudes towards transparency in conducting and reporting research, or towards their perceptions of work climates. Undeserved authorship was perceived by all groups as the most prevalent detrimental research practice, while fabrication, falsification, plagiarism, and not citing prior relevant research, were seen as more prevalent by editors than authors or reviewers. Overall, 20% of respondents admitted sacrificing the quality of their publications for quantity, and 14% reported that funders interfered in their study design or reporting. While survey respondents came from 126 different countries, due to the survey’s overall low response rate our results might not necessarily be generalizable. Nevertheless, results indicate that greater involvement of all stakeholders is needed to align actual practices with current recommendations.
MULTIFILE
The Dutch hospitality industry, reflecting the wider Dutch society, is increasingly facing social sustainability challenges for a greying population, such as increasing burnout, lifelong learning, and inclusion for those distanced from the job market. Yet, while the past decades have seen notable progress regarding environmental sustainability and good governance, more attention should be paid to social sustainability. This concern is reflected by the top-sector healthcare struggles caused by mounting social welfare pressure, leading to calls by the Dutch government for organizational improvement in social earning capacity. Furthermore, the upcoming EU legislation on CSRD requires greater transparency regarding financial and non-financial reporting this year. Yet, while the existing sustainability accreditation frameworks offer guidance on environmental sustainability and good governance reporting, there must be more guidance on auditing social sustainability. The hospitality industry, as a prominent employer in the Netherlands, thus has a societal and legislative urgency to transition its social earning capacity. Dormben Hotel The Hague OpCo BV (Dormben) has thus sought support in transitioning its social sustainability standards to meet this call. Hotelschool, the Hague leads the consortium, including Green Key Nederland and Dormben, by employing participatory design to present a social sustainability accreditation framework. Initially, Dr. David Brannon and Dr. Melinda Ratkai from Hotelschool The Hague will draft a social sustainability accreditation framework informed by EFRAG. Subsequently, Erik van Wijk, from Green Key Nederland, the hospitality benchmark for sustainability accreditation, and Sander de Jong, from Dormben, will pilot the framework through four participatory workshops involving hospitality operators. Later, during a cross-industry conference, Dr. David Brannon and Dr. Melinda Ratkai will disseminate a social sustainability toolkit across their academic and industry networks. Finally, conference and workshop participants will be invited to form a social sustainability learning community, discussing their social earning capacity based on the revised sustainability accreditation.
Moderatie van lezersreacties onder nieuwsartikelen is erg arbeidsintensief. Met behulp van kunstmatige intelligentie wordt moderatie mogelijk tegen een redelijke prijs. Aangezien elke toepassing van kunstmatige intelligentie eerlijk en transparant moet zijn, is het belangrijk om te onderzoeken hoe media hieraan kunnen voldoen.Doel Dit promotieproject zal zich richten op de rechtvaardigheid, accountability en transparantie van algoritmische systemen voor het modereren van lezersreacties. Het biedt een theoretisch kader en bruikbare matregelen die nieuwsorganisaties zullen ondersteunen in het naleven van recente beleidsvorming voor een waardegedreven implementatie van AI. Nu steeds meer nieuwsmedia AI gaan gebruiken, moeten ze rechtvaardigheid, accountability en transparantie in hun gebruik van algoritmen meenemen in hun werkwijzen. Resultaten Hoewel moderatie met AI zeer aantrekkelijk is vanuit economisch oogpunt, moeten nieuwsmedia weten hoe ze onnauwkeurigheid en bias kunnen verminderen (fairness), de werking van hun AI bekendmaken (accountability) en de gebruikers laten begrijpen hoe beslissingen via AI worden genomen (transparancy). Dit proefschrift bevordert de kennis over deze onderwerpen. Looptijd 01 februari 2022 - 01 februari 2025 Aanpak De centrale onderzoeksvraag van dit promotieonderzoek is: Hoe kunnen en moeten nieuwsmedia rechtvaardigheid, accountability en transparantie in hun gebruik van algoritmes voor commentmoderatie? Om deze vraag te beantwoorden is het onderzoek opgesplitst in vier deelvragen. Hoe gebruiken nieuwsmedia algoritmes voor het modereren van reacties? Wat kunnen nieuwsmedia doen om onnauwkeurigheid en bias bij het modereren via AI van reacties te verminderen? Wat moeten nieuwsmedia bekendmaken over hun gebruik van moderatie via AI? Wat maakt uitleg van moderatie via AI begrijpelijk voor gebruikers van verschillende niveaus van digitale competentie?
