Blog in het kader van het onderzoeksproject ‘The Network is the Message‘ Met dit onderzoek willen Hogeschool Rotterdam en Hogeschool Utrecht een antwoord geven op de vraag: “Hoe kan de effectiviteit van communicatie in online sociale netwerken worden beoordeeld en verbeterd?” In deze blog: Doelstellingen zijn van zeer groot belang om te kunnen bepalen of je succes hebt of niet. En bij doelstellingen horen KPI’s of zorgvuldig geselecteerde metrics, die de beste bijdrage leveren om die doelstellingen ook te meten. Hoewel bureaus over het algemeen wel (marketing-) communicatiedoelstellingen hebben voor de totale campagne, worden deze niet altijd vertaald naar specifieke KPI’s voor social media.
DOCUMENT
![Sensible & significant metrics: kansrijke kpi’s voor social media [onderzoek]](https://publinova-harvester-content-prod.s3.amazonaws.com/thumbnails/files/previews/pdf/20250506142018649076.b4e36f9c-5d74-473e-a623-424c10825d73-thumbnail-400x300.png)
The objective of the study described in this paper is to define safety metrics that are based on the utilisation of resources. The background of this research is a specific need of the aviation industry where small and medium-sized enterprises lack large amounts of safety-related data to measure and demonstrate their safety performance proactively. The research department of the Aviation Academy has initiated a 4-year study, which will test the possibility to develop new safety indicators that will be able to represent safety levels proactively without the benefit of large data sets. The research team has reviewed the academic and professional literature about safety performance indicators and has performed surveys into 13 companies in order to explore what, how, and why safety performance indicators are used and whether there is a statistically linear relation between SMS process metrics and safety outcomes. The preliminary results showed that companies do not use data from all SMS processes in the development of safety performance indicators, they do not ground the selection of indicators on specific criteria, they implement SMS process in different ways, but they are eager to use alternative metrics, including ones potentially to be derived on the basis of contemporary safety models and views. As part of the development of alternative safety metrics, safety performance indicators were defined that are based on the difference between required resources and available resources. Resources are people, time, equipment and budget. This work is inspired by the general notion that a large gap between ‘work as imagined’ and ‘work as done’ has a negative influence on the level of safety. Work as imagined in this context is represented by available resources and work as done by required resources. The metrics were defined by a combination of literature research and semi-structured interviews with operational practitioners in the aviation industry. The suitability of the metrics will subsequently be tested in pilot studies within the aviation industry.
DOCUMENT
Following the completion of the 1st phase of the RAAK PRO project Aviation Safety Metrics, during which the researchers mapped the current practice in safety metrics and explored the validity of monotonic relationships of SMS, activity and demographic metrics with safety outcomes, this report presents the concept for the design of new metrics. Those metrics will be based on the hypothesis that the greater the gap between Work-As-Imagined and Work-As-Done the lower the safety performance, and they correspond to a set of references from academic literature, challenges in professional practice,depiction of system structure, and consideration of “soft” organizational aspects. Along with the design of the alternative metrics, this report explains the respective concepts referred in the literature but excluded from the current research, as well as the process and possible difficulties in ensuring various validity types of the new metrics.
DOCUMENT
Albeit the widespread application of recommender systems (RecSys) in our daily lives, rather limited research has been done on quantifying unfairness and biases present in such systems. Prior work largely focuses on determining whether a RecSys is discriminating or not but does not compute the amount of bias present in these systems. Biased recommendations may lead to decisions that can potentially have adverse effects on individuals, sensitive user groups, and society. Hence, it is important to quantify these biases for fair and safe commercial applications of these systems. This paper focuses on quantifying popularity bias that stems directly from the output of RecSys models, leading to over recommendation of popular items that are likely to be misaligned with user preferences. Four metrics to quantify popularity bias in RescSys over time in dynamic setting across different sensitive user groups have been proposed. These metrics have been demonstrated for four collaborative filteri ng based RecSys algorithms trained on two commonly used benchmark datasets in the literature. Results obtained show that the metrics proposed provide a comprehensive understanding of growing disparities in treatment between sensitive groups over time when used conjointly.
DOCUMENT
Various tools for safety performance measurement have been introduced in order to fulfil the need for safety monitoring in organisations, which is tightly related to their overall performance and achievement of their business goals. Such tools include accident rates, benchmarking, safety culture and climate assessments, cost-effectiveness studies, etc. The current work reviews the most representative methods for safety performance evaluation that have been suggested and applied by a variety of organisations, safety authorities and agencies. This paper discusses several viewpoints of the applicability, feasibility and appropriateness of such tools, based on the viewpoints of managers and safety experts involved in a relevant research that was conducted in a large aviation organisation. The extensive literature cited, the discussion topics, along with the conclusions and recommendations derived, might be considered by any organisation that seeks a realistic safety performance assessment and establishment of effective measurement tools.
DOCUMENT
As part of their SMS, aviation service providers are required to develop and maintain the means to verify the safety performance of their organisation and to validate the effectiveness of safety risk controls. Furthermore, service providers must verify the safety performance of their organisation with reference to the safety performance indicators and safety performance targets of the SMS in support of their organisation’s safety objectives. However, SMEs lack sufficient data to set appropriate safety alerts and targets, or to monitor their performance, and no other objective criteria currently exist to measure the safety of their operations. The Aviation Academy of the Amsterdam University of Applied Sciences therefore took the initiative to develop alternative safety performance metrics. Based on a review of the scientific literature and a survey of existing safety metrics, we proposed several alternative safety metrics. After a review by industry and academia, we developed two alternative metrics into tools to help aviation organisations verify the safety performance of their organisations.The AVAV-SMS tool measures three areas within an organisation’s Safety Management System:• Institutionalisation (design and implementation along with time and internal/external process dependencies).• Capability (the extent to which managers have the capability to implement the SMS).• Effectiveness (the extent to which the SMS deliverables add value to the daily tasks of employees).The tool is scalable to the size and complexity of the organisation, which also makes it useful for small and medium-sized enterprises (SMEs). The AVAS-SCP tool also measures three areas in the organisation’s safety culture prerequisites to foster a positive safety culture:• Organisational plans (whether the company has designed/documented each of the safety cultureprerequisites).• Implementation (the extent to which the prerequisites are realised by the managers/supervisors acrossvarious organisational levels).• Perception (the degree to which frontline employees perceive the effects of managers’ actions relatedto safety culture).We field-tested these tools, demonstrating that they have adequate sensitivity to capture gaps between Work-as-Imagined (WaI) and Work-as-Done (WaD) across organisations. Both tools are therefore useful to organisations that want to self-assess their SMS and safety culture prerequisite levels and proceed to comparisons among various functions and levels and/or over time. Our field testing and observations during the turn-around processes of a regional airline confirm that significant differences exist between WaI and WaD. Although these differences may not automatically be detrimental to safety, gaining insight into them is clearly necessary to manage safety. We conceptually developed safety metrics based on the effectiveness of risk controls. However, these could not be fully field-tested within the scope of this research project. We recommend a continuation of research in this direction. We also explored safety metrics based on the scarcity of resources and system complexity. Again, more research is required here to determine whether these provide viable solutions.
DOCUMENT
Daily wheelchair ambulation is seen as a risk factor for shoulder problems, which are prevalent in manual wheelchair users. To examine the long-term effect of shoulder load from daily wheelchair ambulation on shoulder problems, quantification is required in real-life settings. In this study, we describe and validate a comprehensive and unobtrusive methodology to derive clinically relevant wheelchair mobility metrics (WCMMs) from inertial measurement systems (IMUs) placed on the wheelchair frame and wheel in real-life settings. The set of WCMMs includes distance covered by the wheelchair, linear velocity of the wheelchair, number and duration of pushes, number and magnitude of turns and inclination of the wheelchair when on a slope. Data are collected from ten able-bodied participants, trained in wheelchair-related activities, who followed a 40 min course over the campus. The IMU-derived WCMMs are validated against accepted reference methods such as Smartwheel and video analysis. Intraclass correlation (ICC) is applied to test the reliability of the IMU method. IMU-derived push duration appeared to be less comparable with Smartwheel estimates, as it measures the effect of all energy applied to the wheelchair (including thorax and upper extremity movements), whereas the Smartwheel only measures forces and torques applied by the hand at the rim. All other WCMMs can be reliably estimated from real-life IMU data, with small errors and high ICCs, which opens the way to further examine real-life behavior in wheelchair ambulation with respect to shoulder loading. Moreover, WCMMs can be applied to other applications, including health tracking for individual interest or in therapy settings.
DOCUMENT
A literature review conducted as part of a research project named “Measuring Safety in Aviation – Developing Metrics for Safety Management Systems” revealed several challenges regarding the safety metrics used in aviation. One of the conclusions was that there is limited empirical evidence about the relationship between Safety Management System (SMS) processes and safety outcomes. In order to explore such a relationship, respective data from 7 European airlines was analyzed to explore whether there is a monotonic relation between safety outcome metrics and SMS processes, operational activity and demographic data widely used by the industry. Few, diverse, and occasionally contradictory associations were found, indicating that (1) there is a limited value of linear thinking followed by the industry, i.e., “the more you do with an SMS the higher the safety performance”, (2) the diversity in SMS implementation across companies renders the sole use of output metrics not sufficient for assessing the impact of SMS processes on safety levels, and (3) only flight hours seem as a valid denominator in safety performance indicators. At the next phase of the research project, we are going to explore what alternative metrics can reflect SMS/safety processes and safety performance in a more valid manner
DOCUMENT
A literature review, which was conducted during the research project “Measuring Safety in Aviation – Developing Metrics for Safety Management Systems”, identified several problems and challenges regarding safety performance metrics in aviation. The findings from this review were used to create a framework for interviewing 13 companies in order to explore how safety performance is measured in the industry. The results from the surveys showed a wide variety of approaches for assessing the level of safety. The companies encounter and/or recognise problematic areas in practice when implementing their safety management. The findings from the literature review are partially confirmed and it seems that the current ways of measuring safety performance are not as straight forward as it might be assumed. Further research is recommended to explore alternative methods for measuring aviation safety performance.
DOCUMENT
This study evaluates the effectiveness of the European Union's Corporate Sustainability Reporting Directive (CSRD) in quantitatively measuring the transition of companies to a circular economy. First, using the most recent literature review on circularity metrics, a complete overview of the currently available circularity metrics is developed. Subsequently, it is determined which circularity metrics can be generated with the available quantitative datapoints of CSRD. The metrics that can be generated were analyzed on their ability to cover all circular strategies, to represent different Product-as-a-Service systems and to acknowledge the key role of Critical Raw Materials in a circular economy. The study finds that, with data disclosed under CSRD, metrics can be generated to cover all circular strategies. However, gaps remain in representing pay-per-use and pay-perperformance systems and the use of Critical Raw Materials. Recommendations are to include ‘Product utilization’ and ‘Mass of Critical Raw Materials used’ in the data disclosed under CSRD and to have an independent institution report data to enable benchmarking of performances. Finally, this study concludes with an overview of the metrics which enable to measure circular transitions using data disclosed by CSRD
DOCUMENT
