As part of their SMS, aviation service providers are required to develop and maintain the means to verify the safety performance of their organisation and to validate the effectiveness of safety risk controls. Furthermore, service providers must verify the safety performance of their organisation with reference to the safety performance indicators and safety performance targets of the SMS in support of their organisation’s safety objectives. However, SMEs lack sufficient data to set appropriate safety alerts and targets, or to monitor their performance, and no other objective criteria currently exist to measure the safety of their operations. The Aviation Academy of the Amsterdam University of Applied Sciences therefore took the initiative to develop alternative safety performance metrics. Based on a review of the scientific literature and a survey of existing safety metrics, we proposed several alternative safety metrics. After a review by industry and academia, we developed two alternative metrics into tools to help aviation organisations verify the safety performance of their organisations.The AVAV-SMS tool measures three areas within an organisation’s Safety Management System:• Institutionalisation (design and implementation along with time and internal/external process dependencies).• Capability (the extent to which managers have the capability to implement the SMS).• Effectiveness (the extent to which the SMS deliverables add value to the daily tasks of employees).The tool is scalable to the size and complexity of the organisation, which also makes it useful for small and medium-sized enterprises (SMEs). The AVAS-SCP tool also measures three areas in the organisation’s safety culture prerequisites to foster a positive safety culture:• Organisational plans (whether the company has designed/documented each of the safety cultureprerequisites).• Implementation (the extent to which the prerequisites are realised by the managers/supervisors acrossvarious organisational levels).• Perception (the degree to which frontline employees perceive the effects of managers’ actions relatedto safety culture).We field-tested these tools, demonstrating that they have adequate sensitivity to capture gaps between Work-as-Imagined (WaI) and Work-as-Done (WaD) across organisations. Both tools are therefore useful to organisations that want to self-assess their SMS and safety culture prerequisite levels and proceed to comparisons among various functions and levels and/or over time. Our field testing and observations during the turn-around processes of a regional airline confirm that significant differences exist between WaI and WaD. Although these differences may not automatically be detrimental to safety, gaining insight into them is clearly necessary to manage safety. We conceptually developed safety metrics based on the effectiveness of risk controls. However, these could not be fully field-tested within the scope of this research project. We recommend a continuation of research in this direction. We also explored safety metrics based on the scarcity of resources and system complexity. Again, more research is required here to determine whether these provide viable solutions.
DOCUMENT
A literature review conducted as part of a research project named “Measuring Safety in Aviation – Developing Metrics for Safety Management Systems” revealed several challenges regarding the safety metrics used in aviation. One of the conclusions was that there is limited empirical evidence about the relationship between Safety Management System (SMS) processes and safety outcomes. In order to explore such a relationship, respective data from 7 European airlines was analyzed to explore whether there is a monotonic relation between safety outcome metrics and SMS processes, operational activity and demographic data widely used by the industry. Few, diverse, and occasionally contradictory associations were found, indicating that (1) there is a limited value of linear thinking followed by the industry, i.e., “the more you do with an SMS the higher the safety performance”, (2) the diversity in SMS implementation across companies renders the sole use of output metrics not sufficient for assessing the impact of SMS processes on safety levels, and (3) only flight hours seem as a valid denominator in safety performance indicators. At the next phase of the research project, we are going to explore what alternative metrics can reflect SMS/safety processes and safety performance in a more valid manner
DOCUMENT
Various tools for safety performance measurement have been introduced in order to fulfil the need for safety monitoring in organisations, which is tightly related to their overall performance and achievement of their business goals. Such tools include accident rates, benchmarking, safety culture and climate assessments, cost-effectiveness studies, etc. The current work reviews the most representative methods for safety performance evaluation that have been suggested and applied by a variety of organisations, safety authorities and agencies. This paper discusses several viewpoints of the applicability, feasibility and appropriateness of such tools, based on the viewpoints of managers and safety experts involved in a relevant research that was conducted in a large aviation organisation. The extensive literature cited, the discussion topics, along with the conclusions and recommendations derived, might be considered by any organisation that seeks a realistic safety performance assessment and establishment of effective measurement tools.
DOCUMENT
Aanleiding De luchtvaart wordt steeds veiliger. Toch zijn er alleen al in Nederland jaarlijks zo'n 11.000 issues met luchtvaartveiligheid. Wereldwijd vinden er elke dag ongelukken plaats die leiden tot schade aan vliegtuigen. Om de veiligheid verder te verbeteren is er nieuwe internationale regelgeving opgesteld. Onder deze regels moeten de maatschappijen alle incidenten en ongelukken analyseren en zo veiligheidsrisico's identificeren nog voordat deze zich voordoen. Het probleem is dat kleine en middelgrote luchtvaartmaatschappijen onvoldoende vliegbewegingen maken om genoeg goede data hiervoor te hebben. Doelstelling De centrale vraag die de onderzoekers in dit RAAK-project willen beantwoorden: Wat is de relatie tussen veiligheidsmanagement en veiligheidsperformance van luchtvaartmaatschappijen? Het onderzoek wil kleine en middelgrote luchtvaartmaatschappijen helpen bij het meten van de veiligheid van hun bedrijf, zonder dat ze grote hoeveelheden veiligheidsdata tot hun beschikking hebben. Het onderzoek zal geschikte veiligheidsindicatoren identificeren, een longlist ontwikkelen met meetwaarden voor safetymanagement, en een shortlist genereren en valideren van bruikbare meetwaarden. Deze kennis wordt vertaald in een online dashboard voor de industrie, zodat de veiligheid objectiever beoordeeld kan worden. Beoogde resultaten Een concreet resultaat van dit project is een online dashboard waarmee kleine en middelgrote luchtvaartmaatschappijen hun veiligheid kunnen beoordelen, inclusief handleiding. Er zullen masterclasses veiligheid worden georganiseerd voor de luchtvaartindustrie. Het projectteam zal de opgedane kennis verspreiden via wetenschappelijke artikelen in relevante peer-reviewed tijdschriften, een website, presentaties bij bedrijven en tijdens bijeenkomsten, en een afsluitende conferentie.
