A literature review conducted as part of a research project named “Measuring Safety in Aviation – Developing Metrics for Safety Management Systems” revealed several challenges regarding the safety metrics used in aviation. One of the conclusions was that there is limited empirical evidence about the relationship between Safety Management System (SMS) processes and safety outcomes. In order to explore such a relationship, respective data from 7 European airlines was analyzed to explore whether there is a monotonic relation between safety outcome metrics and SMS processes, operational activity and demographic data widely used by the industry. Few, diverse, and occasionally contradictory associations were found, indicating that (1) there is a limited value of linear thinking followed by the industry, i.e., “the more you do with an SMS the higher the safety performance”, (2) the diversity in SMS implementation across companies renders the sole use of output metrics not sufficient for assessing the impact of SMS processes on safety levels, and (3) only flight hours seem as a valid denominator in safety performance indicators. At the next phase of the research project, we are going to explore what alternative metrics can reflect SMS/safety processes and safety performance in a more valid manner
In this paper we present a review of existing aviation safety metrics and we lay the foundation for our four-years research project entitled “Measuring Safety in Aviation – Developing Metrics for Safety Management Systems”. We reviewed state-of-the-art literature, relevant standards and regulations, and industry practice. We identified that the long-established view on safety as absence of losses has limited the measurement of safety performance to indicators of adverse events (e.g., accident and incident rates). However, taking into account the sparsity of incidents and accidents compared to the amount of aviation operations, and the recent shift from compliance to performance based approach to safety management, the exclusive use of outcomes metrics does not suffice to further improve safety and establish a proactive monitoring of safety performance. Although the academia and aviation industry have recognized the need to use activity indicators for evaluating how safety management processes perform, and various process metrics have been developed, those have not yet become part of safety performance assessment. This is partly attributed to the lack of empirical evidence about the relation between safety proxies and safety outcomes, and the diversity of safety models used to depict safety management processes (i.e. root-cause, epidemiological or systemic models). This, in turn, has resulted to the development of many safety process metrics, which, however, have not been thoroughly tested against the quality criteria referred in literature, such as validity, reliability and practicality.
A literature review, which was conducted during the research project “Measuring Safety in Aviation – Developing Metrics for Safety Management Systems”, identified several problems and challenges regarding safety performance metrics in aviation. The findings from this review were used to create a framework for interviewing 13 companies in order to explore how safety performance is measured in the industry. The results from the surveys showed a wide variety of approaches for assessing the level of safety. The companies encounter and/or recognise problematic areas in practice when implementing their safety management. The findings from the literature review are partially confirmed and it seems that the current ways of measuring safety performance are not as straight forward as it might be assumed. Further research is recommended to explore alternative methods for measuring aviation safety performance.
In order to stay competitive and respond to the increasing demand for steady and predictable aircraft turnaround times, process optimization has been identified by Maintenance, Repair and Overhaul (MRO) SMEs in the aviation industry as their key element for innovation. Indeed, MRO SMEs have always been looking for options to organize their work as efficient as possible, which often resulted in applying lean business organization solutions. However, their aircraft maintenance processes stay characterized by unpredictable process times and material requirements. Lean business methodologies are unable to change this fact. This problem is often compensated by large buffers in terms of time, personnel and parts, leading to a relatively expensive and inefficient process. To tackle this problem of unpredictability, MRO SMEs want to explore the possibilities of data mining: the exploration and analysis of large quantities of their own historical maintenance data, with the meaning of discovering useful knowledge from seemingly unrelated data. Ideally, it will help predict failures in the maintenance process and thus better anticipate repair times and material requirements. With this, MRO SMEs face two challenges. First, the data they have available is often fragmented and non-transparent, while standardized data availability is a basic requirement for successful data analysis. Second, it is difficult to find meaningful patterns within these data sets because no operative system for data mining exists in the industry. This RAAK MKB project is initiated by the Aviation Academy of the Amsterdam University of Applied Sciences (Hogeschool van Amsterdan, hereinafter: HvA), in direct cooperation with the industry, to help MRO SMEs improve their maintenance process. Its main aim is to develop new knowledge of - and a method for - data mining. To do so, the current state of data presence within MRO SMEs is explored, mapped, categorized, cleaned and prepared. This will result in readable data sets that have predictive value for key elements of the maintenance process. Secondly, analysis principles are developed to interpret this data. These principles are translated into an easy-to-use data mining (IT)tool, helping MRO SMEs to predict their maintenance requirements in terms of costs and time, allowing them to adapt their maintenance process accordingly. In several case studies these products are tested and further improved. This is a resubmission of an earlier proposal dated October 2015 (3rd round) entitled ‘Data mining for MRO process optimization’ (number 2015-03-23M). We believe the merits of the proposal are substantial, and sufficient to be awarded a grant. The text of this submission is essentially unchanged from the previous proposal. Where text has been added – for clarification – this has been marked in yellow. Almost all of these new text parts are taken from our rebuttal (hoor en wederhoor), submitted in January 2016.
Service logistics in de vliegtuigonderhoudindustrie is een zeer kennisintensieve en concurrerende markt. De meest cruciale factor in deze industrie is het laag houden van de downtime tijdens maintenance, repair en overhaulactiviteiten. Met name opslag, distributie en het managen van spare parts spelen hierin een belangrijke rol. Om tijdig vliegtuigen te kunnen onderhouden, hebben onderhoudsbedrijven vaak duizenden onderdelen, van kleine ophangpennen tot zeer dure motoren, op voorraad. Hierin zit dan ook de paradox: onderhoudskosten dalen door lagere down time en grote voorraden zorgen op hun beurt weer voor hoge warehousing kosten. Het lectoraat Aviation Engineering voert thans een RAAK-MKB project uit waarin primair wordt onderzocht of historische onderhoudsdata kan worden gebruikt voor MRO-onderhoudsplannen die de downtime verlagen. Gaandeweg de uitvoering van dit project is echter gebleken dat niet alle onderhoud van te voren gepland kan worden en dat juist real time data tijdens de vlucht erg relevant is voor snel en efficiënt onderhoud. De doelstelling van dit KIEM-project is enerzijds het vergaren van nieuwe kennis en inzichten over service logistics en het daarmee aanjagen nieuw onderzoek waarin wordt onderzocht of de inzet van real time big data bijdraagt aan het verminderen van de downtime. Anderzijds wordt onderzocht of nieuwe samenwerkingen (met IT-bedrijven) mogelijk zijn die voorraadkosten verminderen. Onderzoek wordt gedaan naar: 1. Knelpunten voor de inzet van real time big data in relatie tot MRO-activiteiten. 2. Vraagarticulatie en samenwerkingsmogeljkheden met nieuwe mkb-bedrijven. 3. Spare part warehousing efficiëntie (parts pooling). 4. Infrastructuur en standaarden voor opslag en toegankelijkheid van gezamenlijke en individuele (bedrijfsgevoelige) data. De HvA, NAG en JetSupport verwachten dat met dit project nieuwe mkb-onderhoudsbedrijven, vliegtuigmaatschappijen en overheden gaandeweg het project gaan aanhaken. Uitkomsten zijn enerzijds nieuwe kennis en inzichten op het gebied van service logistics en anderzijds commitment voor een vervolgonderzoek op het lopende RAAK-project.
