As part of their SMS, aviation service providers are required to develop and maintain the means to verify the safety performance of their organisation and to validate the effectiveness of safety risk controls. Furthermore, service providers must verify the safety performance of their organisation with reference to the safety performance indicators and safety performance targets of the SMS in support of their organisation’s safety objectives. However, SMEs lack sufficient data to set appropriate safety alerts and targets, or to monitor their performance, and no other objective criteria currently exist to measure the safety of their operations. The Aviation Academy of the Amsterdam University of Applied Sciences therefore took the initiative to develop alternative safety performance metrics. Based on a review of the scientific literature and a survey of existing safety metrics, we proposed several alternative safety metrics. After a review by industry and academia, we developed two alternative metrics into tools to help aviation organisations verify the safety performance of their organisations.The AVAV-SMS tool measures three areas within an organisation’s Safety Management System:• Institutionalisation (design and implementation along with time and internal/external process dependencies).• Capability (the extent to which managers have the capability to implement the SMS).• Effectiveness (the extent to which the SMS deliverables add value to the daily tasks of employees).The tool is scalable to the size and complexity of the organisation, which also makes it useful for small and medium-sized enterprises (SMEs). The AVAS-SCP tool also measures three areas in the organisation’s safety culture prerequisites to foster a positive safety culture:• Organisational plans (whether the company has designed/documented each of the safety cultureprerequisites).• Implementation (the extent to which the prerequisites are realised by the managers/supervisors acrossvarious organisational levels).• Perception (the degree to which frontline employees perceive the effects of managers’ actions relatedto safety culture).We field-tested these tools, demonstrating that they have adequate sensitivity to capture gaps between Work-as-Imagined (WaI) and Work-as-Done (WaD) across organisations. Both tools are therefore useful to organisations that want to self-assess their SMS and safety culture prerequisite levels and proceed to comparisons among various functions and levels and/or over time. Our field testing and observations during the turn-around processes of a regional airline confirm that significant differences exist between WaI and WaD. Although these differences may not automatically be detrimental to safety, gaining insight into them is clearly necessary to manage safety. We conceptually developed safety metrics based on the effectiveness of risk controls. However, these could not be fully field-tested within the scope of this research project. We recommend a continuation of research in this direction. We also explored safety metrics based on the scarcity of resources and system complexity. Again, more research is required here to determine whether these provide viable solutions.
In this paper we present a review of existing aviation safety metrics and we lay the foundation for our four-years research project entitled “Measuring Safety in Aviation – Developing Metrics for Safety Management Systems”. We reviewed state-of-the-art literature, relevant standards and regulations, and industry practice. We identified that the long-established view on safety as absence of losses has limited the measurement of safety performance to indicators of adverse events (e.g., accident and incident rates). However, taking into account the sparsity of incidents and accidents compared to the amount of aviation operations, and the recent shift from compliance to performance based approach to safety management, the exclusive use of outcomes metrics does not suffice to further improve safety and establish a proactive monitoring of safety performance. Although the academia and aviation industry have recognized the need to use activity indicators for evaluating how safety management processes perform, and various process metrics have been developed, those have not yet become part of safety performance assessment. This is partly attributed to the lack of empirical evidence about the relation between safety proxies and safety outcomes, and the diversity of safety models used to depict safety management processes (i.e. root-cause, epidemiological or systemic models). This, in turn, has resulted to the development of many safety process metrics, which, however, have not been thoroughly tested against the quality criteria referred in literature, such as validity, reliability and practicality.
A literature review, which was conducted during the research project “Measuring Safety in Aviation – Developing Metrics for Safety Management Systems”, identified several problems and challenges regarding safety performance metrics in aviation. The findings from this review were used to create a framework for interviewing 13 companies in order to explore how safety performance is measured in the industry. The results from the surveys showed a wide variety of approaches for assessing the level of safety. The companies encounter and/or recognise problematic areas in practice when implementing their safety management. The findings from the literature review are partially confirmed and it seems that the current ways of measuring safety performance are not as straight forward as it might be assumed. Further research is recommended to explore alternative methods for measuring aviation safety performance.
In order to stay competitive and respond to the increasing demand for steady and predictable aircraft turnaround times, process optimization has been identified by Maintenance, Repair and Overhaul (MRO) SMEs in the aviation industry as their key element for innovation. Indeed, MRO SMEs have always been looking for options to organize their work as efficient as possible, which often resulted in applying lean business organization solutions. However, their aircraft maintenance processes stay characterized by unpredictable process times and material requirements. Lean business methodologies are unable to change this fact. This problem is often compensated by large buffers in terms of time, personnel and parts, leading to a relatively expensive and inefficient process. To tackle this problem of unpredictability, MRO SMEs want to explore the possibilities of data mining: the exploration and analysis of large quantities of their own historical maintenance data, with the meaning of discovering useful knowledge from seemingly unrelated data. Ideally, it will help predict failures in the maintenance process and thus better anticipate repair times and material requirements. With this, MRO SMEs face two challenges. First, the data they have available is often fragmented and non-transparent, while standardized data availability is a basic requirement for successful data analysis. Second, it is difficult to find meaningful patterns within these data sets because no operative system for data mining exists in the industry. This RAAK MKB project is initiated by the Aviation Academy of the Amsterdam University of Applied Sciences (Hogeschool van Amsterdan, hereinafter: HvA), in direct cooperation with the industry, to help MRO SMEs improve their maintenance process. Its main aim is to develop new knowledge of - and a method for - data mining. To do so, the current state of data presence within MRO SMEs is explored, mapped, categorized, cleaned and prepared. This will result in readable data sets that have predictive value for key elements of the maintenance process. Secondly, analysis principles are developed to interpret this data. These principles are translated into an easy-to-use data mining (IT)tool, helping MRO SMEs to predict their maintenance requirements in terms of costs and time, allowing them to adapt their maintenance process accordingly. In several case studies these products are tested and further improved. This is a resubmission of an earlier proposal dated October 2015 (3rd round) entitled ‘Data mining for MRO process optimization’ (number 2015-03-23M). We believe the merits of the proposal are substantial, and sufficient to be awarded a grant. The text of this submission is essentially unchanged from the previous proposal. Where text has been added – for clarification – this has been marked in yellow. Almost all of these new text parts are taken from our rebuttal (hoor en wederhoor), submitted in January 2016.
Service logistics in de vliegtuigonderhoudindustrie is een zeer kennisintensieve en concurrerende markt. De meest cruciale factor in deze industrie is het laag houden van de downtime tijdens maintenance, repair en overhaulactiviteiten. Met name opslag, distributie en het managen van spare parts spelen hierin een belangrijke rol. Om tijdig vliegtuigen te kunnen onderhouden, hebben onderhoudsbedrijven vaak duizenden onderdelen, van kleine ophangpennen tot zeer dure motoren, op voorraad. Hierin zit dan ook de paradox: onderhoudskosten dalen door lagere down time en grote voorraden zorgen op hun beurt weer voor hoge warehousing kosten. Het lectoraat Aviation Engineering voert thans een RAAK-MKB project uit waarin primair wordt onderzocht of historische onderhoudsdata kan worden gebruikt voor MRO-onderhoudsplannen die de downtime verlagen. Gaandeweg de uitvoering van dit project is echter gebleken dat niet alle onderhoud van te voren gepland kan worden en dat juist real time data tijdens de vlucht erg relevant is voor snel en efficiënt onderhoud. De doelstelling van dit KIEM-project is enerzijds het vergaren van nieuwe kennis en inzichten over service logistics en het daarmee aanjagen nieuw onderzoek waarin wordt onderzocht of de inzet van real time big data bijdraagt aan het verminderen van de downtime. Anderzijds wordt onderzocht of nieuwe samenwerkingen (met IT-bedrijven) mogelijk zijn die voorraadkosten verminderen. Onderzoek wordt gedaan naar: 1. Knelpunten voor de inzet van real time big data in relatie tot MRO-activiteiten. 2. Vraagarticulatie en samenwerkingsmogeljkheden met nieuwe mkb-bedrijven. 3. Spare part warehousing efficiëntie (parts pooling). 4. Infrastructuur en standaarden voor opslag en toegankelijkheid van gezamenlijke en individuele (bedrijfsgevoelige) data. De HvA, NAG en JetSupport verwachten dat met dit project nieuwe mkb-onderhoudsbedrijven, vliegtuigmaatschappijen en overheden gaandeweg het project gaan aanhaken. Uitkomsten zijn enerzijds nieuwe kennis en inzichten op het gebied van service logistics en anderzijds commitment voor een vervolgonderzoek op het lopende RAAK-project.
