Search results

Measuring safety in aviation

Frequency and Variance of Communication Characteristics in Aviation Safety Events

In the aviation sector, communication problems have contributed into 70% to 80% of safety occurrences. However, to date we haven’t depicted which communication aspects have affected aviation safety most frequently.Based on literature, we developed a tool which includes communication characteristics related to actors, signal, coder, channel, decoder, direction, timing, distance, predictability and interference. After achieving inter-rater reliability, the tool was used to analyse 103 safety investigation reports that correspond to events occurred in various regions and which included in total 256 communication problems. The results suggest that communication between humans and representation media, visual and audio signalling and decoding, air-transmitted messages, and verbal, unidirectional, local and synchronous communication contributed most frequently into safety events. Statistical tests showed that the frequencies of most of those characteristics were significantly different across regions, time periods, types of operations and event severity.The tool developed can be used by different organizations and industry sectors to distil and analyse data from mandatory and voluntary reports and identify weak communication areas. Depending on the findings, analysts might need to alert designers of technical systems, inform management of organizations, warn end-users about most frequent pitfalls, modify/enrich communication training and steer research efforts.

How does aviation industry measure safety performance?

In this paper we present a review of existing aviation safety metrics and we lay the foundation for our four-years research project entitled “Measuring Safety in Aviation – Developing Metrics for Safety Management Systems”. We reviewed state-of-the-art literature, relevant standards and regulations, and industry practice. We identified that the long-established view on safety as absence of losses has limited the measurement of safety performance to indicators of adverse events (e.g., accident and incident rates). However, taking into account the sparsity of incidents and accidents compared to the amount of aviation operations, and the recent shift from compliance to performance based approach to safety management, the exclusive use of outcomes metrics does not suffice to further improve safety and establish a proactive monitoring of safety performance. Although the academia and aviation industry have recognized the need to use activity indicators for evaluating how safety management processes perform, and various process metrics have been developed, those have not yet become part of safety performance assessment. This is partly attributed to the lack of empirical evidence about the relation between safety proxies and safety outcomes, and the diversity of safety models used to depict safety management processes (i.e. root-cause, epidemiological or systemic models). This, in turn, has resulted to the development of many safety process metrics, which, however, have not been thoroughly tested against the quality criteria referred in literature, such as validity, reliability and practicality.

Measuring safety in aviation

As part of their SMS, aviation service providers are required to develop and maintain the means to verify the safety performance of their organisation and to validate the effectiveness of safety risk controls. Furthermore, service providers must verify the safety performance of their organisation with reference to the safety performance indicators and safety performance targets of the SMS in support of their organisation’s safety objectives. However, SMEs lack sufficient data to set appropriate safety alerts and targets, or to monitor their performance, and no other objective criteria currently exist to measure the safety of their operations. The Aviation Academy of the Amsterdam University of Applied Sciences therefore took the initiative to develop alternative safety performance metrics. Based on a review of the scientific literature and a survey of existing safety metrics, we proposed several alternative safety metrics. After a review by industry and academia, we developed two alternative metrics into tools to help aviation organisations verify the safety performance of their organisations.The AVAV-SMS tool measures three areas within an organisation’s Safety Management System:• Institutionalisation (design and implementation along with time and internal/external process dependencies).• Capability (the extent to which managers have the capability to implement the SMS).• Effectiveness (the extent to which the SMS deliverables add value to the daily tasks of employees).The tool is scalable to the size and complexity of the organisation, which also makes it useful for small and medium-sized enterprises (SMEs). The AVAS-SCP tool also measures three areas in the organisation’s safety culture prerequisites to foster a positive safety culture:• Organisational plans (whether the company has designed/documented each of the safety cultureprerequisites).• Implementation (the extent to which the prerequisites are realised by the managers/supervisors acrossvarious organisational levels).• Perception (the degree to which frontline employees perceive the effects of managers’ actions relatedto safety culture).We field-tested these tools, demonstrating that they have adequate sensitivity to capture gaps between Work-as-Imagined (WaI) and Work-as-Done (WaD) across organisations. Both tools are therefore useful to organisations that want to self-assess their SMS and safety culture prerequisite levels and proceed to comparisons among various functions and levels and/or over time. Our field testing and observations during the turn-around processes of a regional airline confirm that significant differences exist between WaI and WaD. Although these differences may not automatically be detrimental to safety, gaining insight into them is clearly necessary to manage safety. We conceptually developed safety metrics based on the effectiveness of risk controls. However, these could not be fully field-tested within the scope of this research project. We recommend a continuation of research in this direction. We also explored safety metrics based on the scarcity of resources and system complexity. Again, more research is required here to determine whether these provide viable solutions.

Design, scope and focus of safety recommendations

Literature and industry standards do not mention inclusive guidelines to generate safety recommendations. Following a literature review, we suggest nine design criteria as well as the classification of safety recommendations according to their scope (i.e. organisational context, stakeholders addressed and degree of change) and their focus, the latter corresponding to the type of risk barrier introduced. The design and classification criteria were applied to 625 recommendations published by four aviation investigation agencies. The analysis results suggested sufficient implementation of most of the design criteria. Concerning their scope, the findings showed an emphasis on processes and structures (i.e. lower organisational contexts), adaptations that correspond to medium degree of changes, and local stakeholders. Regarding the focus of the recommendations, non-technical barriers that rely mostly on employees’ interpretation were introduced by the vast majority of safety recommendations. Also, statistically significant differences were detected across investigation authorities and time periods. This study demonstrated how the application of the suggested design and classification frameworks could reveal valuable information about the quality, scope and focus of recommendations. Especially the design criteria could function as a starting point towards the introduction of a common standard to be used at local, national and international levels.

Review of existing aviation safety metrics