Following the completion of the 1st phase of the RAAK PRO project Aviation Safety Metrics, during which the researchers mapped the current practice in safety metrics and explored the validity of monotonic relationships of SMS, activity and demographic metrics with safety outcomes, this report presents the concept for the design of new metrics. Those metrics will be based on the hypothesis that the greater the gap between Work-As-Imagined and Work-As-Done the lower the safety performance, and they correspond to a set of references from academic literature, challenges in professional practice,depiction of system structure, and consideration of “soft” organizational aspects. Along with the design of the alternative metrics, this report explains the respective concepts referred in the literature but excluded from the current research, as well as the process and possible difficulties in ensuring various validity types of the new metrics.
The recently introduced Risk SituatiOn Awareness Provision (RiskSOAP) methodology suggested an indicator to measure the distance between the configuration of a real system and its ideal version or between various system versions. It considers the (in)existence or (mal)functioning of system components, processes and connections based on a binary approach. However, in practice safety requirements can be fulfilled to some degree and each system component might have a different impact on system outcomes. This work suggests the Comparing System Configurations (COSYCO) indicator which introduces (1) the use of continuous values for the behaviour of system components, (2) the inclusion of weights according to the hierarchal system level to which each component belongs, and (3) the consideration of the outgoing connections of each component with other system components. Both RiskSOAP and COSYCO are based on the STPA hazard analysis which is a systematic technique used to define the components and the requirements that the system should ideally meet to achieve its objectives. To demonstrate the applicability and sensitivity of COSYCO, we applied it to a published case for small drones. Drones with same overall differences in the satisfaction of requirements concluded to different values when applying COSYCO, indicating the increased sensitivity of the specific indicator when compared to the RiskSOAP. We envisage that the metric proposed in this work is a first practical and realistic approach to the quantification of the distance between the optimal design and current system states as well amongst various systems and their versions over time.
The remarkable and continuous growth of the unmanned aircraft market has brought new safety related challenges, as those are recorded in various accident and incident reports. Although drones with an operating weight higher than 20-25Kgs are technologically advanced and often subject to standards (e.g., technical reliability, airspace management, licensing, certification), the regulatory framework for (ultra) light drones focuses almost exclusively on the limitations that the operator needs to consider. Thus, the protection from accidents seems to rely mostly on the competency of the operator to fly a drone safely, and his/her observance of the rules published by the respective authorities. In addition, the hazards lying in the interaction between an operator and a small drone have not been systematically studied. In this paper, we present (1) the first results from a System-Theoretic Process Analysis (STPA) based approach to the identification of hazards and safety requirements in small drone operations, and (2) an adaptation of the Risk Situation Awareness Provision Capability (RiskSOAP) methodology in order to quantify the differences amongst 4 drone models regarding the extent to which they fulfill the safety requirements identified through STPA. The results showed that the drones studied satisfy the safety requirements at low and moderate levels and they present high dissimilarities between them regarding the extent to which they meet the same safety requirements. Future work will include: (a) comparison of a larger sample of small drones against the safety requirements, as well as pairwise, and (b) assessment of the degree to which various regulatory frameworks worldwide address the safety requirements generated with STPA and assigned to the authority level.
