Following the completion of the 1st phase of the RAAK PRO project Aviation Safety Metrics, during which the researchers mapped the current practice in safety metrics and explored the validity of monotonic relationships of SMS, activity and demographic metrics with safety outcomes, this report presents the concept for the design of new metrics. Those metrics will be based on the hypothesis that the greater the gap between Work-As-Imagined and Work-As-Done the lower the safety performance, and they correspond to a set of references from academic literature, challenges in professional practice,depiction of system structure, and consideration of “soft” organizational aspects. Along with the design of the alternative metrics, this report explains the respective concepts referred in the literature but excluded from the current research, as well as the process and possible difficulties in ensuring various validity types of the new metrics.
The recently introduced Risk SituatiOn Awareness Provision (RiskSOAP) methodology suggested an indicator to measure the distance between the configuration of a real system and its ideal version or between various system versions. It considers the (in)existence or (mal)functioning of system components, processes and connections based on a binary approach. However, in practice safety requirements can be fulfilled to some degree and each system component might have a different impact on system outcomes. This work suggests the Comparing System Configurations (COSYCO) indicator which introduces (1) the use of continuous values for the behaviour of system components, (2) the inclusion of weights according to the hierarchal system level to which each component belongs, and (3) the consideration of the outgoing connections of each component with other system components. Both RiskSOAP and COSYCO are based on the STPA hazard analysis which is a systematic technique used to define the components and the requirements that the system should ideally meet to achieve its objectives. To demonstrate the applicability and sensitivity of COSYCO, we applied it to a published case for small drones. Drones with same overall differences in the satisfaction of requirements concluded to different values when applying COSYCO, indicating the increased sensitivity of the specific indicator when compared to the RiskSOAP. We envisage that the metric proposed in this work is a first practical and realistic approach to the quantification of the distance between the optimal design and current system states as well amongst various systems and their versions over time.
