Measuring Safety In Aviation – Developing Metrics For Safety Management SystemsProject
The project “Measuring safety in aviation – developing metrics for Safety Management Systems” will be executed by the Aviation Academy of the Amsterdam University of Applied Sciences with a wide range of partners, including authorities, universities and companies. The project will last from September 2015 to August 2019 and be financed through a RAAK Pro grant of the Dutch Ministry of Education.
The scope of the project is to:
- Assess the suitability of existing safety performance indicators;
- Develop a long list of metrics for safety management by modelling the operations of different areas of aviation into individual components and its interactions;
- Generate and validate a short list of suitable metrics for representing safety performance through safety management performance;
- Translate this knowledge into a web-based tool for the industry to evaluate safety in a more objective way than is currently possible.
Currently, the relation between safety management activities carried out by aviation organizations and their achievements in safety performance is anecdotal; companies do not actually know what effect their safety management has on their safety outcomes. This makes it impossible for them to define and assess safety risks that exist under their current safety management approach. They assume certain measures warrant safety, and they adjust their assumptions when these proven wrong. In order to improve safety in aviation new international regulations and guidelines for the management of safety have been introduced to the industry: the so-called Safety Management Systems (SMS). The aim of SMS is to shift safety management to a concept that, next to compliance to safety regulations, also provides evidence of a link between their safety management activities and performance. If companies know the effect their safety management has on safety outcomes, they have insight in what the risks are and to what extent these exist. By being able to “predict” their safety performance, organizations can adjust their safety management to these predictions before incidents occur, resulting thus, in improved safety performance. Moreover, companies will be able to claim to the authorities that their SMS processes are effective.
It is commonly assumed that the “effective” management of safety is correlated with safety performance; however this has not yet been scientifically validated. This study aims to validate the relation between the effective management of safety according to the systemic models, and safety performance defined as “the ability to succeed under varying conditions”. Such validation will allow companies that do not have the benefit of enough direct safety data, representing their safety performance through their safety management performance.
In this framework, safety performance (the dependent variable) will be operationalized initially by employing the traditional and often reactive indicators, such as accident or near-miss rates. Based on recent literature, this study will attempt to operationalize safety performance as “the ability to succeed under varying conditions” (Safety II), and develop in new indicators. Regarding the operationalization of safety management (the independent variable), different methods are available, including compliance-based guidance from ICAO and EASA and more promising novel systemic models such as FRAM and STAMP. Each of these will be evaluated to make an inventory of existing metrics and/or generate new metrics for safety management that are to be correlated with the indicators for safety performance.
The partners who will contribute to the project are: Netherlands Aerospace Group (NAG), JetSupport BV, Nationaal Lucht- en Ruimtevaartlaboratorium (NLR), Koninklijke Luchtvaart Maatschappij NV (KLM), Ministerie voor Infrastructuur en Milieu, Sky Service Netherlands BV, SAMCO Aircraft Maintenance BV, Koninklijke Nederlandse Vereniging voor Luchtvaart (KNVvL), HeliCentre BV, Transavia, Luchtverkeersleiding Nederland (LVNL), KLM Cityhopper, Team HF PartG, Griffith University – Safety Science Innovation Lab, Delft University of Technology – Safety & Security Science Group, Militaire Luchtvaart Autoriteit (MLA), STg. Human Factors in Aviation Group (HUFAG).
The research project (from left to right on the photo below):
- Robbert van Aalst
- Steffen Kaspers
- Dr. Nektarios Karanikas
- Dr. Alfred Roelen
- Selma Piric
Following the initiation of the core project activities after the kick-off meeting, the research team, with the support of the expert knowledge group, completed a review about the existing safety metrics in aviation. The results of the review were highly interesting and mostly valuable for the next phases of the project. We concluded that risk and safety are differently defined even at the level of international bodies, rendering thus difficult a harmonized monitoring framework through metrics. We indicated a plethora of safety metrics proposed by literature or/and applied in the aviation industry, the validity of which has not been yet fully studied. We envisaged that systemic safety models have the potential to complement the existing ones in terms of developing new safety metrics.
In overall, the review confirmed the need for this research project and set the foundation for the next steps. The report will be sent to all partners next month after its approval by the steering committee.