What is the Role of Audit Checklists in Support of Aviation SMS Performance Audits? Considerations by Sofema Aviation Services (SAS) www.sassofia.com
Checklists are ideal as an “aide memoir” for Quality Assurance Aspects (Prescriptive) however they serve less purpose in determining the effectiveness of an SMS.
Can we perform an SMS Audit against a checklist? For example:
- IATA,
- IS-BAO,
- ICAO,
- Flight Safety Foundation.
The answer in part is yes because the checklist will help us to identify that all the expected/required elements are in place. However to note that a checklist will not address performance issues in the way that we need to if we truly wish to understand the effectiveness of the SMS.
Let’s look at the following checklist question about one element of the SMS Process:
Does the organisation use a risk register? – If the answer is yes then the organisation ticks the box and the audit question is complete (or is it?)
Now consider the following:
- Do you have a risk register?
- How many risks have you identified?
- How many risks do you anticipate should be in your risk register?
- How far along are you in this identification process?
- How are you analysing the identified risks?
- Who is analysing (person or group) against what measure or standard?
- How do assess the competence of the person or persons who are assessing the risks?
- What % of the total exposure do you estimate you have addresses across the business?
- What are your 10 (or 20) Ten Risks?
- Is the leadership team aware of the most significant risks?
- How many actions have been developed against the risks identified?
The above is just an example of the audit techniques which are used for performance auditing.
Note that these are not exactly wrong or right questions, because many of them are “steps on the journey” however what is important, is that the organisation knows where it is on this journey and where it is going, otherwise the SMS becomes a rudderless ship!
Now consider that the organisation is an operator and shares that they have identified 30 Risks. (it is not unusual for an airline to identify & address some 200 – 250 risks! in a strong SMS)
Now you ask questions:
- How many risks are you expecting to review (they advise they do not know)?
- You then ask when you expect the process to be 90 % complete and (again they do not know)?
- This now becomes an issue.
The organisation was not able to demonstrate an effective risk management process related to the need to assess business risk in a timely manner and to initiate appropriate responses – no organisational objectives have been set in this regard.
Note: This issue is not against ICAO/IATA etc it is internal to the organisation – the rational being that you as an organisation must be able to engage fully with the process to determine the Risks which most effect your business process.
So the issue is essentially the organisation has not said what it will do so this in fact forms the basis of the finding – the lack of documented process & procedures!
One more question:
- Do you have sufficient manpower to manage your risk register within the time frame which has been set by the leadership team (no time set – another issue!)?
Next Steps
Follow this link to our Library to find & Download related documents for Free.
Sofema Aviation Services (SAS) www.sassofia.com and Sofema Online (SOL) www.sofemaonline.com Provide more than 50 Aviation Safety System Related Training Courses from Implementation to Accident Investigation. Please see the website or contact us at team@sassofia.com if you want to know more about our services.
Tags:
Audit Checklists, aviation, Aviation Audit, Aviation Quality Assurance, Aviation Risk, Aviation Safety Management System, Flight Safety Foundation, IATA, ICAO, IS-BAO, Risk Register, SAS blogs, SMS, SMS Audit, SMS Performance Audits
