Steve Bentley, FRAeS CEO of Sofema Aviation Services Considers the best practices, challenges and additional considerations related to Safety Auditing Methods within Ground Operations
Both SMS compliance auditing and safety system (risk) auditing are crucial for maintaining and improving aviation safety. Compliance auditing ensures the organization meets regulatory standards, but risk auditing goes further by assessing how well the organization manages and mitigates operational risks in practice.
- While compliance audits are essential for certification and regulatory approval, risk audits drive continuous improvement and enhance the proactive safety culture within the organization, addressing emerging risks before they result in incidents.
Introduction
In the context of aviation, Safety Management System (SMS) compliance auditing and safety system (risk) auditing serve distinct but complementary purposes.
Understanding the differences between these two types of audits is essential for ensuring both regulatory compliance and operational safety.
SMS Compliance Auditing
SMS compliance auditing primarily ensures that an organization is adhering to established safety regulations and standards.
- Regulations are defined by authorities such as EASA, FAA, GCAA and GACA, which mandate that operators implement an SMS in accordance with prescribed requirements (e.g., ICAO Annex 19, GACAR Part 5, etc.)
- Compliance auditing revolves around verifying that all the required SMS components are in place and functioning as intended. This includes:
>> Ensuring that SMS manuals, policies, procedures, and processes meet the regulatory requirements.
>> Verifying the existence of documented safety objectives in line with the organization’s safety policy.
>> Confirming that roles and responsibilities (e.g., accountable executive, safety managers) are properly assigned.
>> Ensuring personnel are trained and qualified to perform their safety-related duties as required by the SMS.
>> Assessing the effectiveness of internal safety reporting channels and ensuring compliance with reporting mandates (e.g., voluntary reporting and incident reporting).
>> Checking if key SMS processes like hazard identification, risk assessment, safety performance monitoring, and continuous improvement are being followed in accordance with regulatory guidelines.
>> Compliance audits often follow pre-defined checklists based on regulatory requirements (e.g., GACAR Part 5 ) and relevant industry standards (e.g., ICAO, IATA).
>> A large part of the audit involves reviewing policies, manuals, safety documentation, and training records to confirm that they comply with the applicable regulations.
>> Auditors conduct interviews with personnel (e.g., safety managers and accountable executives) and observe operations to ensure that SMS processes are being implemented as documented.
Compliance Audit Challenges
- SMS compliance audits can sometimes overemphasize paperwork and processes, potentially overlooking how well the SMS is truly integrated into daily operations.
- Keeping up with changing regulations can be a challenge, requiring updates to audit protocols and checklists.
- Employees may view audits as punitive, leading to reluctance to provide accurate information.
Safety System (Risk) Auditing
Safety system (risk) auditing focuses on the effectiveness of an organization’s safety management system by assessing how well it identifies, evaluates, and mitigates risks.
- Rather than simply verifying compliance, this type of audit examines how the SMS is performing in practice, particularly its ability to manage risks to acceptable levels.
- Risk auditing examines the organization’s safety risk management processes and the real-world effectiveness of those processes.
- Evaluating how effectively the organization identifies hazards in its operations (e.g., through safety reports, internal investigations, and hazard tracking systems).
- Assessing the thoroughness and accuracy of the organization’s risk assessments, including how it categorizes risks (e.g., likelihood and severity) and the effectiveness of mitigation strategies.
- Reviewing whether the organization monitors safety performance through data analysis, audits, and trend monitoring to ensure risks remain controlled.
- Examining how the organization uses data from incidents, audits, and safety performance indicators to improve its risk management processes.
Risk-Based Auditing:
- Focuses on auditing high-risk areas (e.g., runway incursions, fatigue management, ground handling risks) where potential safety issues could have the most severe consequences.
- Risk audits heavily rely on safety performance data, such as incident reports, safety performance indicators (SPIs), and real-time operational data, to assess the current state of risk within the organization.
- Auditors are able to use real-world or simulated scenarios to test how well the safety system handles emerging risks, such as unanticipated weather conditions or equipment failures.
- Auditors observe operations in real-time, identifying hazards and assessing whether the organization effectively mitigates risks during actual operations.
- The audit results in the identification of potential risks and assesses how well the organization manages these risks in real-time operations.
- Risk audits typically lead to recommendations for improving risk management processes, strengthening safety controls, and closing gaps in hazard identification.
- Audits should not only assess compliance but also evaluate the effectiveness of the organization’s risk management processes.
- Auditors should look for evidence of proactive hazard identification, risk assessment, and risk mitigation as part of the SMS.
Challenges
- Risk auditing often involves analyzing complex data from various sources (e.g., flight data monitoring, SMS reports, safety performance indicators), which requires auditors to have advanced analytical skills.
- Some organizations may be resistant to risk audits (culture-related), especially if they expose weaknesses in safety controls that require significant operational changes.
- Unlike compliance audits, risk audits involve a certain degree of subjectivity, as assessing the effectiveness of risk management strategies can be challenging without clear benchmarks.
- The organization needs to revise its safety risk profile or adjust its risk controls based on the audit’s findings.
Key Differences Between SMS Compliance Auditing and Safety System (Risk) Auditing
| Aspect | SMS Compliance Auditing | Safety System (Risk) Auditing |
| Purpose | Ensures adherence to regulatory requirements and standards | Assesses the effectiveness of risk management processes within the SMS |
| Focus | Focuses on compliance with regulations, documentation, and procedures | Focuses on the effectiveness of risk identification, mitigation, and safety performance |
| Scope | Covers all components of the SMS, ensuring processes and policies meet regulations | Focuses on specific risks within the organization, such as operational, human, or environmental risks |
| Methodology | Checklist-based, regulatory compliance-focused (e.g., reviewing manuals, policies, training records) | Risk-based, data-driven, and scenario-based (e.g., analyzing incident trends, risk controls, real-world observations) |
| Outcomes | Results in conformance or non-conformance based on regulatory compliance | Provides insights into how well the organization manages risks, leading to safety improvements |
| Reporting | Typically leads to corrective actions for non-compliance | This leads to risk management recommendations, often more qualitative and focused on operational safety improvements |
| Regulatory Involvement | Conducted to meet regulatory obligations (e.g., ICAO, EASA, GACAR) | Conducted to improve safety performance and mitigate risks beyond compliance |
| Challenges | Staying up-to-date with changing regulations, often documentation-heavy | Requires deep data analysis, can be subjective, and involves cultural and operational change |
Best Practices, Challenges, and Additional Considerations
- Focus auditing efforts on high-risk areas to ensure the efficient use of resources and maximum safety impact.
- Conduct a risk assessment before the audit to identify high-priority areas based on historical data (e.g., incident reports, near-miss reports).
- Use tools like Safety Risk Profiles and Bowtie Models to visualize risks and pinpoint areas for detailed review.
- Focus on critical risk areas, such as runway incursions, ground handling risks (e.g., foreign object debris or loading errors), and human factors.
- Align the audit scope with key performance indicators (KPIs) defined in the SMS.
- Periodically revise the risk assessment based on new data (e.g., new hazards, changes in operations, regulatory updates).
Audit Planning
- Ensure audits are well-organized, cover all SMS components, and are conducted within predefined intervals.
- Develop a comprehensive audit plan that details timelines, audit areas, personnel, and tools.
-
- Assign competent personnel with specific knowledge of the audit scope (e.g., flight operations, maintenance, ground handling).
- Gather comprehensive data during the audit to accurately assess compliance and safety performance.
-
- Use a mix of direct observation (e.g., observing ramp operations) and interviews with frontline staff to capture practical insights.
- Review documentation such as SMS manuals, hazard reports, training records, and safety performance reports.
- Apply sampling techniques (e.g., random sampling of safety reports or maintenance logs) to ensure data quality without overwhelming resources.
- Prioritize non-intrusive observation to avoid operational disruptions.
- Maintain an ongoing oversight of the SMS to proactively identify and address emerging risks.
-
- Use dashboards that provide real-time monitoring of safety performance and audit status.
- Ensure management involvement in audit reporting and follow-up to drive accountability.
- Integrate audit findings into safety risk management processes, updating the SMS where necessary.
Next Steps
Follow this link to our Library to find & download related documents for Free.
For more information and training support on SMS within KSA, consider the course GACAR Part 5 – SMS Implementation Review, Development, and Risk Management Processes – 5 Days. For questions and comments, please email: team@sassofia.com.
Tags:
best practices, Additional Considerations, Audit Planning, afety System (Risk) Auditing, SMS Compliance Auditing, safety system (risk), Aviation Ground Operations SMS, Safety Auditing Methods, GACAR Part 5, SMS Practical Safety Auditing, EASA, Risk-Based Auditing, ICAO Annex 19, SAS blogs, GACA, FAA, GCAA, Safety Managers, Safety Management System SMS
