Sofema Aviation Services (SAS) considers MSAT Assessment, Compliance and Effectiveness, with Scalability & Suitability in Mind
Introduction to Driving Effectiveness – Start from Purpose, not Paperwork
- Dual aim: MSAT is explicitly for both compliance and performance. Use it to confirm processes are Present & Suitable and to judge whether they operate and are Effective in delivering safety outcomes.
Important – Avoid a Checklist Mentality: Treat the “what to look for” lines as prompts, not ticks. The objective is to understand how safe the operation actually is and whether major risks are being missed.
Discussion Question – Have you experienced an organisation appearing compliant on paper yet clearly underperform on risk control? What evidence changed your view?
Distinguish Operating from Effective with targeted evidence
- Integrate all Available Information (docs, interviews/observation, data): For “present”, documents may suffice; “operating” needs observed use and records; “effective” requires outcome evidence.
- Outcome Focus for “Effective”: Focus summaries and reports on proactive risk management, safety performance, data intelligence, resilience to disruption, and overall maturity.
- Considering Findings vs Observations: If a process is operating but not (yet) effective, this often warrants an observation (improvement signal), not an automatic finding—unless leadership keeps ignoring evidence the MS is underperforming.
Discussion Question: What extra proof would move your judgment from Operating to Effective?
Make Suitability & Scalability Explicit (before and after the assessment)
- Pre Review: Analyse organisational context before the review, and re-check whether the overall MS is commensurate after you’ve seen it working.
- What scalability is (and isn’t): All MS elements must exist; scalability is adapting depth and complexity to the organisation’s size, interfaces and risks—not creating a “light” MS or dropping core functions.
- Context factors to weigh: size (people, sites, interfaces), approvals held, operational profile, environment, and inherent risk exposures.
- Applicability / Customisation: The tool works across domains; tailor it to sector specifics where needed.
Discussion Question: Consider a small but high-hazard operation (e.g., offshore support). Which two processes must be deeper than their size suggests? Which areas could be leaner without losing control? (If any?)
Integrated approvals: one system, one evaluation—coordinated where needed
- Principle: If the organisation runs an integrated MS for multiple certificates, assess it as one system. Where separate domain teams must be involved, coordinate and share a single, coherent oversight message.
- Why it matters: Integration affects interface management, flow of risk data, and change control—core drivers of effectiveness rather than mere presence.
Discussion Question: In an integrated CAMO + Part-145 MS, what cross-domain evidence would convince you risk controls are effective (not just “present” twice)?
Practical techniques to surface effectiveness (without audit fatigue)
- Sequence smartly: For mature orgs, start with Safety Risk Management and Assurance: “What are your main risks? How do you know? What changed after the mitigations? Are safety objectives being achieved?”
- Credit existing oversight: Bring in intelligence from other audits, investigations, and meetings to inform your maturity view—this is built into tool usage.
- Summarize what matters: Close each review with a performance-oriented assessment summary—strongest processes, ineffective controls, emerging risks, and progress against safety objectives (and any linkage to SSP/SPAS).
Organisation archetypes: apply proportionality without dilution
Use the prompts below to compare Present/Suitable/Operating/Effective across contexts. The point is to test scalability (depth) and suitability (fit) while preserving all MS elements.
Small, focused Part-145 line station (single line, 20 staff)
- Where to go deeper: Reporting culture (376/2014), hazard ID , competence / authorisation control, simple but living change management.
- “Effective” signals: Look for fewer repeat maintenance errors; targeted toolbox talks that close a specific risk; quick cycle from report to fix to verified outcome.
- Scalability call: Combined safety/compliance role acceptable if independence and escalation paths remain clear.
Medium CAMO with broad contracting
- Where to go Deeper: Interface management with contracted AMOs, data feedback loops into risk registers, aand ssurance on airworthiness review findings.
- “Effective” signals: Contract performance terms tied to safety outcomes; recurring issues decline across providers after specific mitigations.
- Suitability call: Contract oversight depth scaled to volume/criticality of outsourced work.
Airline group with integrated OPS + CAMO + multiple AOCs
- Where to go Deeper: Group-level SPIs/SPTs roll-up, cross-domain hazard capture (e.g., fuel, FDM,), Change Management spanning fleet and network.
- “Effective” signals: Enterprise risk changes resourcing/rostering/training; sustained trend improvements across several SPIs post-action.
- Integration call: One assessment; coordinated domain input; single performance narrative.
DOA/POA cluster introducing new tech
- Where to go Deeper: Independent verification in design, handover interfaces to production/testing, and management of change for novel risks.
- “Effective” signals: Design escape rate down; corrective actions address systemic causes; safety objectives achieved despite pace of change.
- Suitability call: Depth of design assurance scaled to novelty/complexity rather than headcount.
Decision lines: when to escalate, when to coach
- Raise a finding when required elements are not present/suitable at initial certification, or when an operating process has lapsed; retain observations to drive improvement where processes operate but outcomes lag—unless leadership persistently ignores evidence.
- Document maturely: Capture the evidence path (what you saw/heard/measured) and keep the narrative centred on risk, performance, resilience, and maturity—not just rules citations.
Discussion Question: Share a borderline case. What tipped you toward “observation” versus a formal finding, and how did you write it to drive improvement?
Quick checklist
- Did we explicitly separate Operating vs Effective and show the evidence gap?
- Did we explain why the MS design is suitable/scalable for this organisation?
- Did we integrate intelligence from other oversight activities to avoid duplication?
- Did our write-up read like a performance narrative rather than a checklist?
Next Steps
Please see Sofema Aviation Services (SAS) and Sofema Online (SOL) for Classroom, Webinar and Online Training. For additional information please email [email protected].
See the following 2 day course available as Classroom or Webinar – Using The EASA Management System Assessment Tool (EASA MSAT) – 2 Days.
Tags:
CAMO, Compliance, DOA. POA, SAS blogs, Effectiveness, MSAT, Suitability, Organizational Contexts, Driving Effectiveness, Scalability, Organisation archetypes, Operating vs Effective, Suitability call, Present & Suitable, Checklist Mentality
