Applying Cost-Benefit Analysis as a Decision Tool

Applying Cost-Benefit Analysis as a Decision Tool for Non-Mandatory Service Bulletins, Modifications & Component Replacement Programmes.

Sofema Aviation Services (SAS) is pleased to present the following guest blog written by our Reliability Expert – Rustom D. Sutaria – Avia Intelligence, Dubai, UAE, 2018

For the Operator to automatically comply with service bulletins will result in increased maintenance costs. For this reason, it is normal for the Operator to implement a Service Bulletin Analysis process. The basis of this process is the application of Cost-Benefit Analysis, without which even a ‘free-of-charge’ Service Bulletin will turn out to be anything but free!!
Organizations rely on cost-benefit analysis to support decision-making because it provides an evidence-based view of the SB being evaluated whilst minimizing the influences of technical opinion, company/regulatory politics, or personal bias. By providing an unclouded view of the consequences of a decision, cost-benefit analysis is an invaluable tool in developing aircraft engineering & maintenance management strategy, evaluation of new/upgraded products for installation on aircraft, or making resource allocation or purchase decisions regarding changes to the AMP in light of potential reliability opportunities.

A cost-benefit analysis (also known as a benefit-cost analysis) is a process by which organizations can analyse decisions, systems or projects. The model is built by identifying the benefits of an action as well as the associated costs and then subtracting the costs from benefits. When completed, a cost-benefit analysis will yield concrete results that can be used to develop reasonable conclusions around the feasibility and/or advisability of a decision for the selection of an upgraded aircraft component or whether or not to embody a non-mandatory Service Bulletin (SB), STC or another form of modification to an aircraft.

In this way, the cost-benefit analysis offers unique and valuable insight when:

• Developing benchmarks for major technical upgrades to aircraft (E.G. Cabin Interior Modifications).
• Deciding whether to pursue fleet-wide service bulletin campaigns.
• Evaluating new or upgraded aviation products or maintenance processes.
• Appraising the desirability of suggested airworthiness or maintenance policies.
• Assessing change initiatives in terms of financial as well as Safety perspectives.
• Quantifying the effects of a proposed change on engineering & maintenance stakeholders and other aviation parties.

In truth, the embodiment of Service Bulletins translates to higher maintenance costs.

True SB costs in this regard are typically between 7 to 13 times the basic cost of the SB. Prudence demands that the SB is carefully reviewed not only for sound economic as well as safety benefits.

Therefore, a cost-benefit analysis will allow the operator to compare the cost of embodying (compliance) the aircraft maintenance service bulletin in the light of the perceived safety and economic benefits. Without a Cost-benefit analysis, the decision to embody may well be the wrong thing to do.

A template is generally used for this purpose. It is important to consider all the relevant criteria which are applicable including:
• The planned duration of the SB embodiment programme,
• The various regulatory, technical and organisational obligations that may need to be satisfied,
• A review of benefits to both safety, reliability as well as monetary cost.

HOW TO DO A COST BENEFIT ANALYSIS

While there are no “standard” approaches for performing a cost-benefit analysis, there are certain core elements that will be present across almost all analyses. Use the structure that works best for your situation or industry. In this paper, we will go through the five basic steps to performing a cost-benefit analysis in the sections below, but first, here’s a high-level of overview:

1. Establish a framework to outline the parameters of the analysis.
2. Identify costs and benefits so they can be categorized by type, and intent.
3. Calculate costs and benefits across the assumed life of the SB programme or initiative.
4. Compare cost and benefits using aggregate information.
5. Analyse results and make an informed, final recommendation.

As with any process, it’s important to work through all the steps thoroughly and not give in to the temptation to cut corners or base assumptions on opinion or “best guesses.” It is important to ensure that your analysis is as comprehensive as possible.

“The best cost-benefit analyses take a broad view of costs and benefits, including indirect and longer-term effects, reflecting the interests of all aviation stakeholders who will be affected by the SB or Component replacement program.”

Therefore, when considering a non-mandatory SB or a component replacement programme, also consider recurring maintenance tasks that may or may not be associated with the SB or replacement component being studied.

HOW TO ESTABLISH A FRAMEWORK

In establishing the framework of your cost-benefit analysis, first outline the proposed program or policy change in detail. Look carefully at how you position what exactly is being evaluated in relation to the problem being solved.

For example, the analysis associated with the question, “does the embodiment of this service bulletin reduce maintenance costs whilst also enhancing despatch reliability?” will be much more straightforward than a broader programmatic question, such as, “how do we implement this service bulletin such that despatch reliability is improved?”

Example:

Once your program or policy change is clearly outlined, you’ll need to build a situational overview to examine the existing state of affairs including background, current performance in terms of reliability, maintenance programme & cost-effectiveness, any opportunities that the SB or component replacement may bring to the table and its projected safety &/or reliability performance and financial benefits in the future. Also make sure to factor in an objective look at any risks involved in maintaining the status quo moving forward (For example, TCH & Regulatory compliance issues).

Now decide on how you will approach cost benefits. Which cost benefits should be included in your analysis? Include the basics, but also do a bit of thinking outside the box to come up with any unforeseen costs that could impact the initiative in both the short and long term.

In some cases, geography could play a role in determining the feasibility of a project or initiative. Will the aircraft present unforeseen problems or challenges at specific line stations? Will there be a need to allocate spares to these particular line stations, or can the aircraft return to base under an ADD (for example)?

If geographically dispersed stakeholders or groups (Line Stations or Base Maintenance Organizations, etc) will be affected by the SB being analysed, make sure to build that into the framework upfront, to avoid surprises down-route. Conversely, if the scope of the project or initiative may scale beyond the intended geographic parameters, that should be taken into consideration as well.

IDENTIFY AND CATEGORIZE COSTS AND BENEFITS

Now that your framework is in place, it’s time to sort your costs and benefits into buckets by type. The primary categories that costs and benefits fall into are direct/indirect, tangible/intangible, and real. The table below shows typical examples:

Direct Costs  Often associated with the production of a cost object/service/project. Acquisition of an SB/Modification/Repair &/or Associated Spares & Mod Packages.
Indirect Costs  Usually fixed in nature and may come from the overhead of a department or cost center. Overheads associated with Maintenance & Production Planning, etc.
Tangible Costs  Easy to measure and quantify, and are usually related to an identifiable source or asset. Aircraft Downtime, facilities, and purchasing/rental of tools.
Intangible Costs  Difficult to identify and measure. Operational Interruption Costs & Incremental fuel burn.
Real Costs expenses associated with delivering the Service Bulletin Manpower costs, required spares, and raw materials to complete the required tasks. Now that you’ve developed the categories into which you’ll sort your costs and benefits, it’s time to start crunching numbers.

HOW TO CALCULATE COSTS AND BENEFITS

With the framework and categories in place, you can start outlining overall costs and benefits. As mentioned earlier, it’s important to take both the short and long term into consideration, so ensure that you make your projections based on the life of the program or initiative. In the case of a SB &/or Component replacement programme, consider changes to the AMP in terms of tasks and frequencies. Look at how both costs and benefits will evolve over time.

TIP: People often make the mistake of monetizing incorrectly when projecting costs and benefits, and therefore end up with flawed results. When factoring in future costs and benefits, always be sure to adjust the figures and convert them into present value. For example, consider costs in 2018 US Dollars.

COMPARE COSTS AND BENEFITS

Here we’ll determine net present values by subtracting costs from benefits. This facilitates the projection of the timeframe required for benefits to repay costs, also known as return on investment (ROI).

The process doesn’t end there. In certain situations, it’s important to address any serious concerns that could impact feasibility from the regulatory or TCH standpoint. In cases like these, it can be helpful to incorporate a “with/without” comparison to identify areas of potential concern, or the basis by which an SB, modification or component replacement programme may be justified.

With/Without Comparison
The impact of a fleet-wide SB, modification or component replacement implementation programme can be brought into sharp focus through a basic “with/without” comparison. In other words, this is where we look at what the impact would be—on the operator, maintainer and other stakeholders, or users—both with, and without, this programme initiative.

The impact of a Service Bulletin is the difference between what the situation would be with and without the Service Bulletin. So that when an SB is being evaluated the analysis must estimate not only what the situation would be with the SB but also what it would be without the SB.

For example:
In determining the impact of the Cockpit Door Surveillance System (CDSS) modification on-board the Airbus A319 the number of hi-jacking attempts that may not succeed as a result of an active CDSS should be deducted from the number of failed hi-jacking attempts attributed to the presence of ‘Sky-Marshalls’ and likewise the additional costs of employing a sky-marshal for each flight would be deducted from the costs of an A319 modified with CDSS.

In other words, the alternative to the SB embodiment must be explicitly specified and considered in the evaluation of the SB. The principle also applies to Component replacement equally.

TIP: Never confuse with/without with a before-and-after comparison.
In the home stretch of the cost-benefit analysis, you’ll be looking at the results of your work and forming the basis to make your decision.

THE RISKS AND UNCERTAINTIES OF COST-BENEFIT ANALYSIS

Despite its usefulness, cost-benefit analysis has several associated risks and uncertainties that are important to note. These risks and uncertainties can result from human agendas, inaccuracies around data utilized, and the use of heuristics to reach conclusions.

Know the Risks
Much of the risk involved with cost-benefit analysis can be correlated to the human elements involved. Stakeholders or interested parties may try to influence results by over- or understating costs. In some cases, supporters of an SB/Mod/component Replacement programme may assert a personal or organizational bias into the analysis.

On the data side, there can be a tendency to rely too much on data compiled from previous SBs, Mods or Component Replacement programmes. This may inadvertently yield results that don’t directly apply to the situation being considered.

Data leveraged in this way from an earlier analysis may not directly apply to the circumstances at hand and are likely to yield results that are not consistent with the requirements of the SB, Mod or Component replacement programme being considered. We recommend the limitation of the use of this process whilst also utilizing heuristics to assess the dollar value of intangibles as a means of providing a quick, “ballpark-type” assessment. However, it must be clearly noted that this kind of approach can also result in errors that produce an inaccurate picture of costs that can invalidate findings.

In addressing risk, it’s sometimes helpful to utilize probability theory to identify and examine key patterns that can influence the outcome. From the aviation safety & reliability perspective sources of concrete data will be available from the Reliability Monitoring & Safety Management systems. This would be the established and preferred approach.

Uncertainties
There are several “wild-card” issues that can influence the results of any cost-benefit analysis, and while they won’t apply in every situation, it’s important to keep them in mind as you work:

1. Accuracy affects value – Inaccurate cost and benefit information can diminish decisions around value.

2. Don’t rely on intuition – Always research benefits and costs thoroughly to gather concrete data—regardless of your level of expertise with the SB, Modification or Component replacement programme at hand.

3. Cash is required to pay for the work. However, it is unpredictable – Revenue and cash flow are moving targets, experiencing peaks and valleys, and translating them into meaningful data as part of your analysis can be challenging. Seek the help of your management accountant.

4. Income influences all decisions – Income level can drive a customer’s ability or willingness to implement an SB, Modification or Component Replacement Programme.

5. Money isn’t everything – Some benefits cannot be directly reflected in dollar amounts (Consider Safety & Reliability Benefits).

6. Value is subjective – The value of intangibles can always be subject to interpretation.

7. Don’t automatically double up – When assessing an SB, Modification or Component Replacement Programme in multiple ways, be mindful that doubling benefits or costs can result in inconsistent results. The CBA could fool you into thinking that the studied programme could be worthwhile when in actual fact it is not.

Controversial Aspects
When thinking about the most controversial aspects of cost-benefit analysis, all paths seem to lead to intangibles. Concepts and things that are difficult to quantify, such as human life, maintenance & airworthiness reputation, the environment, and customer/brand loyalty can be difficult to map directly to costs or value.

With respect to intangibles, using the cost-benefit analysis process to drive more critical thinking around all aspects of value—perceived and concrete—can have beneficial outcomes. Cost-benefit analysis assumes that a monetary value can be placed on all the costs and benefits of a program, including tangible and intangible returns.

As such, a major advantage of cost-benefit analysis lies in forcing people to explicitly and systematically consider the various factors which should influence strategic decision regarding the SB, modification or Component Replacement Programme.

SAS provides the most comprehensive reliability training program including a detailed treatment of Reliability Mathematics – for details of the available training courses please see www.sassofia.com or email office@sassofia.com