Providing online training for business aviation professionals globally.

Blog

Ep. 13 – Automation Overload? An Interview with Patrick Mendenhall

Human Factors, CRM, Podcast | July 4, 2019

Author: Brent Fishlock

My guest today is one of TrainingPort.net’s Subject Matter Experts, Patrick Mendenhall of Critical Reliability Consulting. We’re discussing Automation and Technology Management.

Brent: Hello Patrick, and welcome to the Business Aviation Training Report.

So let’s start off with just a bit of your background for our listeners, can you talk a bit about your aviation experience and how you became involved in Human Factors and Crew Resource Management.

My academic background is in Aerospace Engineering. After graduating, I went to work for Boeing where I worked in production flight test. After a year there, I joined the US Navy to fly fighters where I received the majority of my initial flying experience. I spent my last three years in the navy as an operational test pilot at Air Test and Evaluation Squadron Five (VX-5). 

A significant portion of my work there involved human factors integration. Included in any test plan for a new system would be evaluation of the human interface. For example, was the instrumentation readable in turbulence or in a tactical environment?  Or, were switches placed where they could be reached? Might a button be so small or so closely spaced that they might be subject to misidentification, etc.? This introduction to Human Factors stirred my interest, which carried through when I joined the airline industry in 1989. 

Coincidentally, that was also the timeframe when CRM was really gaining traction and acceptance within the airline industry. I will admit that, like many of my peers, I was a reluctant convert at first, but it did not take long before I recognized the wisdom and value of this program. In 2006, I was invited to help develop and present a CRM course for business aviation and to facilitate that training. Since then, our company has facilitated CRM training not only to numerous flight departments, but also to many “High Reliability Organizations,” or HROs, outside of the traditional aviation sphere. These include healthcare, energy, defense and first responders, among others. 

Automation and technology have certainly made our lives better, but they have come with a whole host of challenges, many of which we did not anticipate nor prepare for. Although not part of the original CRM subjects, Automation and Technology Management has recently become a very critical part of current CRM philosophy. 

Brent: Thanks Patrick, so let’s dive right in. So, with regards to automation and technology, what are some of the common threats?

I think that most technicians (oil rig engineers, anesthesiologists, nuclear power plant operators – and pilots) would agree that the number one threat from automation is complacency. It doesn’t take long to develop a trust relationship with the technology: make the correct inputs and the automation just works. Generally, it doesn’t get tired, doesn’t talk back, doesn’t care about the weather or politics – it just works! Usually…until it doesn’t.

Brent: So let’s talk about that. We have had some large, high profile failures recently like fan blades in very reliable engines separating and seemingly hidden software not performing as designed.

What causes the technology to fail, or at least NOT work the way that we expect?

Well of course, things do break – they wear out and they can get overloaded. Generally, “fail-safes” such as warning systems are designed into technology to reduce or mitigate the effects of the “system” simply wearing out. The more insidious threats are – once again – human caused, whether a flaw built into the original design which will ultimately result in failure (the MCAS system on the 737 Max comes to mind), or incorrect user inputs: “garbage-in-garbage-out”!

Brent: And often, it’s the complex failures that require complex knowledge of the system to render a solution. There isn’t a checklist for everything! (Also, in a training event, there is only supposed to be one failure occurring.)

What can a person do to guard against the complex non-checklist failure?

Well, you have to be ready to intervene at any time. Probably, your best defense is system knowledge; knowing as much as you can about the aircraft and the systems on board. That way, when the checklist fails you and there’s not something to refer to, you have a knowledge base to consult to mitigate the threat.

Brent: As you pointed out before we began recording, Qantas Flight 32 is a great example of systems knowledge taking over a complex failure situation and saving the day. Briefly for our listeners:

This was an Airbus 380 departing Singapore for Sydney in November 2010.

Shrapnel from the No. 2 uncontained engine failure punctured part of the wing and damaged the fuel system, causing leaks and a fuel tank fire. The shrapnel also disabled one hydraulic system and the anti-lock braking system, it caused the No. 1 and No. 4 engines to go into a “degraded” mode and damaged landing flaps and the engine controls for the No. 1 engine.

The crew entered a hold and took 50 minutes to complete the initial damage assessment, which included 58 error messages.

The initial landing distance calculation (which was 50 tonnes overweight) would not calculate a distance, but the crew solved the problem together, calculated a sensible landing distance, and landed the aircraft.

How does this example tick all the boxes of a crew demonstrating automation and system knowledge as well as excellent CRM?

That was the probably the best example of a CRM success in so many ways. Most notably was the effect of crew interaction, which displayed incredible teamwork and leadership. On that flight, there happened to be a lot of talent on the flight deck that day. Aside from the regular crew, there were two check airmen on board performing some training functions. I think there was one guy who was giving a check, and the other guy was checking the checker. Rather than endlessly following all the checklists, the crew worked together and applied their system knowledge, which was pretty phenomenal, to arrive at the most appropriate, most correct solution to each of the various problems.

The amount of information that was pouring out of the ECAM, the engine condition and monitoring system, plus the physical cues, were really overwhelming. There were literally hundreds of issues going on. I think you mentioned that there were 58 ECAM procedures that were in the queue that they had to go through at the maximum point. The way that the system on the Airbus works is it takes the most critical one and puts that one on top, but there were 58, and if they went through every one, they would have probably still been up there. But they made some command decisions.

Of course, among the things that were wrong that you enumerated, the No. 2 engine was on fire and had exploded. I was doing some reading on this, and there only 3 out of 11 fuel tanks that were functioning normally. At least two of the fuel tanks had punctures on the left side, and according to the Captain, half the hydraulics were gone, and half the electrical system was gone. As I’ve said, had the crew strictly followed the procedures, they probably would still be flying right now if they had enough gas, because there were so many things to do. But instead, they dealt with the highest priority items as a crew, and discussed and evaluated those items. There were dozens of examples in this case where the crew combined CRM and system knowledge to safely get the airplane on the ground, so it was a great case study of how right things can go.

Brent: I’d like to go back to automation threats for a minute. So how do we get around these technology threats?

“Trust, but verify!”  I mentioned complacency as being our biggest technology threat. Certainly, we need to be able to trust our automation, but prior to placing our lives and the lives of our passengers and crew we need to verify – to confirm – that the technology is doing what we intended it to do. This is a multi-step process, but one that has proven to be very effective:

  1. The Pilot Flying should always verbalize what s/he is telling the automation to do
  2. The Pilot Monitoring should confirm that the PF is entering an input that conforms to the ATC instructions, flight plan, etc.
  3. Both pilots should verify that the input has been accepted by the automation as intended – this is usually accomplished by confirming the action on the Flight Mode Annunciator (FMA), depending upon the type of input and level of automation
  4. Finally, anticipating the end state, the pilots should confirm that that is where the aircraft has ended up. If you were assigned a turn direction and heading, you always need to confirm that that is indeed the maneuver that the aircraft has performed.

Brent: Can you think of some cases in business aviation where ATM was a factor?

Yes. You are probably familiar with the Challenger 604 that crashed in Iran a little over a year ago: The preliminary accident report from the Aircraft Accident Investigation Board of Iran suggests several CRM related issues that contributed to this accident. Prevalent among them was the failure of the Captain (the pilot flying) to utilize the resources that she had available: the co-pilot, the co-pilot’s instruments, and situation awareness.

In this case, the Captain’s airspeed indicator suddenly showed an overspeed condition during cruise flight at FL370. Her reactions were based solely on her own instruments – never getting verification from the PM’s instruments. It turned out that the FO’s airspeed indicator was operating normally. Although this was not an advanced automation issue, it was certainly a failure to properly utilize the technology and resources available. The outcome was that. in an effort to correct the “perceived” overspeed, the PF ultimately stalled the aircraft and crashed, killing all on board.

Another ATM example occurred at FL410 on a Falcon 2000 near Mumbai fairly recently. Flying on autopilot, the flight was cleared direct to a fix. The PF programmed the clearance into the FMS; this was followed by a yaw damper failure (not noted by crew), followed immediately by an autopilot disconnect (noted by crew). The crew immediately set about trying to restore the autopilot but failed to note the yaw damper failure (which prevented the A/P from re-engaging).

While attempting to re-establish the A/P, nobody was actually flying the aircraft: after 15 seconds, they received a GPWS “Bank Angle” aural warning and the PIC took control at 65 degrees AOB! The aircraft lost 610 feet of altitude in the recovery and continued the flight to destination without the A/P or further incident. The ATM issue occurred when the crew failed to adhere to the most basic rule in aviation: “Aviate – Navigate & Communicate.” The crew forgot to FLY THE AIRCRAFT – aviate – when the A/P disengaged, resulting in an upset.

Brent: You have spoken about an operational automation philosophy. Can you explain this?

This philosophy encompasses four simple rules:

  1. Fly the aircraft – AVIATE – as in, aviate-navigate-communicate in that order
  2. Choose the level of automation which is appropriate for the phase of flight
  3. Verify EVERYTHING
  4. When all else fails, see rule number one

Of course, the number one rule, as mentioned previously, would always apply, regardless of the level of automation being used: “AVIATE!”.  In other words, the PF is assumed to be always flying the aircraft, regardless of the state of the autopilot. This may seem obvious, but just like the previous example points out, this is not always the case.

We should fly with the level of automation which is appropriate for the phase of flight: Just as it is appropriate to engage automation at the highest level when in cruise, it is equally appropriate to hand fly the aircraft (no automation) when cleared for a visual approach.

Every action by the PF that can affect the flight path of the aircraft should be verbalized to the PM, who should then do the mental calculus that will result in his or her agreement or disagreement. The PF action should be verified by the PM as it is taking place (e.g., verifying the inputs to the FMS), and once completed, verifying that the computer is actually doing what you wanted it to do – this is generally done by checking the new status on the Flight Mode Annunciator or FMA. Finally, confirm the “end state”: once the aircraft has rolled out after a commanded turn, is this really what was assigned?

Finally, when all else fails, just fly the aircraft – reduce the level of automation to a manageable level – which might very well be NO automation – and AVIATE!

Brent: You have been an SME for TrainingPort.net for a few years. I understand you have taken on the new Contemporary CRM that is mandated in Canada for commercial operators as of September 2019. This new Contemporary Crew Resource Management has a joint annual training component which requires all personnel to attend. How have those sessions been going?

I feel that the Contemporary CRM training requirement, specifically the Joint Annual Training component is, in principle, a great idea. CRM is an “all hands” program now and it makes sense to get all players on board at the same time and place on a recurring basis. However, from a practical point of view, I think that everyone agrees that this is an extremely challenging requirement.

As always, we are committed to meeting the spirit and intent of the law, but here is the challenge: Consider a company that conducts worldwide operations. Would the JAT obligation require that they shut down operations for a few days so that they can all gather at a central location for JAT training?

There is no getting around the fact that live, in-person, instructor-led training is the most effective way to present most trainings of this type. However, this would be an onerous obligation for a number of large operations in particular.

The solution that we have come up with is to use current technology to hold live, instructor-led Joint Annual Training ONLINE via live webinar, where trainees can participate from wherever they are located in the world. We use the ZOOM platform for this training, and so far, it has been quite effective. With internet access, participants can log in using pretty much any device that has a camera and microphone.

Brent: Thank you Patrick for taking the time to talk with us today. Where can listeners get a hold of you if they would like more information?

Send me an email at Patrick@CriticalReliability.com or call TrainingPort.net at 604.270.1343 and they’ll get you in touch with me.


aviation professional

Engaging and Effective Online Training.

TraingingPort.net Logo
Get a free topic

Required fields are indicated with a red star.

Request a free demo

Required fields are indicated with a red star.