Full text of "Flight deck automation: Promises and realities"

See other formats

Conference Publication 10036

Flight Deck

Automation:

Promises

and
Realities

Proceedings of a
NASA/FAA/lndustry Workshop
held at Carmel Valley, California
August 1 - 4, 1988

r\j/\sA

(NASA-CP-10036) FLIGHT O^CK AUTiJMATinN: N90-n384

PROMISES ANH RtALlTItS (NASA) 200 p

CSCL OlD

Unci IS

NASA Conference Publication 10036

Flight Deck

Automation:

Promises

and
Realities

Edited by

Susan D. Norman

Ames Research Center

Moffett Field, California

and

Harry W. Orlady

Orlady Associates

Los Gatos, California

Proceedings of a

NASA/FAA/lndustry Wori<st)op

organized by

NASA Ames Research Center

heldatCarmel Valley California

August 1 - 4, 1988

r\j/\sA

National Aeronautics and
Space Administration

Ames Research Center

Moffett Field, California

1989

PREFACE

TABLE OF CONTENTS

PAGE

INTRODUCTION ^

Susan D. Norman, Chair

PANEL DISCUSSIONS

Current Flight Deck Automation: Airframe Manufacturing and

FAA Certification "

ReaUties of Operating Automated Aircraft: Air Carriers and

FAA-Aircraft Evaluation 23

PRESENTATIONS and INVITED PAPERS

Field Studies in Automation 37

Dr. Earl Wiener

The Advanced Automation System (AAS) for Air Traffic

Control ^7

Alden Lemer

The Effects of Automation on the Human's Role:

Experience from Non-Aviation Industries 63

Dr. David Woods

Lufthansa Cockpit Systems Survey: A-310 89

Captain Peter H. Heldt

WORKING GROUP REPORTS

Automation and Air-Ground Communication 101

Crew Coordination 109

Understanding Automated Systems 119

Training for Advanced Technology Aircraft 125

Error Management 1^^

Design and Certification 139

SUMMARY and CONCLUSIONS 151

Susan D. Norman, Chair

PRECEDING PAGE BLANK NOT FILMED

111

,^B^JJ^_JWlHIIOIMU• w»

APPENDICES

A . Aircraft Automation Philosophy: A Source Document 1 65

B. Workshop Participants 193

C. Instructions to Working Groups 197

D. Automation Terms and Acronyms 207

PREFACE

Issues of flight deck automation are multi-faceted and complex. The rapid
introduction of advanced computer based technology onto the flight deck of
transport category aircraft has had considerable impact on both aircraft operations
and the flight crew. As part of NASA's responsibility to facilitate an active
exchange of ideas and information among members of the aviation community, and
since the timing appeared appropriate for a discussion of the effects of these
changes on the roles of the crew and the automation, a NASA/FAA/Industry
workshop devoted to flight deck automation was organized by the Aerospace
Human Factors Research Division of NASA- Ames Research Center.

The workshop was held at the Carmel Valley Inn in Carmel Valley, California,
August 1-4, 1988. Participants were invited from industry and government
organizations responsible for design, certification, operation, and accident
investigation of transport category, automated aircraft. Attendees included
representatives from airframe manufacturers, air carriers, the NTSB, the FAA,
NASA and the university community.

The workshop took a broad systems level perspective and discussed the design,
training and procedural aspects of flight deck automation, as well as the crew s
ability to interact and perform effectively with the new technology.

The goals of the workshop were to clarify the implications of automation, both
positive and negative, and to identify issues regarding the design, training and
operational use of flight deck automation. Participants with operational, training,
or design experience were crucial to achieving these goals.

A small preliminary, NASA workshop attended by NASA-Ames research staff and
three human factors specialists was held to prepare for the industry-wide
workshop. Thereafter, a source document on automation philosophies was
prepared, and that report is included in the appendix. It is intended as an
introduction to the concepts and ideas regarding flight deck automation which were
discussed at this workshop.

The remainder of this report presents the final results of the August workshop. The
ideas and concepts in this document were developed by the workshop participants
and written primarily by the NASA- Ames research staff. Since the workshop was
small and informal, this report is not a verbatim transcript of the proceedings
Instead, the findings have been synthesized, where necessary, into an aggregated
format and individuals and organizations have occasionally been de-identified. This
format was chosen to facilitate a free exchange of information at the workshop.

The workshop consisted of four major sessions:

Introductory panels: Informal presentations by industry and
government representatives on the design, operation, and certification
of the automated cockpit.

Formal papers: Formal presentations by members of the aviation
community pertaining to automation in an operational context.

Workshop group discussions: Six working groups were convened to
discuss specific aspects of automation. These were:

Automation and Air-Ground Communication

Crew Coordination

Understanding Automated Systems

Training for Advanced Technology Aircraft

Error Management

Design Philosophies and Certification Issues

Summary session: Summary and concluding remarks by the chair and
other participants.

FLIGHT DECK AUTOMATION:
PROMISES AND REALITIES

INTRODUCTION

Susan D. Norman
Chair

In developing the format and content for this workshop, considerable attention
and time were given to the relationship between the aircraft design, certification
criteria, operational procedures and training for transport category aircraft.
Extensive discussions with aviation and human factors specialists were held at
NASA-Ames, NASA-Langley and at various group meetings, including the ATA
task force on human factors. The chair is much indebted to these people and
discussions with them concerning the technical content and format of the
workshop.

In these discussions, it became clear that one important aspect of cockpit
automation was the increasmgly complex interaction between the processes of
design, training, operations, and certification of the aircraft. The complexity of
the new technology necessitated a clear understanding of the effect these processes
have on one another in an operational setting. This was particulariy important
because, in time constrained situations, automated systems are often not as
flexible as their human counterparts nor are their designs normally able to
function as effectively as a human in nonstandard conditions.

With this in mind, the panels and working groups were structured to explore the
interdiscipUnary, system level aspects of the automated cockpit. This workshop
was to be a unique opportunity to understand the implications of how one element
of the process impacted another; for example, how the implementation of a
design or philosophy could impact the operational procedures or the interface
with ATC.

In preliminary discussions, it was also clear that the role of the pilot and crew
was a key factor. Had it changed and if so, how? Was any apparent change
moving in an appropriate direction? As a humorous exaggeration of the potential
for a new role for the pilot, the question was asked if participants knew what the
air transport crew of the 21st Century would look like? The answer was that the
crew would be composed of two members, a pilot and a dog. The pilot was
responsible for feeding and caring for the dog, and the dog was there to bite the
pilot if he touched anything.

Although certainly not intended to be realistic, this joke drives home the point
about the potential role of the flight crew in future aircraft. Of the many
unanswered questions presented to this workshop, those regarding the role of the

PRECEDING PAGE BLAI\iK WOT FILMED

^j^j^J^^wnwiiMW «^

crew were the most difficult. Is the potential for change in the role of the crew a
valid concern?" and, if so, "Is this the direction the industry, as a whole, wishes
to go?"

Finally, it was intended that the workshop focus on operational realism, because
real, versus perceived, problems and benefits need to be specified. Although there
was considerable discussion of the theoretical aspects of cockpit automation,
including philosophies of automation, the workshop was not intended to be
theoretical in nature. It is important to understand and assess the existing situation
before any changes, future research programs or philosophies are developed. A
view toward the future is important, but a critical understanding of the current
situation must form the basis for any discussions of the future.

Panel

CURRENT FLIGHT DECK AUTOMATION:

AIRFRAME MANUFACTURING AND FAA CERTIFICATION

Prepared by Susan Norman and Kathy Abbott

Moderator: Sam Morello, NASA-Langley Research Center
Members: Harty Stoll, Boeing Commercial Airplane Company
John Miller, Douglas Aircraft Company
Don Armstrong, FAA Aircraft Certification

INTRODUCTION

The new generation of automated aircraft has increasingly used technology on the
flight deck to enhance factors such as safety of flight and economic performance.
The process of developing a new aircraft begins with the design where many
fundamental and irrevocable decisions are made. Therefore, a discussion of
automated aircraft technology should begin with the philosophy of design.

The panel members were asked to describe, based on their extensive experience, the
current philosophy of automation and to cite examples which illustrate how this
philosophy has been implemented. The interrelationship of the certification process
and its impact on the design were also discussed on this panel, since the FAA
regulations must form the basis for an overall, system wide check on the
performance of the aircraft in an operational environment.

The major points from this session have been summarized and reorganized by topic
to capture the essence of the presentations made by the panel members. The design
and certification issues are discussed separately.

DESIGN IN AN AUTOMATED ENVIRONMENT

The manufacturers presented their perspectives with examples from the most
recently designed aircraft. The figures were presented by Mr. Harty Stoll of the
Boeing Commercial Airplane Company.

Benefits of Automation

It is important to state that automation has been a clear and continued benefit in
terms of safety of flight. There is no question that the advances made in the
reduction of the accident rate associated with human error have occurred to some
extent because of the introduction of automation onto the flight deck. But, because

PRECEDis^G FAGu 3d.A..;; f\uT FILMcD ««■ JL

the workshop focussed on what can be done to improve safety of flight, most of the
discussions centered on issues and problems as opposed to benefits. This is not
intended to be a reflection on the technology itself.

WORLDWIDE 737 OPERATORS DECEMBER 31. 1987

OVER 200 OPERATORS TWO PILOT CREW (1.4B0 AIRPLANES) 25 MILUON FLIGHT HOURS

ABU DHABI

AER UNCUS

AEROLINEAS ARG

ARRO TOURS

AIR ALOERIE

AIR ATLANTA

AIR niCLaiUM

AIR BERLIN

AIR CAUVORNIA

AIR COMORRS

AIR DJIBOUTI

AIR EUROPE

AIR FRANCE

AIR FLORIDA

AIR GABON

AIR GUINEB

AIR HAITI

AIR LANKA

AIR LIBERIA

AIR MADAGASCAR

AIR MALTA

AIR MALI

AIR NAURU

AIR NEW ZEALAND

AIR TANZANIA

AIR ZAIR8

AIR ZIMBABWE

AIRWAYS INTL

ALASKA AIRLINES

ALL NIPPON

ALOHA AIRLINES

AMERICAN AIRLINES

AMERICA WRSr

ANGOLA AIRLINES

ANSBTT AIRUNES

ARAB INTL AIRUNES

ARAMCO

AUSTRALIAN A.F.

AUSTRIAN AIRLINES

AVIANCA

BAHAMASAIR

BAVARIA

BEIJING

BRAATHENS

BRANIFF

BRAZIL

BRITANNIA

BRIT AIRTOURS

BRITISH AIRWAYS

BRITISH MIDLAND

BUSY fEE

CAAC

CAMEROON AIR

CANADIAN AIRLINES

CAYMAN AIRWAYS

CONTINENTAL

CHALLENGE

CHINA AIRLINES

CONDOR

COPA-PANAMA

CORSE AIR

CP AIR

CRUZEIRO

DAN-AIR LONDON

DELTA AIRLINES

DOME PETROLEUM

DRAGON AIR

EAGLE AIR

EG&G-LAS VEGAS

EGYPT-COVT

EGYPTAIR

EL AL

ELDORADO

EMIRATES AIRLINES

ETHIOPIAN

EURALAIR

EUROPE AERO

EXECUTIVE AIR

FAR EASTERN AT

FAUCETT

FRDKRAL EXPRESS

FLKillTPATH LTD

FRONTIER

GUANGZHOU REGION

GULF AIR

GUYANA AIRWAYS

IIAPAG-LLOYD

IIISPANIA

IBKRIA

ILG TRAVRL

INDIAN AIR FORCE

INDIAN AIRLINES

INDONESIA GOVT

INTER EUROPEAN

IRAN-GOVT

IRANAIR

IRAQI AIRWAYS

ISRAEL INLAND

ITEL AIR

JAT

KABO

KOREAN AIR FORCE

KUWAIT AIRWAYS

LAUDA AIR

LINHAS

LINA-CONGO

LADECO

LAN-CHILE

LUFTHANSA

LUXAIR

MAERSK AIR

MALAYSIA AIRLINES

MARITIME INV CO

MARK AIR

MECOM CO

MEXICO-GOVT

MBY AIR

MIDLAND MONTAGU

MIDWAY AIR

MIDWAY EXPRESS

MONARCH AIRLINES

NATIONAL AIRLINES

NEW YORK AIRLINES

NIGER-GOVT

NIGERIA AIRWAYS

NISSHO-IWAI CO.

NOGA

NORDAIR

NORDSTRESS

ORION AIRWAYS

PACIFIC EXPRESS

PACIFIC SW

PACIFIC WESTERN

PAKISTAN INTL

PAN AM

PSOPLB EXPRESS

PBTROLAIR

PIEDMONT

PLUNA

POLYNESIAN AIR

PRESIDENTIAL UR

QUEBECAIR

ROTTERDAM AIR

ROYAL AIR MAROC

ROYAL BRUNEI AIR

ROYAL DUTCH AIR

ROYAL SWAZI

SABENA

SAHSA

SAT

SAUDI ARAB GOVT

SAUDIA

SAVAR

SHARJAH-GOVT

SINGAPORE AIR

SKYBUS INC

SOBELAIR

SOUTH AFRICAN

SOUTHWEST-JAPAN

SOUTHWEST-US

SPANTAX

STAR JET CORP.

SUDAN AIRWAYS

SUN WORLD

SUNLAND AIRLINES

SURINAM AIRWAYS

TACA INTL AIRLINES

TAME-ECUADOR

TAP/AIR PORTUGAL

TEXAS AIR

THAI AIR FORCE

THAI AIRWAYS

TRANS AIR

TRANS EUROPEAN

TRANSAVIA

TRANSURASIL

TRANSPACIFIC

TUNIS AIR

UNIT AR EM-GOVT

UNITED AIRLINES

UNITED AVIATION

UNITED TECH

UNIVERSAIR

US AIR FORCE

US COVT-NASA

USAIR, INC.

VARIG AIRLINES

VASP

VENEZUELA-GOVT

VIVA

WESTERN AIR

WIEN AIR ALASKA

XIAMEN

YEMEN AIRWAYS

YUNNAN

ZAMBIA AIRWAYS

THREE CREW (ONE AIRUHE-AIR FRAWCE)

Figure 1

Although it is difficult to quantify the actual level of improvement in the accident
rate associated with human error, there are some statistics which indicate this
general trend. First, it is important to recognize the large number of flight
operations and departures in today's air transport system. Figure 1 lists the
worldwide Boeing 737 operators as of December, 1987. It shows that there are
about 200 air carriers and 25 million flight hours per year.

Second, the accident rate for crew caused accidents has been decreasing. Figure 2
shows the data as a function of aircraft and its associated level of automation.
Although there are many factors which impact these data, it is clear that the
overall accident rate has not increased, and automation has been a factor m this
reduction.

CKEW CAUSED ACCIDENT VS/ AUTOMATION

AUTOMATION

Ed
Q

oi
w

w
o

M
U

a
<:

Q
U3
CO

:=>
<
o

ATTITUDE
HEADING HOLD
AUTOPILOT

von/ AUTOPILOT

C/AMODE

AUTO REV. THRUST

AUTOTHROTTLE
ALTITUDE HOLD A.P.
AUTO SPEED BRAKES
NORMAL REF. NAV.

VERnCAL SPEED A.P.
AUTOLAND
AUTOSPEED BRAKE
FLAP LOAD RELIEF

Aim) FUEL MGMT.
AUTO GEN. MGMT.
AUTO AIR COND.
AUTO PRESS.
AUTO STDBY POWER
CWS AUTOPILOT

FULL AUTOPILOT
FMC - SINGLE
GLASS COCKPrr
IRC

ELECTRONIC ENG CONTROL
DUAL FMC

L NAV/V NAV AUTOPILOT
FULL AUTO SUBSYSTEMS
AUTO C/W READOUT
QUIET/DARK COCKPIT

EFIS/EICAS
DUAL FMC
AUTO IGNITION
WINDSHEAR ALERT

707

727

747

737
-100/-200

737
-300/-400

757/767

Figure 2

The Development Process

The design process for developing a new aircraft is complex. Improvements in
the design to reduce the potential for human error are an active part of the design

process. Manufacturers consult flight test pilots and operational carriers to
determine their needs and assess the impact of technology.

Boeing has a formal Right Deck Design Committee which reviews accident data
and makes specific recommendations regarding the design of the flight deck.
These new design options are available only because of the new automated or
computerized technology which exists today. Examples of the accident related
causes and associated design improvements which were incorporated into the
Boeing 767/757 design are given in Figure 3.

In addition to examining accident statistics, Boeing also reviews major problems
and identifies their associated subproblems. A recommendation is then made for
an improvement in the design. As examples. Figure 4 lists the major
recommendations for the 747-400/737-300.

Boeing Flight Deck Design Committee

Examples of Accident Data Reviewed
Subsystem Management Accidents— Worldwide Air Carriers

1968-1980

Accident Related Cause

Crew omitted pitot heat

Wrong position of standby power
switch

Flight engineer and captain con-
ducted unauthorized trou-
bleshooting

Electrical power switching not co-
ordinated with pilots

Flight engineer shut off ground
proximity warning system

Faulty fuel management

No leading edge flaps on takeoff
with digital computer

Confusion over correct spoiler
position

Crewman did not follow pilot's
instruction

Mismanaged cabin pressure

7fi7/7j;7 Desifn Tmnrnvements

Auto on with engine start

Automated standby and essential
power

Simplified systems; delete
maintenance functions

Auto switching and load shed-
ding — no crew action required

Shut ofl" on forward panel in
full view of both pilots

Auto fuel management with alert
for low fuel, wrong configuration,
and imbalance

Improved takeoff warning

Dual electric spoiler control

Full-time caution and warning
system

Dual auto system with auto
switchover

Figure 3

Boeing Flight Declc Development Committee
Major Problems Identified

Problem

1 ) Lack of timely aircraft
position information

2 ) Engine control & moni-
toring caused high
workload

3 ) Inadequate caution and
warning system

4 ) System management
causes high workload
and errors

5 ) Inadequate design evalu-
ation methods

6) Display reliability

Sub-Problem

Approach information
dlfflcult to read

Exact airplane positions
not always known — high
workload

Difficult to plan ahead

Difficult to rapidly set
desired thrust

Large number of throttle
adjustments

Lack of alerting for out-
of-tolerance condition

Excessive aural and nui-
sance alerts

Lack of standard color
usage

Alerts not centralized or
categorized

Increase of hardware and
functions of systems in-
creases error

Monitoring of out-of-
tolerance conditions in-
creases workload

Increase procedures

No means to evaluate de-
sign before they are In-
tegrated in cockpit

Single systems limit er-
ror detection

Panel temperature reduces
reliability

Recommendation

MAP display
FMS

INS-type information

FMS, MAP plan mode
Electronic engine control

Alerting means Included
with engine Instruments

Quiet, dark flight deck

Standardize colors

Central alert with im-
proved logic — simplified
systems

Simplify systems
Simplify panel hardware

Add redundancy and au-
tomation so no Immedi-
ate crew action required

Develop new computer-
ized workload techniques

Digital computers
CRT displays

Liquid crystal display
Reduce LRUs
New concept for equip-
ment cooling

Figure 4

7 ) Irritation and crew fa-
tigue items

g ) Inadequate or uneven ii-
lumination

9) Poor readability of dis-
plays

10) Inadequate landing vi-
sion and vision for col-
lision avoidance.

Cockpit noise

Air conditioning and cir-
culation

Eye fatigue

Seat comfort

Inadequate stowage

Instruments and lighted
pushbuttons too hot

New displays make light-
ing balance more diffi-
cult

Improved contrast

Visual angle too small

Low minimums require
better down vision

Windows not designed to
collision avoidance

• New equipment and re-
quirements

New pushbutton concept
required

Automatic dimming con-
trols

New equipment to higher
standards

• New window design
criteria

Figure 4 (continued)

Design Philosophies

The Boeing design philosophy is best characterized by an emphasis on simplicity
first, then redundancy, and last automation (Figure 5).

EFFECTIVE SYSTEMS DESIGN:

Simplicity

Redundancy
Automation

Figure 5

As examples of the implementation of this philosophy, die design of the Boeing
747-400 has centralized and reduced the crew alerting system so that it is simpler
for the crew to determine what is happening. Another example is the fuel system
on the 747-400. The original design specified 5 fuel tanks per wing because of
structural considerations in the wing. But, from an operational standpoint, this
design needed a total of 10 boost pumps (two per tank). The resulting fuel
crossfeed procedures were determined to be overly complex and the design team
elected to simplify die overall system by implementing a baffled tank structure.
The resulting system had only 3 tanks per wing and used some automation in the
middle tank to assist in die reduction of operational complexity. This illustrates
the importance of first designing to simplify which may, in turn, reduce the need
to automate.

The Boeing automation philosophy is summarized in Figure 6. Automation does
have a vital role in aircraft safety design. This is best exemplified in the
automation of engine controls. With the introduction of appropriate automation,
the engines can now be fire-walled simply by moving the tiirotties forward and it
is not necessary to adjust any other controls.

Another important issue for the design of state-of-die-art aircraft is the pilot role.
John Miller of Douglas indicated that the MD-11 has been designed with die
criteria that any irrevocable action, such as engine shutdown, must have a manual
pilot action.

Automation Philosophy

. Simplified/Minimized Crew Procedures for Subsystem Operation

• Reduces random and systematic error

• Increases time for primary pilot functions

• Prevents requiring any immediate crew action

• Reduces subsystem mismanagement accidents

• Centralizes crew alerting for error reduction

• Allows "fire walling" engine controls

• Allows two member crew operation

• Improved Navigation Information

• More exact airplane position indication (MAP)

• Reduces fuel usage

• Higher reliability and improves accuracy

• Reduces crew error

• Reduces workload, allows more pre-planning

• Improved guidance and control

• Reduces workload

• Allows low minimum operation

• Allows manual, semi-automatic or automatic pilot flight

• Increases precise guidance information

Figure 6

Both manufacturers' design philosophies included the concept that the pilot should
primarily be responsible for flying the aircraft. A minimum of crew procedures
facihtated this task. For example, during the MD-11 design process, procedures
were minimized where possible, particularly when the procedure could be pre-
specified (i.e., there were no options or choices). For example, the MD-11 has six
CRTs, and in the case of a failure of one, there is a specific, recommended
reconfiguration for the five remaining displays; similarly with 2 failures, etc.
Instead of making this reconfiguration process a procedure, the decision was
made to automate it, because the reconfiguration could be predetermined. The
crew would then be able to focus their attention on flying the aircraft, instead of
referring to a procedural manual to reconfigure the CRT displays.

Regarding function allocation, both manufacturers indicated that the subsystem
management area was most amenable to automation because of the highly
proceduralized nature of the task. The Boeing philosophy of allocation is

illustrated in Figure 7. Those tasks associated with the guidance and control of the
aircraft have remained with the most direct crew involvement because these tasks
are dynamic and critical.

Functions Allocated to Crew

Guidance

Control

• Separation

• Navigation

• Systems Operation

INCREASING

CREW
INVOLVEMENT

Figure 7

Lessons Leamed

With the introduction of systems such as the CDU, a lot of heads-down time
resulted, especially at first. During the original design, such systems were not
intended to be used during periods of frequent ATC changes, particularly in the
terminal area. However, a great deal of emphasis was placed on the FMC/CDU
during the certification because it was a new system. It was also emphasized
during training, which may have caused an over-emphasis during check rides.
The result was that pilots may have felt obligated to use it in the terminal area
even though this was not the original design intent.

CERTIFICATION IN AN AUTOMATED ENVIRONMENT

A brief overview of the FAA certification process was presented by Don
Armstrong. There are three basic phases: setting standards, substantiating designs
by analyses and laboratory tests, and flight tests.

With respect to the standards, the rules attempt to be as objective as possible. For
automatic digital systems, there are three basic requirements:

1) The system must perform the intended function.

2) The system must meet all applicable requirements; e.g.,
specific rules governing electrical loads, circuit protection,
electromagnetic interference, flammability, etc.

3) The system must interface properly with all other systems, and
with the crew.

However, new concepts often do not have suitable standards, and it becomes
necessary to work with the applicant to determine appropriate certification
criteria. In fact, the rules are inherently two to three aircraft generations behind
the state of the art of the technology to which they apply.

The next aspect of certification involves design substantiation, which involves
three aspects for automatic and/or digital systems: software testing, hardware
testing and the integrated system level tests. Software is categorized into three
classes according to the importance or criticality of failures: critical, essential,
and non-essential. Software which is critical to continued safe flight and landing
(where failures can have potentially catastrophic results) is classified as critical.
The most rigorous analyses and tests are conducted to assure that such failures
will be improbable or extremely improbable. Equipment whose failures result in
httle effect on continued safe flight is classed non-essential, and their failure rates
can therefore be relatively high.

Hardware is given the usual "shake and bake" testing which has been the industry
standard for years. The hardware and software are then tested in an integrated
manner to assure that the components function as expected.

The final phase of the certification process is flight testing. Every major sequence
of events (changes from one mode of operation to another) is included in the
flight scenarios. However, since it is not possible to evaluate every conceivable
situation or series of mode changes, the flight tests must inevitably be spot checks
of families of logical sequences. Cockpit layouts, including labeling, hghting, and

annunciation under normal and abnormal conditions, are evaluated. Finally,
workload effects are assessed, but, in the end, the final flight test results represent
the consensus of the subjective evaluations of the flight test pilots.

This workshop offered an opportunity to air some concerns regarding the current
certification process in the hope that a broader perspective could be gained and
that certain issues might be further clarified. Some of these issues are:

1) The current rules do not have any comprehensive human factors
requirements; instead, the subjective evaluation of the flight test
pilot determines certifiability. This is worrisome not because of the
subjective evaluation itself, but because the design engineers have
made critical design decisions long before the flight tests occur, and
changes are routinely resisted due to cost and schedule impacts. In
the absence of rules which incorporate human factors criteria, the
question is whether these design decisions adequately reflect the
concerns of the FAA test pilots and the line pilots they try to
represent.

2) Although this is difficult to admit, modifications by a manufacturer
to previously approved models, or installations of new systems to
existing aircraft by modifiers resulting in STCs*, are not usually
rigorously evaluated for human performance criteria such as crew
workload. This is becoming an increasingly important issue because
of the increased complexity of these modifications, particularly in
the business jets.

3) It is difficult to focus on standardization since the dominant
manufacturing strategy is to meet the customer's requirements. For
example, it is possible to present information using different
colors, signals on or off, aural messages which are loud or soft,
high numbers up or down, data which are always present or
sometimes inhibited, etc., etc. The situation is further complicated
by the fact that the industry has diverse opinions. These opinions
even differ within the same manufacturer, let alone between the
various FAA offices, and even perhaps between the test pilots
within a single office. There are many cries for standardization,
but the FAA is understandably reluctant to impose its will because
to do so not only dictates design, but may also inhibit innovation.

* Supplemental Type Certificates

4) Flight test pilots of necessity make individual subjective
assessments. The primary concern here is that the lack of standards
for such assessments may not allow predictable results to be
achieved among the small universe of pilots in the several FAA
offices.

5) There is a critical need for a definitive, widely accepted training
course in human factors for FAA test pilots.

In conclusion, a quote from St. Paul in his Epistle to the Romans was cited: "You
wish to have no fear of the authorities? Then continue to do right and you will
have their approval, for they are God's agents working for your good."

Panel

REALITIES OF OPERATING AUTOMATED AIRCRAFT:

AIR CARRIERS AND FAA-AIRCRAFT EVALUATION

Prepared by Susan Norman and George Steinmetz

Moderator: Earl Wiener, University of Miami
Members: Al Ogden, United Airlines

Jack Wisely, TWA

Stu Grieve, Britannia Airways, Ltd.

Rod Lalley, FAA-Aircraft Evaluation

INTRODUCTION

The previous panel discussed the design and certification processes. In the real
operational world, however, unforeseen events occur which can cause problems
even with the most thoroughly planned and carefully engineered systems. With this
reality in mind, the representatives of the air carriers and an FAA aircraft
operational evaluation organization were asked to discuss issues associated with the
introduction of the new generation of automated aircraft and to cite examples,
based on their operational experience, which illustrated these issues.

This paper summarizes the panel presentations and discussions, but is not intended
to be a consensus of the panel members. It is organized by major topics which may
overlap in content because automation, itself, is a highly complex subject. Since
most of the important points made during the panel discussion are related to the
technology and not the carrier or manufacturer, references to specific air carriers
and manufacturers have, for the most part, been deleted.

LESSONS LEARNED FROM OPERATIONAL INCIDENTS

Benefits

The benefits associated with autoflight are substantial and this fact should not be lost
even though most of the panel discussion, and hence the report, concerned how to
improve our use of automation. As an example, the Boeing 767 was one of the first
aircraft to employ substantial automated technology. United has been flying this
aircraft since 1982, and they have never had an accident or incident resulting in
damage to the aircraft. This, in part, is a result of the extremely good engineering
of the aircraft. There has been only one incident requiring an NTSB investigation
which was later discontinued.

A specific example of one of the benefits of automation is the fact that United has
never had a reported altitude bust on the Boeing 767 when the aUitude was
correctly set.

Operational Crutches

One of the lessons learned regarding new technology aircraft is that we should
not build operational crutches to get around improper designs. An example was
given to illustrate the importance of this lesson. During the early days on the 767,
a problem occurred (which has now been fixed) with the electronic fuel control.
An over-sensitivity in the engine EGT (exhaust gas temperature) sensors
occasionally caused spikes in the EGT data which, in turn, could lead to an
automatic down trim of the engine. This was a potentially serious problem,
particularly during takeoff. The engine manufacturer promised to reprogram the
engine control software, but die air carrier management felt that a temporary fix
or 'operational crutch' was necessary until the permanent fix could be
implemented. A new operational procedure was instituted which specified that the
EEC (electronic engine control) should be tumed off during takeoff. However, in
turning it back on shortly after takeoff, a crew inadvertently manipulated the fuel
control switches, which were located close to the EEC switch. This resulted in the
shut down of both engines, which were soon restarted.

In this example, a human engineering design, a management decision and a fast
action on the fuel control switch all came together to contribute to a serious
incident.

Training/ Operational Procedures

In training programs, it is important to remember that the pilot is flying an
aircraft, not a computer. Due to the novelty of the technology, some early
training programs emphasized the automatic and computer aspects of the aircraft
with some loss of emphasis on basic airmanship. As a result, instruction in
fundamental flying knowledge, such as altitude and power settings to effect
different configurations, may have suffered. More recent training programs
emphasize basic manual flying skills, including appropriate flap and power
settings.

At one air carrier, some situations occurred during the 1970s on the 727 which
emphasized the need to develop a crew procedure for every potential failure. The
resuhing training conditioned the pilot to expect that there is a procedure to
follow for every failure. This operational philosophy worked well before the
introduction of the Boeing 767 but did not work well on this aircraft. This is in
part due to the fact that formulating procedural solutions for automated

technologies in advance can be difficult because of the sheer complexity of pre-
determining all potential failures. But, fortunately, the 767 is very well
engineered and no major problems resulted. One positive way to improve this
situation is to teach pilots an understanding of how the systems work together and
how they impact each other.

Understanding how the system works is further illustrated by a situation
regarding holding patterns which occurred during the early operations of the
767. Some pilots apparently did not understand how the flight management
computer (FMC) drew the holding pattern. While they were busy inserting the
holding pattern data into the FMC, one crew failed to remember to slow the
aircraft down to the recommended holding pattem speed. This, coupled with a
strong tail wind, resulted in an overshoot of the assigned pattem.

A team approach to new aircraft training is important. One carrier's
representative indicated that simulation is absolutely essential and improved
facilities are needed.

When highly automated aircraft were initially introduced, one carrier had
concerns about older pilots and their ability to adjust to the new technology. The
older pilots, however, had little difficulty learning the new systems. The only
difference was that the younger pilots leamed more quickly.

Requalification training should be very thorough and this is an area where
improvement may be needed. For example, "de-programming" the expectations
of a pilot who is retuming to a less automated aircraft may be necessary. A pilot
transitioning from a highly automated aircraft to a less automatic one may
unconsciously expect it to automatically level off at altitude, but some older
technology is not capable of this.

An important issue is mixed fleets where pilots fly both sophisticated and non-
sophisticated aircraft. The issue is not the individual aircraft, but mixing aircraft
designated as a common type which have differing design philosophies into one
fleet. At least one carrier split its fleet due to concem about this issue.

Mode Misapphcation

It is possible with the new computer technology for the crew to assume that the
aircraft is operating under one control mode when, in fact, it is not. This is
particularly important when default modes are involved. One example of a mode
misapplication which resulted in a high altitude stall waming occurred on the
767. The aircraft was climbing with the vertical speed mode engaged. In this
mode, no automatic limits on vertical speed are designed into the system. This

means that the aircraft automation will never over-ride or limit the selected
vertical speed even if a staU is imminent or the speed is too high. The fact that
this is the default mode makes it particularly troublesome due to the potentially
serious consequences. Such default modes should be avoided in future designs.

Over Reliance on Automation/Electronic Map Failures

Although rare, electronic maps have failed and crews should not be trained to be
overly dependent on them. It is, however, easy to become accustomed to these
map displays because of their usefulness and quahty. In training, the use of charts
needs to be re-emphasized as well as the need to maintain good situational
awareness.

When things do go wrong, there may be a reluctance by the crew to turn the
automatics off. There is a tendency to try to use the automatics to cope with
rapidly changing circumstances even if there is not enough time to enter the new
data into the computer (Editor's note: the term "reprogram" is commonly used
for this process, but technically reprogramming involves modification of the
system software, not data entry). Data entry into the FMC (flight management
computer) can also be a procedural concern. The early FMCs were slow and it
was easy to get ahead of the display. Pilots should be trained to check the
correctoess of the numbers before the "execute" button is pressed.

ATC/Automated Aircraft Compatibilitv

Current designs for the automated navigation systems interface very well with the
existing ATC system as long as there are no flight plan changes. However, the
reality is that changes (sometimes frequent) to the flight plan and hence the FMS
(flight management system) are required for weather, traffic, enroute spacing
slow downs, etc. There may be a tendency to teach airline crews that the
automated systems can accommodate these changes. Unfortunately, the data entry
may often take more time than the ATC environment allows.

In addition, frequent ATC changes to the flight path can render the automatic
system useless because the data take too long to enter, particularly below 10,000
feet. For example, vertical navigation systems work well above 10,000 feet, but
sometimes are not very workable below this altitude due to speed and altitude
restrictions. Since these restrictions are often deleted from the SIDs (standard
instrument departures), changes are hard to keep up with. The design of the MD-
11 may not have this problem since speeds can easily be changed
(reprogrammed). The integration between the mode control panel and the FMC
wiU also be a good feature of the MD-1 1 and 747-400.

Another example of the problems which can arise due to the mismatch between
the new automated aircraft and ATC, involved a crew mis -manipulation of the
mode control panel and an ATC inability to permit the aircraft to descend when
requested. As a result of the inability to descend, the aircraft got further and
further off of its planned descent path. In an attempt to recover the situation, the
crew shut off the autopilot and recycled the computer. This fixed the pitch
problem, but unfortunately the crew did not realize that, although they were
tracking the localizer, the automatic capture had also been removed. This
illustrates the need to improve the compatibility between ATC and the automated
aircraft, as well as the need for crews to understand how the automatics work.

The controllers need to understand the capability of the newer generation of
aircraft. For example, an ATC controller may give a change in course or
clearance and expect to see a change in the aircraft course displayed on the radar
screen. But with a modem aircraft, a course change may not be immediate,
because the crew is busy entering the new course data into the flight management
computer and not executing the requested course change.

As a last point, the national airspace must be viewed as a system. Many of the
benefits of improvements in the cockpit cannot be realized without improvements
in the ATC system as well.

Computer Sensitivitv/Main tenance Issues

There have been examples where an automatic system has caused movement of a
control surface. In particular, some uncommanded flap extensions at altitude have
occurred which were later determined to be a maintenance problem. Although
these situations have resulted in increased workload for the crew, no serious
situations have occurred. Computer over-sensitivity is a factor that should be
considered during certification.

An example of computer under-sensitivity occurred during a windshear condition
on a commercial flight. The changes in pitch commanded by the autopilot caused
the autothrottle to inappropriately change the engine thrust. The inability of these
two systems to get together contributed to a 3000 foot per minute descent rate
which was arrested at about 600 feet. The pilot had to take manual control of the
aircraft and elected to execute a go around. No accident resulted. In summary,
the inability of the autopilot to respond adequately to the rapidly changing
meteorological conditions caused large pitch excursions close to the ground.

Training and expertise of maintenance personnel are also factors. Many
maintenance personnel have their primary experience with older technology

aircraft yet they are now responsible for maintaining the new, computerized
systems.

Design

Primary flight displays should be structured to reduce clutter, as well as to be
appropriate for the mode of flight. As indicated previously, electrical failures,
aldiough rare, do occur and the designs should maintain critical flight instruments
under these conditions.

Consideration should be given to improving the CDUs to reduce the current
keying requirements, i In busy terminal areas, there should be minimal
interaction with the CDU systems. The CDU does, however, invite crews to "play
with it."

Soft Failures

Soft failures are situations which were not anticipated by the manufacturer, so
there are no pre-specified procedures, but something is clearly wrong with the
cockpit display (i.e., the failure is not significant enough for the computer to
indicate a failure). Such failures cause difficulties because of the nearly
impossible task of predicting them. This, in turn, causes problems in preparing
appropriate operational procedures.

One carrier has had at least two documented cases of computer "soft" failures. In
both cases, the display showed no electronic course line. The first time this
happened, crew training had been oriented toward reliance on the electronic map
and there were no pre-defined operational procedures. The crew was not exactly
sure how the IRU (inertial reference unit) and the FMC were linked together.
They landed the aircraft safely but were not able to correctly diagnose the
problem. It was written up as a triple IRS failure which had, in fact, not
happened since the display still had other navigational data. Changes were then
made to the educational program and as a result, the second time this happened,
the crew was better prepared for the situation. The captain simply selected the
VOR mode and safely continued the flight.

Basic Standards

Basic standards regarding implementation of automation across aircraft are
needed, particularly for default modes. For example, the normal default mode,
when the autopilot is engaged, should be speed and pitch hold. In the example

^ Editor's note: The Boeing 747-400 and MD-11 have improved this.

cited previously, where the default mode is the vertical speed hold, the situation
would not be very comfortable with one engine out.

Minimum Equipment List (MEL)

We may do ourselves a disservice by the specification of equipment on the MEL
which allows a degradation of the total system. One example is the need for an
operational APU (auxiliary power unit). Under certain conditions, it is not
required. Yet, this is clearly not an optimum situation if an engine failure should
occur during a Category n landing.

Workload

Workload on the pilot not flying, particularly in a terminal area while the
aircraft is being flown manually, can be very high.

Britannia Airways Ltd. has used heart rate^ data to augment subjective pilot
ratings of workload. Captain Stu Grieve of Britannia showed data on heart rate
of pilots under various conditions. Heart rate measurements were taken for crews
flying the Boeing 767 and the 737 which have very different levels of
automation. Both take-off and landing flight phases, as well as different operating
modes, were measured. The difference between the 767 and 737 is illustrated in
Figure 9 for similar ILS approaches at Luton using the flight director. The heart
rate for the 767 approach is about 10 beats/minute lower than for the 737.

Figure 10 compares the heart rate responses during standard instrument
departures out of Luton. On the 767, the autopilot is engaged at about 500 feet
before the aircraft is cleaned up. On the 737, due to noise abatement procedures,
the autopilot is engaged after the flaps are retracted and the aircraft is in trim.

As a last comparison. Figure 1 1 shows the difference in heart rates for different
operating modes during a standard instrument departure from Luton in the 767.
Compared to hand flying (bottom trace), heart rates are reduced when an
autopilot (top trace) is used. Rates are also reduced when a flight director which
is driven by the flight management system (FMS) is used (middle trace).

In summary, for the take-off and approach to landing phase, the Boeing 737
crews had generally higher heart rates than the 767 crews. However, the rates
during the acmal flare to touch down flight phase were approximately equal for

2 Editor's note: Since heart rate can be affected by factors other than workload, data interpretation
can be an issue. These data are, however, presented without interpretation.

both aircraft. These heart rates were also higher for actual flight conditions than
would be expected in the simulator and this was probably due to an inability to
properly simulate the real world, particularly wind conditions.

Heart Rate
bpm

130

100

130

100

IF/D ilS)

B737

10.

B767

Land

APPROACH and LANDINGS— LUTON
Comparison of Heart Rate Responses for the B737 and B767 — Same Pilot

Figure 9

Heart Rate
bpm

130

100

J L

A/P

B767

130

100

-10.

Rolling

B737

A/P

SIDS— LUTON
Comparison of Heart Rate Responses for B737 and B767

Figure 10
30

Heart Rate
bpm

B767 SID» LUTON

130

100

130

100

A/P

Hand-Flown F/D

130

100

10a

Rolling V„

Hand-Flown

B767 SIDs— LUTON
Comparison of Heart Rates for Different Operating Modes— Same Pilot

Figure 11

CONCERNS FOR CERTIFICATION OF AUTOMATED AIRCRAFT

The Airbus 320 accident in Europe which occurred in July, 1988, just prior to
the workshop, was discussed. The information available at that time was,
however, necessarily incomplete. Since more complete reports are now available,
the accident discussion is not included here. Several important points regarding
operational certification of advanced technology aircraft were, however, raised
and these are described.

Crew alerting: Crew alerting should be examined carefully during
the certification process for automated aircraft. In the A320
accident, apparently minimal crew alerting systems indicated the
onset of an unsafe air speed. There were, apparently, no aural or
tactile (stick shaker) warning devices. The only indication was the air
speed indicator itself. A question, from the audience, was raised
conceming aircraft designs which prevent a pilot from taking action
instead of alerting him/her. Escape from windshear is another
problem area.

Manual operation of an automated aircraft: Since the accident
occurred during a manually controlled fly-by, the implications for
operation of automated aircraft in the manual mode need to be
carefully examined. Procedures for operating highly automated
aircraft in the manual mode need to be fuUy understood.

Crew experience requirements: As a general concern, less
experienced pilots may be more likely to be assigned to aircraft like
the A320 because of its smaller size. The impact, if any, of relatively
limited flight crew experience on safety of flight is not known.

Crew over-confidence in automation: Questions have been raised
regarding the apparent over-confidence of crews regarding the
capabilities of automation. In the A320 accident, the question was
asked, "Would the crew have attempted such an extremely low, slow
fly-by in a more conventional aircraft?" Although there is no way to
answer this question, the consensus seemed to be "No." It was also
stated that the A320 is a very impressive aircraft and is capable of
maneuvers that are not possible in conventional airplanes.

Pilots switching between automated and less automated aircraft: The
ability of crews to switch between automated and less automated
aircraft in a safe manner remains a concern. If crews become
accustomed to automatic protection features, will they be able to
adjust to a less automated aircraft environment without considerable
additional training?

Circumstances where automatic protection is a clear benefit: The
A3 20 accident may, however, prove to demonstrate die benefit of the
automatic flight envelope protection features. Although preliminary
information suggests tiiat the alpha floor protection was inhibited,
the aircraft appears to have remained stable as it descended into the
trees. The angle of attack protection probably had the beneficial
effect of lessening the severity of the impact.

The concerns raised about automated aircraft certification are best summarized
by the phrase "vuhierable systems, fallible humans." The unfortunate A320
accident was perhaps an example of one or both.

OPERATIONS SUMMARY

In his summary, Capt. Al Ogden of United stated that a combination of factors
are frequently involved in incidents with automation. Some of these are:

1) Inadequate operational knowledge. Lack of adequate operational
knowledge can lead to a failure on the part of the crew to
understand how to operate the systems. Training should
emphasize the integration of the systems and how they work
together. We tend to teach how the system was designed by the
manufacturer, not how the systems are operated or integrated.
We tend to teach "How does it work?" when we should be
teaching "How do you work it?"

2) Cockpit discipline. Allocation of responsibilities between the pilot
not flying (PNF) and the pilot flying (PF) should be rigorous. For
example, at United, the PF does not set the altitude window. The
focus of communication in the cockpit is the mode control panel
(MCP). United has been very successful with this approach for the
MCP but less successful with the FMC/CDU.

3) Cockpit resource management: Workload needs to be carefully
controlled. Too many tasks can be placed on one person,
particularly when data entry ("reprogramming") is required.

4) Split mode operation: Operation in spUt mode (i.e., operation of
the autopilot without the autothrottle) is discouraged at United.
For example, either aU autoflight or all manual is encouraged.

5) Switch discipline: Simply stated, this means know which switch
you are going to move before you move it.

Capt. Ogden also suggested the following list of needs:

1) Better Technical Publications are needed particularly on "How
do you work it?" Current training material explains very well
how the systems work, but is not as good on how to work the
automatics.

2) Analysis before action should be stressed and peripheral
distractions should be eliminated before beginning the analysis
process.

3) Tighter standardization of operational procedures is needed.

4) Better definition of tasks and priorities would be helpful.

In summary, automation is a definite plus which has many benefits including
workload reduction, but it is not a panacea. Automation must work for us, and
not the other way around.

PRESENTATIONS

AND
INVITED PAPERS

Invited Paper
FIELD STUDIES IN AUTOMATION

Dr. Earl Wiener, University of Miami

Susan Norman: Dr. Wiener has been conducting a field study with several major
carriers regarding the introduction of automated aircraft. Although the smdy is not
yet complete, Dr. Wiener requested and was granted permission from the
cooperating carriers to present the interim findings. Part of the success of the
workshop was due in part to the diversity of opinion presented. Dr. Wiener was
asked, as an unbiased observer, to candidly raise some of the more critical issues
regarding the pilots' operational perspective of automation. His findings and the
resulting discussion are reported here.

Dr. Wiener:

This afternoon, I'd like to present an interim report on a field study I am
conducting on the 757 in cooperation with two large airiines. Field studies are a
window on the real worid. The ASRS database is another window on the real worid.
By looking through these and other windows, we can leam important things about
the operation of the world. The title of this study, shown in Figure 1, involves error
analysis, but error analysis is just one of its features.

ERROR ANALYSIS AND PREVENTION
IN HIGHLY AUTOMATED AIRCRAFT

A FIELD STUDY OF 757 CREWS

EARL L. WIENER, PRINCIPAL INVESTIGATOR
EVERETT PALMER, CONTRACT TECHNICAL MONITOR

Figure 1

PREC:

^^g^ji,,,^0ma»^^^

A field study brings together a large number of very diverse groups. The first job is
to actually bring them together. I sincerely appreciate the whole-hearted support
I've gotten from the two airlines and from the safety committees of ALPA. I
particularly appreciate it because both of these airlines in the last few years have
had acquisitions and other major challenges. It would have been very easy for them
to keep a proposed research project on the shelf.

As a historical perspective, in 1979, I came to NASA- Ames to work with Ren
Curry on a new automation project. We spent a year trying to figure out what the
automation project should be. I felt it would be interesting to get out into the field
where real people were doing real jobs and see what was going on. Ren launched a
study on the Boeing 767 and we agreed our main interest was in crew transition. We
wanted to see what happened in those early months when people transition from
traditional aircraft to those with new technology.

That study was done at Republic Airlines. It was ideal for what I wanted to study.
They had experienced, traditional DC-9 pilots, one of the largest DC-9 fleets and
were transitioning into MD-80s. The MD-80 was the first of what I considered
modem aircraft in the Republic fleet. So except for whatever miUtary or corporate
experience their pilots might have had, the DC-9 represented their most advanced
technology. Furthermore, since they were transitioning from a DC-9, the 2 crew
versus 3 controversy would not be a factor in their transition to this new advanced
cockpit technology aircraft.

AUTOMATION FIELD STUDIES

Wiener (1985)
Curry (1985)
Wiener (1988)

MD-80 Crew Transition

B-767 Crew Transition

B-757 Error Analysis

Crew Coordination

Training

Workload

Figure 2

My present study on the 757 seeks to extend beyond just crew transition into flight
crew errors in advanced technology. I'm interested in crew coordination
particularly in the modem aircraft. Training, which is an obviously important
factor, has long been of interest to human factors investigators. Finally I want to
talk about workload and workload management.

WHY DO FIELD STUDIES?

• Realism of the operational world

• Line crews are untapped source of data

• Problems often do not appear until years of line
experience

• Opportunity for an impartial "outsider" to study the
system

• Feedback to: operational world

designers

research world (NASA)

Figure 3

Why are field studies important? One reason, from a researcher's point of view, is
that they give us a chance to get out of the laboratory into the real world. Line
crews are a great source of information because they are the ones actually involved.
There is a strong feeling on the part of line crews that their experience and advice
are not being sought. Too often, only the perspective of management pilots or
officers in the union are sohcited, but line pilots are the ones who see (and know)
the way these airplanes operate in the real world.

There is a lot to be said for this view. For as you know, many problems do not
appear until after the airplane has "matured." Line flying is the acid test of design.
You never really know how a design will work until it gets out on the line and is
flown under a variety of conditions. Finally, one focus is to provide feedback from
the operational world to those who are not in the operational world.

A final important factor is that this research is impartial. I do not design, sell, or
operate aircraft. My purpose here is to be able to feed back the results of my
research to those who can use it — the operational world, designers of the aircraft
and their systems, and the research world.

BASIC INFORMATION SOURCES
IN FIELD STUDIES

Questionnaires
Interviews — crews
Interviews — check airmen
Attendance at ground schools
Jumpseat observations

Figure 4

A basic source of information in field studies is questionnaires. We have an
elaborate set of questionnaires for use with volunteers. We also use face-to-face
interviews, one-on-one, or one-on-two with the crews. I interview check airmen,
management pilots, simulator instructors and ground crew instructors to get their
views — what they like and don't like. I also attended ground schools at both airlines
and have made many jumpseat observations.

Let me also mention the source of our volunteers. Once the study has received
approval from management and ALPA, we appeal for volunteers with a form for
them to sign up on. They fill in their own ID code and no copy of the ID codes are
retained after the data have been analyzed, so we really do not know who the people
are. We got 201 volunteers out of this process.

Figure 5 shows all of the seats previously held on the airline. It averages about 3
seats per person. The 727s predominate. The 727 is the only airplane common to
the two participating airlines. It is the main source of pilots to the 757 largely due to
seniority reasons. Although the pay scale is not much more than a 727, it's the next
logical step in career progression. An interesting fact is that people do not tend to

stay on the 757 very long. They leave the 757 to fly the heavier aircraft. When I
went through transition ground school with 4 others, only one of them did not have
a bid for a heavier airplane in a very short time. This, of course, creates cost
problems in training for the airlines.

SEATS

PREVIOUSLY

HELD

AIRCRAFT

CAPTAIN

F/0

S/0

TOTAL

DC-9

132

—

160

727

113

124

318

A-300

DC-10

747

L-lOU

TOTAL

228

256

225

709

Figure 5

Figure 6 shows the seat these pilots held immediately before going to 757 school.
The 727 predominates. You may wonder about those people who held captain's
positions in heavy aircraft. There were two reasons for their 757 bids. One
significant reason was that they got tired of international and other long flights.
However, the major reason was that they were interested in the new technology.
And crews were bidding downward into lower paying seats because they wanted to
experience the 757 technology.

The same motivation seems to be true of others. There is not much of a money
advantage in moving from the 727 to the 757. The driving motivation was flying
the most modem plane. It was an entirely personal motivation which very much
affected the attitude these pilots had as they went through training and onto the line.

Now I'd like to show you one of 36 attitude questionnaires. These are Likert or
"intensity" scales which measure attitude. The questionnaire makes a positive or
negative statement. The respondent either agrees or disagrees with the statement, at

some level of intensity. The scale goes from strongly agrees to strongly disagrees.
Figure 7 is one example.

SEAT HELD IMMEDIATELY
PRIOR TO 757 SCHOOL

CAPTAIN

F/0

SIO

TOTAL

DC-9

—

B-727

102

A-300

L-lOll

DC-10

B-747

TOTAL

166

Figure 6

Figure 7 shows an attitude question on the problem of error and automation. It is
more typical of the kinds of responses in this study. The responses are almost split
down the middle as to agreement.and disagreement There was very little strong
agreement or disagreement on this question. You could probably not collect data
with a more symmetrical curve if you tried.

We should stop talking about "pilot opinion" as if it's uniform, because the answers
we received are varied. For example, one question involved altitude alerts. The
MD-80 does not have an aural tone on the altitude alert and the air carrier's fleet
had about 136 DC-9s with aural alerts. They also had 7 DC-9-80's (MD-80s)
without them. When I asked crew what do you think about not hearing the aural
waming, the answers split right down the middle. Half of the pilots said they feared
they might now bust altitudes because they had been flying DC-9s for years with the
aural alerts. The other half said it was good riddance. They said we've got enough
tones in the cockpit already. It may not be possible to standardize based on pilot
opinions.

The first series of questionnaires was in 1986 (light columns). The dark columns
show results from series of questionnaires given in 1987. The questions were the
same on both.

13. We make fewer errors in the B-757,
than we did in the older models.

25 ■

35 •

■z.
o

Q.
UJ

OL 20

I—
z

o 15

'^ 10 +
5

Phase 1
n=1 66

Strongly
Agree

Agree Neutral Disagree Strongly

Disogree

PILOT'S RESPONSE

Figure 7

The advance of automation has been based largely on three assumptions which Ren
Curry and I have questioned. The assumptions are:

1) Automation will reduce workload. (This was the basis of the
President's Task Force endorsement of the 2-man cockpit; so that
automation could take the place of the third crew member).

2) Error will be reduced. This is a traditional engineer's approach to
reducing error — to automate and try to remove the human, "the
source of the error," from the operating loop. I question this
because even though automation does change the nature and
location of human error, it does not eliminate it altogether. In fact,

we believe that automation can be an amplifier. It tends to tune out
small errors and create an opportunity for gross errors.

3) Automation will be uncritically accepted by crews. We can show
that automation is accepted by some and rejected by others, and that
some parts are accepted while others are rejected.

We found that there was also concern about safety. Basically, pilots liked the
airplane and the automation. Many did not feel it was an advance in safety because
of the opportunity for error. For example, one question was to relate an error that
the pilot had committed or seen someone else make. A very large percentage of the
answers were altitude bust errors which occurred for a variety of reasons including
human error.

An interesting thing about the 757/767 and the A3 10 automation, is that it creates an
opportunity for new or unseen errors. For example, names of waypoints and
identifiers must now be stored in the data base. Now pilots must be able to spell. If
they are cleared to an intersection which is not on the LEGS page, diey must be able
to spell. It's not unusual to hear on the radio, " You are cleared to so and so
intersection." The pilot responds, "How do you spell that?" Of course, the FAA
created this problem when they went to 5 -letter words for intersections. For
example, if you were cleared to the Bridge intersection in San Francisco, how do
you spell bridge? It's BRUJ. But, in the Seattle area, you might be cleared to an
intersection pronounced "Laker," but spelled LACRE.

Another example involves waypoints. A flight from Dallas to San Francisco, for
example, passes over 2 Las Vegas 's — LVS in New Mexico and LAS in Nevada. If
you were cleared direct to Las Vegas and picked the wrong one, there is a very
good opportunity to penetrate a MOA (Military Operations Area).

I rode in a 767 from Dallas to San Francisco. I didn't say anything about the Las
Vegas problem but I had been thinking about it. During the trip the captain said let
me tell you something my co-pilot did last week. "We were cleared to Farmington,
which is FMN and in northem New Mexico. My copilot reached down and punched
in FAM and we were ready to go to Missouri. Of course the moving map display
saved us." The point is that a new source of potential error has been created. (Note:
late in 1988 a pilot submitted an ASRS report. He had done just that. He punched in
FAM and soon after noticed a distance to tiie next waypoint of 1 100.)

The ALPA mapping committee is concemed with the naming issue and they gave
me a list of interesting intersection names (Figure 8).

INTERSECTION NAMES IN NEW JERSEY AREA.

BIGGE, BOGGE, BUFFY,
HARTY, GOOFY, KITTE,
KIPPI, FLYPI, CATTE

Figure 8

The examples in Figure 8 are all within about a 4 inch square on a high-altitude
chart of the New Jersey, Eastern Pennsylvania area. Try reading that list 3 times
quickly without stumbling over it. This, of course, is not peculiar to automation but
it is a problem with ATC with any type aircraft. I showed this slide at a meeting in
Washington in December in 1981. A staff member of the Canadian Safety Board
came up later and told me that they recently had a loss of separation — a very near
midair collision. The intersection names were confusing (BADDE, BANNY) and
contributed to the problem. Sometimes I wonder if there is any human engineering
at all, considering the Korean 007 incident, and with names of the tracks in the
North Pacific like NIPPI, NINNO, NOKKA, etc.

Figure 9 indicates the resuhs of a question regarding workload. The distribution of
answers is my favorite — a very nearly perfect symmetrical distribution. It shows
the answers from qualified 757 pilots.

Comment from audience: Doesn't the symmetrical distribution of the answers say
something about the question? How do you validate the question? Or does it also say
that the question, itself, may not be important?

Response: If you mean the wording of the question, it is all-important.

As to whether the question, itself, is important, it is necessary to see all 36 questions
together and then look at the ones that are grouped. This morning I've only been
showing fragments of the data.

18. Automation does not reduce

total workload, since there

is more to nnonitor now.

strongly Agree Neutral Disagree Strongly

Agree Disagree

PILOT'S RESPONSE

Figure 9

Comment from audience: Earl, one thing that this might lead us to look at is the
amount of negative carry over from what's called Phase I ground school/Phase II
simulator type training where there is a very strong emphasis on using the
automated features. Early in the introduction of our airplanes in 1982 there was a
reluctance to accept automation, but, as the pilots became more comfortable, they
not only accepted automation but became more dependent upon it. Some training
programs in the initial phase of ground school had this emphasis. This question may
indicate a negative carry over as a result of the fact that the pilot has not been
trained to look out the window or because he just does not have the time because of
his workload. The question is, is it pilot-induced workload or is it man-made
workload because of the design of the aircraft? Or is the basic problem the design of
the training program?

The most consistent complaint we hear about the training program is that too much
time is spent on the CDU and on the automatics, and not enough time on the basic
flying of the airplane.

Response: There is considerable concern about this question of the amount of time
spent focused inside the cockpit by both pilots when the aircraft is below 10,000 feet
coming into a terminal area. This is something we really must explore.

There is a movement that Ren Curry first identified when he talked about "turn it
off' training. As people gained experience with automation, more and more they
were doing the opposite of what was expected. They started turning it off below
10,000 feet coming into the terminal areas particularly in an area like Los Angeles
where "musical runways" or changes in runway are frequent. This is a training
problem, a standardization problem, and a crew coordination problem as well.

Much of "who does what" breaks down in the terminal areas and if you ask line
pilots or check airmen, the one thing they are concemed with is heads down in the
terminal area. It is interesting that I heard the same thing 5 to 7 years ago when I did
the MD-80 study. The MD-80 doesn't have half of the heads down opportunities
that the Boeing 757 does. Essentially, the MD-80 has a mode control panel, but not a
CDU.

That is one of the problems with the workload question. More and more people are
saying essentially, "tum it off if you get busy. You can see why I call that the
paradox of automation. (Figure 10) When the workload gets heavy you tum off the
thing that was designed to reduce the workload. Now again, is it a training
problem? Is it a crew coordination problem? I think more tiian anything else it is a
concept problem. There is something wrong with the basic concept of the design of
the automatic features today. And that's why I say that years from now we'll look
back and call this the era of clumsy automation.

THE PARADOX OF AUTOMATION

"A LOT OF TIMES WE JUST CLICK IT OFF
AND GO BACK TO MANUAL IF THE LOAD
BECOMES HEAVY/'

- 757 pilot

Figure 10
Question: Is the automation the problem or the interface with the automation?

Response: I'm not making a distinction. Clumsy automation in my mind refers
largely to the entire program.

Comment from audience: I am confused. Are you only addressing the reduction of
workload when you mention the possibility of using automation to get rid of
excessive workload? If you consider the new systems automation in the cockpit such
as the FMS and the advanced autopilot, which have not been mentioned, it is another
story. The present transports only address the way that the flight engineer has been
removed by management of flight engineer duties. Now we are addressing a whole
new set of tools.

Comment from audience: I'd like to comment in this area too. I think that the
statement about automation — turning off the CDU'or the autopilot and using
automatic flight coupled with a CDU was ambiguous or misleading. This is quite a
different issue from turning off the EICAS or the other automated systems and
flying the airplane in a truly manual state.

Response: I don't think anybody has any argument about the EICAS. The problem
is control in navigation or the speed control aspects. But this is essentially an
interface problem. The CDUs are difficult to operate and a sizeable number of
experienced crews were saying, "when the workload gets heavy I click it off."

Comment from audience: Is this almost a mis-use of the automation? For example,
the crew should not try to reprogram under those conditions. The ASRS has been
getting a lot of incidents where the pilots say "we shouldn't have been
reprogramming." In many cases, it's not the way they were taught, but they
reprogram anyhow. This concern about aircraft and their automation may have
been overstated. Some of the problems we see may not be related to the automation
itself, but arise because we use less than optimum procedures and then do not train
very well for the less than optimum procedures. The automation is then judged as
no good. That, of course, is very much overstated. In ASRS structured callbacks to
pilots, we ask how they handle this very real problem. More and more pilots are
responding, "I only reprogram when I can."

Response: This is a delicate balance we must reach. Another aspect is an ATC
system that simply is not cordial to these aircraft. The classic case is coming into
Los Angeles from the east where we are having problems, because of all the
reprogramming required. The crew is set up for 24R and about the time you get to
Twenty-Nine Palms or Hector, ATC switches to 25R. If the crew is a good guesser,
they know where they will end up and are ready for about 4 routes — back and
forth, over CIVET, coming in over Los Angeles, one more runway change. And
that's when there is a division of opinion. Should you click if off, or should you
program it? There are very strong opinions about that. I don't think we're going to

resolve this question, i.e., the incordiality of the ATC system, before the end of the
century.

Comment from audience: Your study includes a number of pilots— their
preferences and opinions, but Figure 10 represents only one viewpoint which
happens to be a very powerful opinion. It is probably not fair to overgeneralize,
particularly when another pilot stated, "I can change it in 3 seconds." Figure 10 is a
single viewpoint whereas statistics require hard numbers. A single statement may
give the erroneous impression that everybody is clicking it off and this could be
incredibly danming to the system.

Response: I agree; that's a valid criticism. I did not mean to say that this opinion
represented the whole population. I am trying to simply put forth a few ideas from
the pilots I flew with and interviewed, and I'll try to give both sides of the issue.

Comment from audience: We understand what the pilots are trying to say in Figure
10, but a statement like this requires more detailed information to understand what
is behind it. Examining only one aspect of automation from one particular reply can
be misleading. We have learned a lot regarding the interface of this particular
system and it can be improved. We're all working on it, but the quote on "the
paradox of automation" must be used in the proper context.

Response: I agree, but I want to describe the paradox of automation. Very
frequently, but not all the time, pilots turn off the automation in response to heavy
workload .

Comment from audience: It is very important to be specific regarding what aspect
of automation. Exactly which systems are they turning off? They don't tum off all
the automation. For example, the EICAS system is still used.

Response: Yes, that is a good point. Not all automation is tumed off. Crews may
tum off the LNAV, VNAV, and go to basic autopilot and flight director mode.
They don't tum off the yaw damper, for example. But there is still a paradox which
is not a lack of training, a lack of utility or even a lack of devotion to automation.
Some aspects of this technology are occasionally inappropriate to use during the
required maneuvers of one particular system.

Comment from audience: From the standpoint of our operations, until the advent of
the 757/767 aircraft, when did anyone ever think you could make an NDB (non-
directional beacon) approach on an autopilot? We suddenly had an airplane that not
only gave you an autoflight system that you could do an NDB approach on the
autopilot, but it gave you a moving map so that you not longer had to put your
finger on the chart and follow along as you were also flying the approach.

We put all this information in front of the pilot and showed him how to use it. He
was trained on this tool and given the "Madison Avenue" approach about how great
automation was by the manufacturer, the vendor, the public in general. Then, when
the pilot flew into the Miami area, the controller told him, "maintain 220 knots until
further advised and expect clearance for an NDB approach." And shortly another
ATC clearance to "Slow to 170 knots." At that point, the pilot must slow to 170 and
configure the airplane. But, the autopilot won't respond fast enough. The only way
the pilot can get out of this dilemma is to tum the automatics off, pull the speed
brake, drop the gear, and get caught up.

It's not a failure of automation. It is because the man and the machine have not been
able to interface with the ATC system. It's not a case of whether you have an EICAS
or whether you have a yaw damper system. It's a case where the human, machine
and the operational environment are not compatible. That's the real question. So, is
that a paradox of automation? Quite certainly it is.

We train crews, give them tools, show them the disciplines and procedures, but they
cannot make effective use of these techniques in the field. At the end of a 15 hour
flight, would you rather do an NDB approach on the autopilot or would you rather
do it by hand? But to force the choice of the hand flight mode, because the situation
is not working the same as the training courses, creates the problem. Therein lies
the paradox.

Chair's comment: The idea of Earl's presentation is to raise some of these issues and
by the discussion, I believe we have identified an important issue. I suggest that we
summarize this issue and continue with Earl's presentation.

Comment from audience: I'd like to support the previous statement. There is a
paradox. The problem was correctly stated as well as the solution. It is working
with ATC. An approach into Los Angeles is a prime example.

Comment from audience: One brief comment, just for balance. Earl mentioned that
pilots haven't seen controllers in the jump seat lately. I'm getting exactly the same
response from the controllers in the Centers. It has been years since the pilots have
been in the Center; it goes both ways, which points out the need to look at the system
in terms of the mis-match. Things like simply asking controllers to do more is not
the answer to the problem.

We can ask both people to do more. More controller jump seating and more pilot
trips to the Center. will certainly help coordination.

Comment from audience: I'd like to make an observation. Several times it has been
mentioned that one of the problems is an ATC system which is not friendly to
automated airplanes. This may suggest a conclusion that it is OK for the older
airplanes. But the last minute runway shift is as bad for the 727 as it is for the 767,
because the crew has still got to pull out the chart and do the briefing. We should not
conclude that ATC should be more sensitive to the automated airplane or that such
aircraft should be given special treatment.

Response: This is not a single problem. The fundamental problem is the ability to
deal with everything from Lear jets to 767s, right on through the system and I don't
see a change coming in the near future.

There is another problem area regarding what people have to do to "cheat" on the
automated system. My students know how to "cheat" the computer, but I didn't
think I was going to see it in the 757s. Things like getting down early so you can
make it work. For example, crews do ingenious things like programming fictitious
tailwinds, or inserting an altitude for thermal anti-ice (TAI) (but not turning it on)
so it will get the aircraft down earlier.

I would like to discuss training next. This is an area that requires a lot more
thought. There is a change in training for automation that may be qualitatively
different. Questions such as, "would you introduce the automation first as most of
the airiines do now?." This is a basic question, but we really don't know the answer
at this time.

Skill deterioration or more dramatically, automation apathy, is another issue. In the
field studies we have done, I don't see any such signs. Pilots are not concemed about
it as long as they personally keep up their skills by hand-flying. In the MD-80, the
pilots expressed concem, but when they went back for their first Proficiency Check
(PC) (in a DC-9-30 simulator), they had not suffered any skill loss. In fact, they said
they flew better when they went back to the -30 simulator and did their first PC than
before they had -80 time.

Question: Were they flying both airplanes?

Response: Yes, they were at that time. Later, about half way through the study, they
flew in separate schedules, based on flying only the -80 for 9 months. Each of the
pilots works out his own "training program" using hand-flying, flight-director
only, or occasionally raw data only. I called it the "Personal FAR" in the -80 study.
About 90 percent of the people do this. Another related issue is a copilot who flies
automated aircraft and then suddenly qualifies as captain on the DC-9-10 or 737-
100. Skill loss has not been the problem, but our methodology may not be very
sensitive to it. It's always a pleasure to find something that is not a problem.

Question from audience: Have you looked at carriers that prepare their pilots to fly
automated all the time?

Response: The carriers differ on this. As a matter of company policy, one airline
requires that crews do not use it all the time. Another company, for example in the -
80, had a "we bought it, you use it" policy. However, the pilots would simply click it
off and fly manually, when necessary.

With respect to training, the opinion of one flight manager was that there was a too
rapid introduction of the technology into the training program, and not enough
information on the basic airplane.

Another interesting comment from a captain about PCs in checkrides. He said,
"Formerly, when I went for a checkride, the FAA was always turning things off.
Now they tell me I have to tum things on. They don't want to see you operating an
airplane without automation."

This comment in Figure 1 1 came from a young first officer and in my opinion, it
generally reflects pilot attitude. They all thought that they were "going back" —
going back to a 727, "going back" to a DC- 10. It was a phrase we heard often.

'TLL TELL YOU WHAT IT'S LIKE TO GO FROM
THE 757 TO THE 747: IT'S THE GREAT LEAP
BACKWARD"

Figure 11

As a final note. Figure 12 summarizes the high enthusiasm for the 757, although I
have mentioned some reservations about the safety aspect. Workload may be
increased or decreased. Of course, it appears to be increased at the phase of flight
where you would not want an increase, and a decrease at the phase of flight where
you would not want a decrease. I don't think that is a reflection on automation,
rather that it is in the nature of the flight itself. It's a major challenge for those
designing the future systems.

Some first officers are very fast. They can have route 1 and route 2 both loaded
before I can even see what they are doing.

There is a great variety in time required. There can be differences between captains
and first officers and the speed of each group. One person can be a whiz on
computers and really loves it while another person may have a different kind of
wisdom. You can see it in the training. And, of course, I only observed a small
number. I talked to the simulator instructors and training captains and they
confirmed this. As you watch people in the initial phases of ground school, the first
officer learns quickly, but some of the captains had difficulty. But in the two weeks
after they left ground school and went to the simulators, the captains had
accelerated— taking advantage of their experience. In the simulators, the captain
suddenly caught up.

Question: Do you think the term programming, or reprogramming, is the correct
word to use in the cockpit? I think we should not use those two words.

Response: Why is that? Because you don't consider it "programming," but rather
"loading data" or something like that?

Comment: It's an airplane, not a computer. I think those two words are dangerous,
if one gets used to them, because we lose track of the fact that we are still flying an
airplane.

Comment: In our early training, the primary reluctance for the captains was over
this issue. The word "program" placed a cloud of uncertainty, especially over the
older crew members. In 1982, I went through training with a group of people
whose average age was 48-53. They were very, very reluctant. Some of these
people had not been through ground school in 16 or 18 years. They approached the
training in a very timid manner, and when the word "programming" was used, they
either literally had to be held back from pressing the execute button too soon or
would have paralysis when it came to placing their finger on a key and making a
keystroke on the CDU. They could not bring themselves to do that because of,
again, the specter of "programming."

Comment: If there could be a better word or term appHed to it, I beheve we could
reduce probably 40 percent of our problems. I tell our people, "slow down 10
percent and improve your accuracy 40 percent," and you could probably take away
another large percentage when you simply change the terminology.

Response: There are still difficult problems to solve on the CDU, like entering a
route with two jet airways intersecting each other. And, you may not want to call it
programming, but that's what it is. It is not simply data entry. For example, there

If the money and quality of trips

were the sanne, what would be your

first choice of aircraft to fly?

-z.
o

I —

Q.
U.

UJ
CD

120

110-

100
90 t
80
70 •■
60 •
50 ■
4.0 •
30
20
10

n=133

B-757 DC-10 B-727 B-747 L-1011 A-300

AIRCRAFT

Figure 12

I have data that I don't have time to present to you today, but it will be published in
the final report for the research project. The final questionnaire will have
information on how many ADF approaches, VOR, localizer, CAT-II, CAT-III,
autoland and "man-made" approaches.

Question: On the programming of the CDU, from your observation, what is the
distribution on the time required to reprogram the CDU? For example, in a musical
runway situation requiring reprogramming.

Comment: You know, of course, that this is not the only time that reprogramming
is required. Reprogramming time is largely a function of experience. There seems
to be a critical point at about an experience level of 200 hours. It was about 50 hours
in the MD-80, and the differences were not as great as with the 757. But in the 757 it
took about 200 hours to get really proficient. Some of the pilots are very proficient.

are two jet airways and you have to fly one, and then change to the other, and there
is no named intersection. In my mind, that's programming.

Susan Norman: In the new technology, we should be clear about the meaning of
terms. We have prepared a list of terms (see appendix) and hope that they will be
useful. I hope you will consider Earl a valuable resource for discussion, but in the
interest of time, I must conclude this discussion.

THE ADVANCED AUTOMATION SYSTEM (AAS)
FOR AIR TRAFFIC CONTROL

Alden Lerner, Federal Aviation Administration
Washington, DC

PURPOSE

The advanced automation is tomorrow's air traffic control system. It will provide a
new automation system that includes new computer hardware, software and
improved controller work stations. The advanced automation system will provide:

• the capacity to handle the projected traffic load through the 1990s
and beyond,

• the capability to perform new functions to be introduced into the
ATC system through the 1990s,

• increased productivity through the introduction of new sector
suites,

• a high degree of reliability and availability, and

• the capability for enhancement to perform other functions subse-
quently introduced into the system.

APPROACH

The advanced automation system (AAS) will be designed through a top down,
evolutionary, total system approach. Controller sector suites will consist of
common consoles used for both enroute and terminal (approach control) functions.
They will incorporate an improved man-machine interface, including the use of
• color displays and electronic presentation of flight data to enhance controller
productivity. The advanced automation system will make possible the full
integration of enroute and terminal operations in the area control facilities.

Transition to the AAS will consist of five phases:

1) implementation of the Peripheral Adapter Module Replacement
Item (PAMRI),

2) implementation of the initial sector suite system (ISSS) to provide
new controller work stations,

3) implementation of temiinal advanced automation (TAA) functions
using AAS hardware and software,

4) implementation of tower control computer complexes (TCCC), and

5) implementation of Area Control Computer Complex (ACCC) for
the remaining AAS enroute functions.

Phase one, implementation of the Peripheral Adapter Module Replacement Item
(PAMRI), which includes replacement of the PAM and Data Receiver Group
(DRG) equipment, will be implemented prior to ISSS equipment delivery. This will
provide an enhanced ability to interface with additional radars, and wiU provide a
capability for use of higher data transmission rates for radar site interfaces.

In the second phase, initial sector suites will be installed in enroute facilities served
by the host computers. The sector suites (work stations) will be the first AAS
components to be operational, providing early gains in controller productivity. The
consoles will feature large, multiple color displays that will provide situation
traffic, weather, and flight data as well as a "look ahead" plarming capability.
Although each console will have its own embedded microprocessors to drive the
displays and perform related tasks, most of the data processing required for
controlling traffic will be performed by the host computers. First delivery of an
ISSS will be to the FAA Technical Center in Atlantic City in September, 1990,
where it will undergo extensive test and evaluation. First site delivery to the Seattle
Center is scheduled for April, 1992. The equipment is expected to be operational at
all sites by June, 1995.

The third phase will be the implementation of the TAA for TRACON functions.
Following the transition to ISSS for enroute control, the old control rooms will be
refurbished to accommodate the additional sector suites necessary to provide
approach and departure services at the approximately 200 airports that now have
their own terminal radar control rooms. Deliveries of new AAS hardware
processors, software, and additional sector suites to support terminal operations
will begin in 1994.

The fourth phase will be the installation of TCCCs in selected airport traffic control
towers. The TCCC will include new controller position consoles with supporting
computer hardware and software and new electronic displays for flight data and
radar information. Controllers will use the radar data as an aid in tracking aircraft

in the terminal area, as they may be required to provide limited approach and
departure services. The contractor will begin delivering TCCCs to upgrade the first
150 airport towers included in the basic contract in 1994. The last site (258th, if all
contract options are exercised) would become operational in 1999.

The fifth phase in the evolution to a full AAS will begin in 1994 with the installation
of the remaining computer software required for the enroute air traffic control
functions (ACF). Additional sector suites will be installed to enable conversion of
today's ARTCCs into tomorrow's ACFs. The hardware/software associated with
this step will be named the area control computer complex (ACCC). Once this is
completed, the host computers at each location can be removed along with the
current back-up system in the enroute centers. The completion date for this phase is
August, 1998.

Included in this phase would be the addition of the Automated Enroute Air Traffic
Control (AERA) software into the system to aid controllers in effective route
planning. Known as AERA-1, it will probe requested flight routes to detect
potential conflicts with other aircraft, violations of protected airspace, and
conformity with air traffic flow restrictions. With this information, the controllers
can select the safest and most fuel-efficient route available.

Future AAS enhancements not covered by the contract will include AERA-2 and,
possibly AERA-3. AERA-2 would present controllers with solutions to the
conflicts detected by AERA-1 and, then, pass along their decisions to pilots by
digital data link. AERA-3, which is still in the concept stage, would include some
degree of autonomy for the AAS computers to detect and resolve problems, make
decisions and provide clearances to pilots under human direction, but without
human intervention.

PRODUCTS

The scope of the AAS project includes:

• Advanced automation system design

• Advanced automation system software for both terminal
and enroute ATC operations

• Advanced automation system computer hardware

• ISSS (20 Continental U.S. ARTCC)

• ACCC (ACFs — including Anchorage, Honolulu, and the
New York TRACON)

• Support systems at the FAA Technical Center and the FAA
Aeronautical Center.

IMPACT

The introduction of the new controller work stations and new communications
networks will greatly impact the controller's physical environment. Further, the
organization of the new equipment into ACFs which will result in the co-location of
terminal and enroute functions in the ARTCC will impact the procedures for
managing the duties of air traffic control, maintaining the systems, and organizing
the work force.

The Advanced Automation System is tomorrow's air traffic control system. Its
sophisticated equipment and programs will improve upon present air traffic control
by:

• enhancing flight safety with new automatic separation-assurance
techniques,

• increasing flight efficiency through more direct, conflict-free
routing,

• reducing congestion and delays through better traffic-metering
techniques,

• increasing controller productivity through new controller work
station,

• handling projected air traffic growth without corresponding in-
creases in personnel,

• providing a system life of 20-30 years; new hardware/software can
be added to the basic design,

• tying together all of the FAA's primary enroute and terminal air
traffic control facilities into an integrated, automated system,

• permitting the consolidation of all radar services into approxi-
mately 23 strategically located Area Control Facilities (ACF),

• providing greater system reliability through a requirement that the
AAS be available 99.9995 percent of the time (maximum down
time of about 2 1/2 minutes per year).

AAS DELIVERY SCHEDULE

Initial Sector Suite System

Delivery to FAA Technical Center ^ September, 1990
Delivery to first ARTCC - April, 1992
First site operational - October, 1993
Last site operational - June 1995

Tower Control Computer Complex

Delivery to FAA Technical Center - April, 1992
Delivery to first airport tower - March, 1994
First site operational - February, 1995
Last site (150th) operational - March, 1999

Terminal Advanced Automation System

Delivery to FAA Technical Center - December, 1992
Delivery to first ARTCC - march, 1994
First site operational - February, 1995
Last site operational - February, 1998

Area Control Computer Complex

Delivery to FAA Technical Center - January, 1993
Delivery to first ARTCC - September, 1994
First site operational - March, 1996
Last site operational - February, 1998

Invited Paper

THE EFFECTS OF AUTOMATION ON THE HUMAN'S ROLE:
EXPERIENCE FROM NON-AVIATION INDUSTRIES

Dr. David Woods, Ohio State University

Susan Norman: Automation technology has been employed by other industnes
for many years. Although the operating environments are not exactly like aviation,
the experiences regarding the technology itself are surprisingly similar. Dr. Woods
was asked to select a few representative examples with relevance to aeronautics and
to draw some general conclusions about their applicability in the transport aviation
envirormient.

Dr. Woods:

I have been asked to comment on the effects of introducing new automated
technology. Rather than start with broad generalizations, I have selected a variety
of specific, actual cases where field studies or controlled studies have been
conducted to determine the effects of new technology on both productivity and the
quality of human performance. The cases that I have chosen to discuss are relevant
in some fashion to aviation, and they are listed in Figure 1.

An important point is that these examples are based on field study results, not
opinions. Of course, there can also be questions of interpretations in field studies. I
was personally involved in some fashion in several of these smdies. Some are based
on reviews of other people's studies on the effects of automation. It will only be
possible to discuss each case briefly but more detailed reports are included in the
references.

The term "automation" has been used so much that it has taken on several meanings.
In several of the examples which follow (process control and computerized
numerical control or CNC), automation refers to autonomous machine systems
because the jobs are performed in a semi-autonomous way, without direct manual
intervention. The other examples concem "automation" in the sense of new
information systems capabilities such as diagnostic systems.

Studies or Field Experience on the
Human Role in Highly Automated Systems

• Computerized Numerical Control in manufacturing

• Banking information systems and processes

• Steel processes

• Nuclear Industry:

• Computerized alarms systems

• Disturbance Analysis systems

• Computerized procedures

• Expert systems

Figure 1

TECHNOLOGY CENTERED AUTOMATION

These cases all illustrate a common underlying philosophy of automation which has
been called "technology centered" automation. The assumptions of this approach to
developing new levels of automation are noted in Figure 2.

The issue is not to automate or not to automate; it is how to automate. How should
the level of technology be increased? What should we do with the technological
capabilities? Choices can be made about how to introduce the technology which
require careful thought. There is no technological imperative — only one way to use
technology.

The fundamental assumption behind most automation development is that people
and machines are comparable; one can be substituted for the other on each subtask;
and the tasks to be performed are independent. In other words, changing the
allocation on one task has no effect on other tasks. It has been difficult to make

progress in the field of human factors because we have been locked into this "can we
substitute a machine for a person" approach to function allocation.

Technology Centered Automation

philosophy of person-machine comparability

allocate to people tasks that are expensive or difficult to assign to
machine

system problems are solved by attempting to shift human functions to
the machine

person is often a convenient and cheap manipulator or perceiver

de facto human role: "plug the holes in the thoroughness of the
designer's work"

Figure 2

The result has been that tasks which are expensive or difficult to assign to the ma-
chine are allocated to the people. We tend to automate the tasks that are automatable
at a reasonable cost. If there are system problems, they are usually dealt with by
transferring more human functions to the machine— solve performance problems
by reducing the human's role in the system. There has been no global systems
context where the person and machine work together.

There is also a tendency to use the person as a convenient and cheap manipulator or
perceiver for the machine, because it is difficult to automate perception. This is
particularly true in the development of expert systems as we shall see later.

In the technology centered approach to automation no thought is given to the
human's role in proper system function. Instead, as Jens Rasmussen has pointed out,
the de facto human role is to "plug the holes in the thoroughness of the designer's
work" (Rasmussen, 1979).

COMPUTERIZED NUMERICAL CONTROL

One of the early automation cases, which is often misdescribed in terms of what
actually happened, involved computerized numerical control (CNC). This case,
summarized in Figure 3, has been one of the most investigated cases of changes in
automation. The original goal was simply to eliminate the skilled machinist to save
money. What actually happened depends on the exact machine application, but in
general, the human role, as well as the skills needed to carry it out, changed (cf..
Noble, 1984; Corbett, 1985; Kidd, 1988). The operator was now responsible for
preventing gross machining errors such as machining the wrong part. But if the
operator was not skilled and knowledgeable in the tooling process, unscheduled
down time and gross machining errors occurred.

Computerized Numerical Control

original ^oals: eliminate skilled machinist

actual results: changed human role and skills to avoid unscheduled
downtime and to minimize the costs of machining errors

critical human role is to adapt to unanticipated variability with success
depending on:

• machinist learning and inferring what is the
computer plan,

• where it is vulnerable to breakdown in the face
of different circumstances (e.g., tool wear),

• how the computer plan is progressing in a par-
ticular case,

• devising and using ways to directly or indirectly
control the CNC system

Figure 3

The critical human role was to adapt to "unanticipated variabiHty." Conditions still
arose in the machining tool process that went beyond the capabilities of the CNC
machine programs. The human had to help the machine to adapt. The machinist

now had to understand something about the plan resident in the computer program.
He did not have to be able to duplicate it or to write the program, but he did have to
understand what it was trying to do— its intentions. He had to understand where it
was vuhierable to breakdown and what factors gave it trouble.

Tool wear, a classic case particularly in the earlier systems, was a factor that could
challenge the automated systems. The operator monitoring requirements went up —
as the machining progressed through a run, the operator needed to understand how
the plan execution was progressing. Was it going off track? Was it starting to have
problems? This required supervisory control (a theme which repeats in many of the
cases to follow). As a supervisor, the operator now had to adjust and redirect the
system occasionally. The designers, however, rarely provided convenient ways to
accomplish this because they did not think about the machinist's role. It was up to
the machinist to devise new ways to control the CNC systems.

BANKING INFORMATION SYSTEMS

Another example is a study of a shift in the information technology used in the
banking industry (Adler, 1986; summarized in Figure 4). Again the exclusive goal
was to reduce the number and skill levels of the people in the system. The term
"peripheralization" of the human role was first applied by Adler during this study.
What he meant is that, as the level of automation increases, the person's role is to
manage and care for the health of the automated system. The new human role was to
manage the operational environment so that the system would stay within its range
of capabilities.

While there were many detailed variations in the skill consequences of this
particular example, there was an overall increase in skill requirements, because the
main role of the people in the system had changed. They were primarily dealing
with anomalies around preplanned routines. There were many preprogrammed
specific banking transactions, but customers often had situations that were a little
different. Somehow the bank operators had to understand what each special case
meant in terms of the automated system. They needed to understand the banking
processes as well as the automation. They could no longer have a narrow task
orientation as they had in the pre-automation days. A broader perspective of the
whole system was now required.

There was also a great deal of concem because the automated system was much
more fragile. The degree of inter-coupling or interaction between the system parts,
as well as the level of integration among the various aspects, was increased with
more automation.

Banking Information Systems:
original foal: reduce number and skill levels of tellers
actual results:

1) peripheralization of human role

2) increase in teller skill requirements: main role is to deal
with anomalies and contingencies around preplanned
transactions

3) fragility of new system due to increase in level of inter-
coupling

4) data integrity is a major issue in part due to low de-
tect ability

5) new error vulnerabilities/consequences (error fre-
quency/cost relationship changes)

6) need for greater training in several areas including the
overall computer processing system and accounting
procedures to avoid or correct errors

7) in general, the increase in level of automation produced:

• new types of task responsibility,

• new degrees of abstractness of tasks,

• new levels of task interdependence.

Figure 4

Data integrity also became a major issue. A tremendous amount of effort was spent
to be sure that the data in the system were accurate. If they became corrupted, wide-
ranging effects on the system were possible. This was important, in part, because of
low error detectability. The result was an increase in the need for "situational
awareness," i. e., an understanding of the state of the overall system and what it is

doing because errors must be detected. These errors, in this case, are not
necessarily human errors, but simply bad data in the system.

Another common theme is that changes in level of automation produce a shift in the
kind and cost of errors. It was not simply minimizing errors. For the first time,
new kinds of errors were possible while others were made impossible. The
vuhierabilities and consequences of errors had changed. The frequency of some
kinds of errors may have been reduced, but some of the new failure forms had
different, higher consequences. The concem was the fragiUty of the system. It was
now difficult to localize and contain errors, since they tended to spread throughout
the system in ways that were hard to detect. Usually mistakes were apparent only
when something came crashing down, such as a customer related problem.

As a last point, the need for greater training was not anticipated and there was quite
a bit of scrambling. Again, remember that the initial reason for instituting
automation was to save money by decreasing the skill requirements. But what
actually happened was people had to understand more about the overall process —
accounting and the banking processes. They had to have some understanding of the
overall computer processing system. Otherwise, they could not do their job of
helping to avoid or correct errors in the system.

In summary, this example illustrates how shifts in level of automation can produce
new kinds of task responsibilities and new levels of abstracmess which in turn
require more conceptual skills on the part of the system operators.

PROCESS CONTROL INDUSTRY

A classic example of automation in the process control industry occurred in a steel
plant (Hoogovens Report, 1976; summarized in Figure 5). Again, the original goal
was to reduce the number and skill level of the operators. The actual results were
quite different. For a period of time, down time was actually increased due to
automatic system breakdowns. An in-depth study of the effects of automation in this
case was prepared and the report states, "the need for the operator to intervene
directly in the process is much reduced, but the requirements to evaluate
information and to supervise complex systems is higher."

Steel Processes

original ^oal: reduce number and skill of operators

actual results: increased downtime due to system breakdowns

"The need for the operator to intervene directly in the process is much
reduced, but the requirements to evaluate information and supervise
complex systems is higher'

automatic control or manual control; no provisions for supervisory
control

• no mechanisms for operators to under-
stand or track what the automatics were
doing

• no mechanisms for operator to direct the
new automation

• authority/responsibility double bind.

Figure 5

Again the theme is the same as in the banking case. The problem was the designers
had designed the plant to work in one of two modes: either automatic control or
manual control. Obviously, there was a manual backup to run the system, but there
were no provisions at all for supervisory control and no mechanisms for the op-
erator to understand or track what the automatics were doing, hi fact, initially, the
operator received barely any training or information about what the automatics
were about at all. hi short, there were no mechanisms for the operators to direct the
new automation.

The situation was made even worse by an authority/responsibility double-bind. The
operator was (or thought he was) responsible for the proper operation of the
system. The effective authority was, however, in the hands of the machine
automatics. The operator was there just in case anything happened or to help the
automatics in situations that were not practical to automate at this time, hi actuality,
the operator was unable to coordinate control of the system. The resulting level of
performance was the performance of the automatics alone. Unanticipated situations

arose which went beyond the capabihties of the automatics. Furthermore, when
trouble arose, the consequences tended to be broader because the system was more
integrated following the increase in automation. The overall result was an increase
in down time after the introduction of the automation.

AUTOMATED FACTORIES

A case, closely related to the Hoogovens experience that is going on today, is in
process manufacturing, where the objective is the automated, or "lights out,"
factory (if there are no people, then there is no need for Hghts). Steve Miller at
Carnegie-Mellon has an interesting program of research in progress in this area
(cf.. Miller & Bereiter, 1986; Bereiter & Miller, 1986; Bereiter and Miller, 1988).
He is working with a major corporation and some of their automated lines to study
the effect on the human's role. Again, the inter-coupling or relationship between
the system components is increased through new automation, and the critical human
role becomes fault management such as avoiding unscheduled down time. However,
there has been very little support provided for this human role, as summarized in
Figure 6.

Automated Factories

original goal: lights out factory

actual results: automation increased level of intercoupling

critical human role is fault management to avoid unscheduled
downtime

Figure 6

NUCLEAR INDUSTRY: COMPUTERIZED ALARMS

In the nuclear industry, there are a variety of cases, and I have selected a few which
are representative of the important issues. These examples can be hard to dig out
because, in this world as in many others, no one wants to discuss technology
failures. The feeling is it would be better if, like old generals, they just faded away.
No one wants to investigate what happened or why, but the lessons for the future are
important

In the British nuclear power industry in the seventies, the tile annunciator alarm
systems were computerized (Pope, 1978), A tile annunciator alarm is an an
engraved tile that is back-lit, and each tile represents a setpoint or component status.
If the variable crosses the setpoint (or if a component is in a particular state — pump
off), the tile lights up. If there is another setpoint on the same variable, it is
represented by a separate engraved tile. There can be thousands of these tiles in a
nuclear power plant, and an avalanche of alarm signals occur during plant upsets.

There are serious difficulties associated with fault management in this situation
related to data overload (Lees, 1983). A computer solution was devised to deal with
the shortcomings in the old system. The computer based alarm system contained the
same raw alarm data — setpoint crossings and component status changes. This data
was now organized in a chronological list (in part, because it was easy to build with
the technology of the day).

When the new computerized system was installed, it was discovered quickly that the
operators could not effectively monitor and track plant state during upset
conditions. The old pilot annunciator system had to be re-introduced. The reason
for this failure was that the designers of the new system did not understand some of
the serendipitous benefits of the old tile annunciator system (the inherent spatial
organization) which were eUminated by the shift to the new technology. In the old
tile annunciator system, even though there were huge numbers of signals, they were
spatially distributed. Each tile was spatially assigned to a location and the operators
could determine some things about the overall problem based on the pattern of
lighted tiles.

The old system made use of human pattern recognition capabilities, and, with
experience, operators could learn to recognize pattems (Kragt & Bonten, 1983). A
chronological list of very raw alami information, however, comes in fast and gets
very long, very quickly. The operator had to refer back down the list and to scroll
back and forth through the screens trying to find out what happened and to build an
integrated picture out of the raw alarm data. This was extremely difficult to do with
the chronological organization, exacerbating and not relieving the data overload
problem.

This theme, summarized in Figure 7, is common in many of the information system
oriented increases in automation. Designers will be technology driven and build the
automation that makes the most sense in terms of cost and the use of the technology.
But the cognitive consequences for the problem solving task, fault management in
this case, are rarely considered.

Nuclear Industry: Computerized Alarms

shifted from tile annunciator alarm systems to a form of computer
based alarm system

original ^oal: improve alarm systems, use computers

actual results: the tile annunciator system had to be reintroduced

serendipitous benefits of the tile system were eliminated in the
computerized system that greatly increased the difficulty of fault
management

Figure 7

Another example occurred on ships when a new electronic display and control
system was introduced into the engine room. Automation designers frequently try
to finesse the cognitive consequences of changes in the person-machine system by
relying on the flexibility of computer based systems-the designer's only
responsibility is to make all of the data available and accessible; it is the domain
practitioner's job to find the right data at the right time. However, a case described
by Moray (1986) illustrates how flexibility alone is not enough in the development
of more automated information systems.

In this case, a new, fully computerized ship engine control room was developed.
There were three CRT screens, and the operator could call up a variety of computer
based displays and controls on whichever CRT he or she desired. A human factors
review of the system predicted that, at some time in the life of this system, the
operator would call up the computer display for the starboard engine controls on
the port CRT and the computer display for the port engine controls on the starboard
CRT-a violation of stimulus-response compatibility guidelines. This could lead to
an execution slip where the operator would unintentionally manipulate the wrong
ship engine.

Shortly thereafter, during simulated shiphandling with the new system, this
situation arose and the predicted resuh followed. Alarms indicating starboard
engine trouble occurred. The operator correctly diagnosed the situation and
attempted to control the starboard engine. He manipulated the engine controls on

the starboard CRT display which display the port engine controls. If this had
occurred at sea during a difficult navigation period, the ship could well have run
aground.

NUCLEAR INDUSTRY: DISTURBANCE ANALYSIS SYSTEMS

The next example. Figure 8, is one of the best illustrations of a failed attempt at
automation which faded away without comment or investigation. It was several
projects that went on about the same time in three different countries to build
"disturbance analysis" systems in the nuclear industry. The projects were attempts
to address problems in operator diagnosis of faults and the deficiencies in alarm
systems by automating fault diagnosis (artificial intelligence techniques were not
used).

Nuclear Industry:
Disturbance Analysis Systems

multi-year, multi-million dollar projects in U.S., Germany, Japan,
circa 1980-1984

original ^oal: carry out automatic fault diagnosis; reduce operator role
in diagnosis

actual results:

1) unable to automate fault diagnosis and did not
make the operator a better diagnostician

2) exacerbated operator data overload

3) projects transformed or abandoned

Figure 8

However, diagnosis in a large complex system like a nuclear power plant is
inherently a very difficult task (Woods, 1988). For example, there are typically
multiple faults and interacting factors that explain the pattern of symptoms. The
disturbance analysis projects were unable to do automatic fault diagnosis and they
did not make the operator a better diagnostician.

In part, this occurred because the data overload on the operators was exacerbated.
The disturbance analysis system generated large amounts of potentially useful,
more integrated information. But now the operator had the task of interpreting all
this data, finding out what was useful and integrating it with the large amounts of
time varying data available from other information sources. While this experience
has generated more interest in human-computer techniques for managing large
amounts of dynamic data, the lessons are largely going unnoticed. Today, another
attempt is underway to automate diagnosis via artificial intelligence techniques.
Interestingly, the label "disturbance analysis" is taboo in the nuclear industry.

COMPUTERIZED PROCEDURES

Another case (Figure 9) concerns computerizing procedural information. There
were a variety of goals such as improving data retrieval from large libraries of
procedural information. In the end, however, the attempt was abandoned because
people got lost in the system.

The data base application in question was a computerization of paper-based
instructions for nuclear power plant emergencies. The system was based on a
network data base "shell" with a built-in interface for navigating the network. The
shell already took into account the basics of human-computer interaction so it was
assumed that all that was needed was to enter the domain information. However,
several factors combined to produce a keyhole effect. These factors were the
organization of the network, the standard navigation features, and the fact that
power plant incidents are dynamic and new events can occur which change how the
operator should respond.

In summary, the operator could only see one procedure step at a time; it was
difficult to refer back to a previous step or to look ahead to see what steps have to be
done next (Elm & Woods, 1985). There was no attempt to provide the operator
with a broad picture of the response plan being executed or where it was going.
Trials with the system revealed that the people who used the database shell to
generate the system got lost. The people who wrote the procedures and tried to use
this system got lost. Experienced operators, who knew what step in the procedures
should be executed given the current plant state, also got lost.

Computerized Procedures

Shift from paper based to computerized procedures.

original ^oals: improve maintenance of procedure information,
improve operator procedure retrieval, ensure rote procedure
following, use new technology

actual results: attempt abandoned due to user getting lost in the
network of data

Figure 9

The end result was that the attempt to computerize the procedures in this way was
abandoned. A side note is that a redesign was done based on a proper cognitive task
analysis of procedure usage. The new design utilized several techniques to avoid
keyhole effects and to support the user. Interestingly, almost aU of this design could
have been implemented within the base capabilities of the interface shell.

EXPERT SYSTEMS

The next case addresses expert systems for troubleshooting. In this one I and my
colleagues were able to investigate how technicians used an industrial expert system
developed from a shell in the standard iterative refinement approach to
troubleshoot an actual electro-mechanical device (Roth, Bennett & Woods, 1987).

The original goal was to reduce the technician's skill requirements. A second goal
was to provide management with extremely tight control over the technicians who
were responsible for troubleshooting the devices.

The expert system was designed to be a stand alone problem solver who would
simply output a solution. But, what was the person supposed to do? The person was
expected to go to the remote site and dial into the expert system to initiate a run. The
machine would then use the person as hands and eyes; the person's role was simply
to serve the machine, to collect data, to carry out various kinds of tests, to make
observations about device behavior and, finally, to implement the repair selected by
the machine expert.

If this model of problem solving was correct, the data that we collected should look
like that in Figure 10. The machine expert would direct the technician to perform a
series of data gathering activities (observe, measure) until the machine produced a
hypothesis and the operator effected the directed repair (e.g., replace the bad part).
Essentially, the expectation is a straight, linear execution of the machine's
instructions until the solution is found.

How much of the time did this happen? About 20 percent of the time. Almost 80
percent of the time, the protocols looked like the one in Figure 11. The machine
generated multiple hypotheses, not a single one. One part of the human role was to
filter the machine's solutions, and decide which was really the right one. People had
to figure out if the machine was on track; as a result, they had to go through their
own diagnostic process. Were the machine generated hypotheses inconsistent with
the operator's experience? If so, the operator would need to supervise the machine.
But, remember that the machine design provided no mechanism for the technician
to direct the machine or to gain access to the knowledge or tools available within the
expert system. So the operators tried to adapt and invent ways to interact with the
system. Specifically, the operators tried to trick the automatics in order to get the
machine to do what needed to be done. In one case, when the machine response did
not make any sense, the operator went back a step and tried the opposite answer to
see what would happen and if the machine's behavior would be more plausible.

The actual results were that a successful diagnosis required significant technician
participation and skill. But, no mechanisms had been provided for the technician to
interact with the expert system, other than in trivial ways. No mechanisms were
provided for the technician to understand and track the expert system's diagnostic
process.

The important thing about this study was that we were able to analyze how the
system performed under various unanticipated conditions, such as underspecified
instructions. This required judgement, because the technician had to use knowledge
of the domain in order to interpret what was really required. Misinterpretations
and mis-entry errors also occurred, as well as errors in the knowledge base itself
(the machine would sometimes make a mistake due to an incorrect rule in its
knowledge base).

Adaptation to special conditions was also a crucial issue. For example, it might not
be possible to carry out the expert system's plan, because some assumption behind
that plan was not true. For example, a tool was not available to do the requested test
or the system assumed a particular device was operable, when, in fact, it was not.
And therefore, the test requested by the machine could not be carried out until the
device was at least minimally operable.

KEY

INFORMATION

OBSERVATIONS

MEASUREMENT

HYPOTHESIS/

VERIFICATION

PROCEDURES

( END ]

CANONICAL

SOLUTION

PATH

MACHINE-
DIRECTED
ACllVITIES

TECHNICIAN-
INITIATED
ACTIVITIES

CHECKS FUNCTIONAL
DIAGRAM;

SOLUTION

Figure 10

KEY

SELECTS MORE GENERAL
SYMPTOM DESCRIPTION

OBSERVATION

CANONICAL

SOLUTION

PATH

1
1

MACHINE-
DIRECTED
ACTIVITIES

TECHNICIAN-
INITIATED
ACIIVITIES

OBSERVATION

VfEASUREMENT^ IMPASSE:
TOOL

RESULTS OF OBSERVATION ARE
INCONSISTENT WITH INITIALLY
REPORTED SYMPTOMS: OBS
REPEATED

OBSERVATION

HYPOTHESIS/

VERIFICATION

PROCEDURES

UNAVAILABLE.
JUDGES LIKELY
RESULT

INCOMPLETE
OBSERVATION.
INCOMPLETE
RESPONSE

KE INTERVENTION: SUGGESTS RESTARTING;

TECHNICIAN RESTARTS. SELECTING MORE SPECIFIC
CHOICE DUE TO KNOWLEDGE GAINED FROM TROUBLE
SHOOTING EPISODE.

Figure 11

HYPOTHESIS

REJECTS

HYPOTHESIS

REJECTS

MEASUREMENTS

OBSERVATION

HYPOTHESIS

REJECTS

MEASUREMENT

OBSERVATION

In summary (Figure 12) we found that there was a significant role for the
operational people in this system. But the expert system, if anything, hindered
human diagnosis. It did not even provide a memory trace of all the diagnostic tests
that had been run up to that point in time on the system. The operator was
functioning without any aid whatsoever, even as simple as a electronic notebook.
Yet he was forced to make a parallel, independent diagnosis in order to do the job of
supervising and interpreting the information provided by the expert system. Again,
the question is not whether the expert system did or did not help; the question is
whether there is support for the human's role. In this case, there was none.

Expert Systems for Troubleshooting

original ^oal: reduce technician skill requirements to reduce costs and
give management tighter control over technicians

actual results:

1) successful diagnosis required significant technician participation
and skill (almost 80% of test cases) due to unanticipated
variability

2) no mechanisms were provided for the technician to direct the
expert system

3) no mechanisms were provided for the technician to understand or
track the expert system's diagnostic process (black box design)

4) some technicians discovered ways to manipulate the expert
system

5) technicians who passively followed the instructions of the
machine made more execution errors

6) technicians had to perform partial diagnosis on their own without
any assistance or information from the expert system

Figure 12

LESSONS OF TECHNOLOGY CENTERED AUTOMATION

In summary, Figure 13 lists the lessons learned when a technology centered
approach to automation is employed. First, the human role in the system is changed
in unforeseen ways. However, if the new system supports the new human role, the

person will be able to effectively accomplish the task. Second, there is a myth about
de-skiUing. Sometimes skills are reduced, but in general, skills are changed. The
mix of skills, the kinds of people needed to operate a system, change. Third, the
critical human role is to adapt to unanticipated variabilities and to amplify the
requisite variety in a system. The person is there to help the machine do the job.
Finally, new kinds of error forms and new kinds of system breakdown patterns can
happen. The frequency and/or relative consequence of an error can be changed, and
these new error forms, as well as their consequences and frequencies, need to be
examined to be sure that everything has moved in a positive direction.

Lessons of Technology Centered Automation

shifts in automation have changed the human role in system
performance in unforeseen ways

de-skilling myth— changed pattern of skills; it does not eliminate
human skill

critical human role is to adapt to unanticipated variability
new error forms and types of system breakdowns

Figure 13

We can also summarize the fallacies in automation across cases like those discussed
above:

• incomplete automation where the machine is assumed to be fully
autonomous when it is semi-autonomous;

• failures to understand and support the human's new or changed
role;

• brittle machine performance due to designer's overconfidence bias;

• pseudo-consultants; machine locus of control.

HUMAN CENTERED AUTOMATION

In contrast. Figure 14 summarizes the lessons learned from a human centered
automation approach. First, a human locus of control is required. The operator
must have effective authority over areas of task responsibility and this cannot be
merely lip service authority. The operator must have control of the machine
resources including some degree of freedom of action and methods to instruct or
direct the lower order machine agents. The machine should always provide support
to the human. Designs requiring a choice between fully automatic or fully manual
operation must be avoided.

Human Centered Automation

1. human locus of control:

• effective authority as well as responsibility

• control of machine resources, i.e., degrees of freedom of action
including ways to instruct or direct lower order machine agents

• the machine should always provide support to the human (e.g.,
avoid cases of forcing the human to chose between doing the task
completely by himself or letting the machine do it completely by
itself)

2. human as supervisor:

• monitoring of lower order agent

• greater need for high level situation assessment

• human tracking of machine state (what does it know, what
is it doing, why is it doing it, what will it do next)

3. support for error detection and recovery:

• communication breakdowns between multiple agents

• use machine intelligence for constraint checking and
critiquing

• support human situational awareness

Figure 14
82

Second, the general role of the human supervisor in these highly automated systems
is to monitor the activities of the lower order agents. This means there is a much
greater need for high level situation assessment and new information displays to
help accomplish this job. The need for human tracking of what the machine is
doing, why it's doing it, and where it's going to go next is also greater. Third, the
human role in error detection and recovery needs to be actively supported.

DISCUSSION

Question: Are you saying that automation should not be used at all?

Response: No. As I said at the beginning, the issue I've been working on is not
whether or not to use technology. It will be used for a variety of reasons. The
question is hfiw is it used. Studies that I've done related to process control, as well as
other studies, have all indicated that more leverage can be obtained from
technology improvements and that some negative post-conditions or consequences
of automation can be avoided if the effect on the human role is taken into account.
Support mechanisms must be provided. It is not possible to say, in any absolute
sense which is better— automation or lack of it. Rather, it is a question of the choice
in level of automation. What post-conditions result for the human's role and how
are they supported? The total system performance should improve. The question is
not whether the person alone can do a better job than the machine alone. I want the
two together to do it better.

Question: What do we have to change? What do we have to do to work around this
problem?

Response: We must be able to provide operational people like you with the tools
(decision automation technology and information systems) to understand the
possible error forms. Can we develop ways to use the technology to provide
effective explanations of the basis for a machine-suggested diagnosis? My research
and system development program with the process control industry is to provide
designers with such tools. My mission today was not to go through these techniques.
We do not by any means have all the answers, but we do have some. And I think
together we can develop more.

Comment: Some of the major points I learned from this presentation are:

Our pilot training needs to be changed a little bit. We have to spend enough time in
teaching the pilot what the designer expected the machine do when he designed it.
This may sound like a cliche except that when we trained the pilot, we said, "push
this button to go up, push this button to go down," but we have not spent a lot of
time teaching how to do this manually, efficiently and safely. We should convince

the pilot that the designer of the flight management computer intended the machine
to fly the aircraft exactly like the pilot would. If the pilot understood this, then
when the machine is about to make a gross error, the pilot would recognize the
gross error and be sensitive to the fact that the gross error is about to occur and will
be able to correct it.

Another place where our training has to be changed is the machine has to indicate
the priority for fault management to the human. In the power plant example, the
computer system could have worked if things had been prioritized. In some aviation
cases, they are, but in other cases, a laundry list does not help you at all. Now the
EICAS system in the 767, for example, is good in some ways and not quite perfect
in other ways. A list of yellow messages is fine as long as there are not more than
about 4. As soon as there are more than 4, it is difficult to sort them out. To the
designer, the answer may be simple — ^prioritize by indenting the less important
ones. The problem is that for the human being, indentation means a subset of the
primary set. So that does not work too well either. In fact, we are very vulnerable
to EICAS misreading and there needs to be a better method of prioritizing the
display.

The last and most critical thing is we have to teach the pilot that it is not a question
of either all automatic or all manual. We have to teach him when to interrupt the
machine's action in a supervisory manner. It should not be an all or nothing
approach but somewhere in between. In other words, the crew may need to
eliminate the FMS and use the remote control panel, but they may not need to
disconnect everything. That's the direction our teaching must take if we are to help
the human interact with the machine.

Comment: Your examples in the banking industry indicated where they used to
have problems across die board in the automation area. But these situations have
been rectified. It seems to me that the technology has now reached the point where
we've got to take the blinders off of the engineers and designers so that they
understand that the technology does not stop at the front of the display. The
problem is how to display the information. The technology problem goes right to
the actual application and that's where all the problems are. Designers were only
concemed with how to generate information contained in the computer and how to
get it to the controller's CRT. The designers then incorrectly assume that their
problem is finished. The controller/pilot must not be left to figure out how to use
this information.

Question: Sometimes it's difficult to get a list of examples of what not to do. Do you
have a list, even a small list?

Response: Figure 14 is an attempt at this list. A variety of cases would be required
to develop a more thorough list, but it should now be possible to put such a list
together.

Question: Dave, it seems to me that problems occur in trying to design a totally
automatic, safe system. Conversely, success happens when the automation was
designed as a tool for the operator. We should stick with this philosophy in the
cockpit. It should be designed as a tool for the pilot. Up to this point, I've looked at
the FMS as another navigation tool, an extension of what we had in the past. The
new systems should not take the place of the pilot and that is the important issue.

Response: I think you're right. The problem researchers and developers like niyself
have is determining what it means to design the automatic system as a tool for an
operator. And I think the down side is that lip service is given for that philosophy.
Sometimes it seems as if we're moving along in the right direction when m point of
fact, we may be completely undermining the operator's role in the system who may
soon be only running alongside the automatics.

Question: Dave, everybody wants guidelines. Do you believe that there will come a
time when you can sit down and make up a list of general guidelines?

Response: I think the answer is partially yes, but it would look very different than
the traditional human factors guidelines. The problem is that the typical guidelines
usuaUy do not relate to the real design process or the work constraints which must
be considered. Such guidelines are therefore irrelevant.

Question: The examples you gave here, particularly for banking, illustrated how
the automation can lead people astray which at least gets people thinking about it.

Response: That is where guidelines wiU be useful. They may not be guidelines in the
traditional sense, but they are pointers into relevant pieces of the literature. As
examples, they function as reminders for designers. For example, something as
simple as a computerized meter, a meter on a computer screen, doesn't have to look
like an analog instrument. It can have all kinds of interesting features, most of
which you rarely see. The designer could use this part of the database to get a long
reminder list of issues relevant to his appUcation.

REFERENCES

Adler, P. New technologies, new skills. California Management Review,
29:9-28, 1986.

Bereiter, S. and Miller, S. Investigating downtime and troubleshooting in
computer-controlled production systems. In Fourth Symposium on
Empirical Foundations of Information and Software Sciences, Atlanta,
GA, 1986.

Bereiter, S. and Miller, S. Sources of difficulty in troubleshooting automated
manufacturing systems. In Ergonomics of Hybrid Automated Systems,
Elsevier Science, Amsterdam, 1988.

Corbett, J. M. Prospective work design of a human-centered CNC lathe.
Behavior and Information Technology, 4:201-214, 1985.

Elm, W. C. and Woods, D. D. Getting lost: A case study in interface design. In
Proceedings of the Human Factors Society, 29th Annual Meeting, 1985.

Hoogovens Report. Human factors evaluation: Hoogovens No. 2 hot strip
mill. Technical Report FR251, London: British Steel
Corporation/Hoogovens, 1976.

Kidd, P. T. The social shaping of technology: The case of a CNC lathe.
Behavior and Information Technology, 7:193-204, 1988.

Kragt, H. and Bonten, J. Evaluation of a conventional process-alarm system
in a fertilizer plant. IEEE Transactions on Systems, Man, and
Cybernetics, SMC-13:586-600, 1983.

Lees, F. P. Process computer alarm and disturbance analysis: Review of the
state of the art. Computers and Chemical Engineering, 7:669-694,1983.

Miller, S. and Bereiter, S. Impacts of automation on process control decision
making. Robotics and Computer-Integrated Manufacturing, in press.

Moray, N. Modelling cognitive activities: Human limitations in relation to
computer aids. In E. Hollnagel, G. Mancini, and D. D. Woods, editors.
Intelligent Decision Support in Process Environments, Springer-Verlag,
New York, 1986.

Noble, D. F. Forces of Production: A Social History of Industrial Automation.
Alfred A. Knopf, New York, 1984.

Pope, R. H. Power station control room and desk design: Alarm system and
experience in the use of CRT displays. In International Symposium on
Nuclear Power Plant Control and Instrumentation, Cannes, France,
1978.

Rasmussen, J. On the Structure of Knowledge: A Morphology of Mental
Models in a Man-Machine System Context. Technical Report M-2192,
Riso National Laboratory, 1979.

Roth, E. M., Bennett, K. B., and Woods, D. D. Human interaction with an
"intelligent" machine. International Journal of Man-Machine Studies,
27:479-525, 1987.

Woods, D. D. Coping with complexity: the psychology of human behavior in
complex systems. In L. P. Goodstein, H. B. Andersen, and S. E. Olsen,
editors. Mental Models, Tasks and Errors, Taylor & Francis, London,
1988.

TTIFTHANSA COCK PIT SYSTEMS SURVEY; A-310

Captain Peter H. Heldt
Chief Technical Pilot— Lufthansa

INTRODUCTION

Lufthansa is using cockpit surveys to obtain up-to-date information and feedback
from their flight crews as a basis for cockpit specifications.

1976-Survev

In 1976 we conducted a survey of our existing fleet of: B-707, B-727-200, B-737-
100, B-747-200, DClO-30 and A300-B2. At that time, development of modem
avionic equipment indicated an imminent change in cockpit design. Questions
regarding whether and how to continue with further automation on the flight deck
were of special interest. The results of that survey were published shortly after
introduction of the A3 10-200 and the B737-200 Adv. Both aircraft have Flight
Management Systems (FMSs), a new generation of auto flight systems, and are 2-
crew airplanes.

This paper includes data only for the A3 10-200.

Our A3 10-200 fleet consisted of thirteen A3 10-200 airplanes at the time of the
survey "snap shot." The A3 10-300 and A300-600 were not yet delivered. Vertical
navigation was only in an introductory stage with so-called OP+-I- status. Thus
questions regarding VNAV could not be included.

The Questionnaire

The questionnaire was anonymous. The only personal data requested were:

Function (CMl or CM2)

Types of airplanes flown previously

Hours logged on A3 10.

All questions were to be answered with standardized ratings on a five-point scale
with a neutral center position. Free text comments were encouraged. The entire
volume of documented written comments represent a valuable source of
information for system designers. The questionnaire consisted of two main parts.

?rtu-.V'

l^ttiX-J"™™""

Part 1: Cockpit lay-out, general handling qualities and airplane sys-
tems

Part 2: Electronic crew interfaces: EC AM, EFIS, AFS, FMS

Part 2 was subdivided into the following four main topic areas ac-
cording to a standard man-machine interface model:

I. Physical Interface (reach and see) — control location, reach and
handling, display location, readability, color and lighting, etc.

II. Interface Dialogue or Operational Considerations
(understanding) — Ease of understanding of operational rules, display
rules, interlocks, and amount and kind of required training,

III. Interface Tools (usability) — General usefulness, adequateness
and importance of features.

IV. Organizational Aspects of the interface (appropriateness in the
operational environment) — Factors like reliability, logistics, ATC-
constraints, etc.

Some Statistics

Total number of questions per questionnaire = 654
Number of questionnaires distributed = 202
Number of questionnaires returned =121 (60%)

ECAM (ELECTRONIC CENTRALIZED AIRCRAFT MONITORING)

General Observations

Pilots with more than 500 hrs on the A3 10 gave more positive ratings than those
with less than 500 hrs.

Phvsical Interface (Reac h and See^

Generally, the physical interface was judged positively. The new visual display
units are welcomed. Only a few aspects were criticized:

rnntrast and Brightness

Contrast and brightness of the displays were judged as inadequate under daylight
conditions. CRT surfaces often are contaminated by dust and fingerprints. Display
wiping and cleaning is a common preflight procedure.

Tntp.rface Dialogue Understanding the ECAM)

Automatic sequencing, display rales, and readability all received positive ratings.
However, the aural wamings were mentioned as being too loud. All pilots attest to
their effectiveness as "attention catchers," but the comments of the pilots differ
regarding the issue of direct failure recognition via specific tones. Sixty-four
percent of the pilots state that the present number of different aural wamings is
"just right." (The A3 10 has 7 aurals plus voice waming. The A300-B2/B4 had 13
aurals + voice, which received a clear "overdone" in our 1976 survey).

Learning to operate and understand the ECAM does not seem to represent a
problem during normal operation.

TnteH^ace Tools/S upport of the ECAM

The principle of computer-aided guidance during normal and abnormal operations
is generally welcomed. The general ECAM performance to cope with system faults
is judged to be of value by 87% of the pilots. The system displays are judged to be
important or very important by 97%, while only 21% of the pilots find the
dedicated waming light display panel (WLDP) of any value.

This favorable overall rating however, also includes some comments regarding
ECAM's weaker elements: Checklists offered by ECAM received mixed ratings.
The comments mainly center on the idea that the checklist design sticks too much to
the 3-crew basic A300-B2/B4, i.e., the procedures involved to operate the aircraft
systems are basically unchanged. Pilot actions are sensed by push-buttons, and this
calls for some additional activity ( "monkey switching").

Because of limited redundancy and computer capacity the ECAM system requires
the pilot to change from screen to paper checklists and vice versa. This is perceived
as being confusing. 99% of the pilots state that thorough training is required to
handle this aspect.

Generally, pilots felt that proficiency in dealing with abnormal siUiations cannot be
achieved during line operarion. Therefore, they uniformly called for more
simulator or refresher training.

Or ganizational Interface (Fit into the Environment)

The survey results showed that reliability, maintenance and logistics do not seem to
represent any problems. Ninety-Five percent of the pilots indicated that they have
never or have seldom seen an ECAM failure, nor did they have difficulties in
getting spares when needed.

Many pilots claim that the cruise phase, where most T/0-inhibits are cancelled,
begins too early (1,000 ft/GND). In dense traffic areas the crew is still busy with
departure procedures.

EFIS (ELECTRONIC FLIGHT INSTRUMENT SYSTEM)

General Observations

As with the ECAM more positive ratings were assigned to the questions by crews
with more than 500 hrs. flight time.

Phvsical Interface (Reach and See)

Again, these interface aspects were judged overall very positive. Changing personal
instrument scanning technique from electro-mechanical instruments to CRT-
displays seemed to present no problem. The following minor complaints should be
considered:

• Vision and cross cockpit readability of LCDs for decision height
and flight path vector is reported to be only marginal.

• The location of the navigation display (ND) which is obstructed by
the control column is not optimal.

• The quality of the artificial voice for radar height is rated as good
or very good by 94% of the respondents, but our pilots state that it
is too loud.

• As with the ECAM, the EFIS displays require frequent cleaning to
reduce any dust or reflection which is reported to be annoying
during daylight operation.

Oppratinnal Interface
(Understanding F.FTS Display Formats)

• Our pilot ratings substantiate that these new visual display units are
superior to electro-mechanical flight instruments and are self-
explanatory (intuitive) to a large extent.

• The detailed speed scale obtained an excellent rating; 94% said it is
advantageous.

• The display of preselected altitude in the primary flight director
(PFD), which is just a duplicate of the glareshield selection,
triggered a request for a "fuU blown" PFD presenting all "basic T"
data.

• The indication of radar height in the PFD consists of a plain
numerical readout, but is supported by artificial voice. Eighty-
three percent of the pilots confirm that their awareness of radar
height and rate of descent is satisfactory. This implies that aural
signalling for essential flight parameters is adequate and acceptable.

• On the other hand, presentation of drift angle is flawed: 7 1 % say it
is poor or very poor.

• Pilots did not report encountering any problems with EFIS during
training. Line and simulator training are considered most efficient
and valuable, while using the airplane operations manual (AOM) is
considered less efficient.

The. F.FTS as a Tool rSnpporr of the Pilot)
Again EFIS are generally accepted. Some details should be mentioned:

• The design of the speed scale again received high marks.

• 95% of the pilots say that their speed awareness is helped.

• The standby airspeed indicator is thought to be rarely used for
reference.

• The flight path vector (FPV) received less approval. This new
display element needs further development. Some pilots want a
FPV with flight director (F/D) commands.

Among the various ND Modes, the MAP Mode is judged to be the
most important one, followed (by a considerable distance) by the
PLAN, ROSE and ARC mode. 90% of the pilots state that the ND-
CRT-display is an eye-catcher.

AUTOFLIGHT SYSTEM

Physical Interface TRea ch and See^

The similarity of the FCU control knobs in the glareshield encourages errors.
Twenty percent of our respondents mentioned that the knobs are to
indistinguishable from one another, particularly due to their close physical
proximity, thus this design is likely to induce mistakes. A typical comment states:
"all in a row look pretty good, but it takes time to identify."

The legibility of the labels is good under daylight conditions; at night, legibility is
impaired. Lighting level of different panels is not synchronized. A lot of conmients
criticize the PRESET SPD/MACH indication and label.

Dust behind the glass window of LCDs hampers readability.

Eighty percent of respondents find it important that most FCU parameters are
repeated in the PFD and ND or are summarized as flight mode annunciation
(FMA). However, 51% report having missed a repetition of the vertical speed (V/S)
value and the SPD/Mach value in the PFD.

AFS Operational Interface (How to Use It)

The multiple functions for the SPD knob seem to be acceptable. This does not hold
for the altitude selector (30% critical reports). Whenever rotary push/pull knobs or
simple push-buttons are intended to perform the same function, push-buttons are
preferred.

There are a variety of comments on the preset/SPD/Mach function and its
readability:

• The preset function is required but actually operating the button
was judged to be too difficult.

• The rotary knob for the V/S is strongly criticized by 47%
sionally the knob is turned in the wrong direction.

Occa-

• 42% have experienced inadvertent changes of dialed selections
when operating the selector knobs. Some comments mention that
this is the reason why push-buttons are preferred for "engage"
functions.

• The information of the FMA in the PFD regarding color, quantity
and arrangement seems to be highly appreciated.

• 74% of the crews use 100% of all AFS functions. Every second
copilot indicates a desire for additional functions. Captains seem
just happy with the modes provided.

• There is a common request for a "*" symbol for speed and altitude
in profile mode in order to see what the A/P is going to do next.
Crews report that throttle movement often is the only cue they have
to know that a mode change has occurred.

• Line training (learning by doing) is regarded as the best means of
instruction for the AFS, according to 83% of respondents. The
simulator is accepted by 1/3. The AOM is used by only 19%,
whereas 49% consider manuals a poor training method, and 30%
report never using the book.

AFS as a To n! in Dailv Operation
mow Well Can a Task be Performed)

The number of modes for lateral, vertical navigation and thrust control are
considered adequate by 83% of the respondents.

Use of the AFS

T/0: 75% use AFS (thrust only) plus FMS in NAV + profile (PROF).
11% report performing the T/0 manually with FD.

Climb: 91% report using all features, only a small group reports
flying manually.

Cruise: AFS is uniformly reported to be used almost 100% of the time.

Descent: No uniformity here, although the largest group uses AFS +
FMS in NAV. Profile descent speed is reported to often be too slow for
ATC requirements.

A pproach: 55% report flying the approach manually with ILS. Non-
precision approach is flown manually (39%). Others report using
AFS+FMS in NAV. Visual approaches are flown manually without FD
(78%).

Landing: Landings are performed manually by 95%; control wheel
steering (CWS) is reported to be almost never used.

The guidance quality of the AFS in lateral modes is judged to be accurate, but in
LAND mode the ILS capture of the system is criticized. Comments report
overshoots when capturing. Also, during altitude capture, oscillations are reported
to occur. Throttle-movement is judged as too active during certain flight phases.

Comments also note that synchronization of both engines by die auto throttle system
should perform better.

The AFS is sometimes switched off for passenger comfort reasons.

Or ganizational Interface: (Pit into the Environment')

Eighty-five percent of respondents feel that ATC requirements can be met with the
present system. Seventy percent indicated they rarely encountered missing
functions due to lack of spares or maintenance problems.

• CAT in is reported to almost never be impaired by AFS malfunctions.

FLIGHT MANAGEMENT SYSTEM (FMS)

Phvsical Interface: (Reach and See)

• Cross cockpit reach of the opposite CDU is reported as an incon-
venience.

• Readability of the keyboard is judged to be good under daylight
conditions but poor during night flight (53%), due to dimming
performance.

• Size of the keyboard is criticized by 1/6 of the pilots.

• The arrangement of the keys is accepted by the pilot group with
more than 500 hrs. False inputs reportedly do occur however (keys
are too close to each other).

• The parallax of the "line select" keys fosters inadvertent inputs.

• 22% judge the FMS CRT as too small and too overloaded to locate
the desired information quickly (graveyard of numbers).

• 65% ask for a color display.

Opp.ratinnal Interface: HVorking w ith the FMS)

• The FMS menu structure and the scratch pad operation is much
liked.

• Rigid input format rules frequently produce "incorrect data" mes-
sages.

• As is the case with the AFS, learning by doing seems to be an ap-
pUcable principle for the FMS.

FMS as a Tool

• Pilots report that in some situations it is difficult to find specific
data quickly, (although these situations are relatively rare). Typical
cases are: return to departtire, route change, or waypoint change.

• The response time or computational performance of the FMS is
judged to be far too slow.

• Lateral Nav is judged outstanding while the available vertical NAV
is just average (note: no profile (PROF) existed in 1986).

• Due to overall satisfaction with the FMS, aircraft control is gen-
erally delegated to the FMS or FCU. During approach and landing
phase, the FCU is preferred in order to stay head up.

• Inputting navaids which are not in the current database and entering
waypoints via the scratch-pad is judged to be workable without
undue hardship.

• Bearing/distance to a waypoint is judged highly important.

• SEC FPL (secondary flight plan) is judged important by 78% of
respondents.

• Misaligned maps occasionally occur and are annoying when they do
happen.

The following additional items are strongly desired features (should be
given high priority):

display of minimum safe altitudes,
engine out departure procedures,
airport layout.

CONCLUSIONS

Overall, our pilots like automation. However, flying with the automatics must be as
good or better than flying manually. Some problems do occur with automation.
"Keeping the pilot in the loop" is a mandatory requirement for any automated
function. P^S and ECAM are both well liked. However, both systems are not yet
optimally designed. Initial development of the FMS was promoted and tested by a
relatively small group of pilots. Further development should be based on a broad
(international) range of airline experience. Advanced flight management systems
must incorporate an improved crew interface, higher computational performance,
and a better fit to the ATC-environment.

WORKING GROUP
REPORTS

Flight Deck Automation: Promises and Realities

Final Report of the Working Group on

AUTOMATION AND AIR-GROUND COMMUNICATION

Compiled by Renate Roske-Hofstrand

Chair: Jack Wisely, TWA

Vice Chair: Renate Roske-Hofstrand, NASA-Ames Research Center

Members: Steven Alvania, FAA

James Danaher, NTSB

Alden Lemer, FAA

George Steinmetz, NASA-Langley Research Center

INTRODUCTION

This report is based on discussions held at the NASA-Lidustry workshop entitled,
Cockpit Automation: Promises and Realities, held in August 1988 at the Carmel
Valley Inn, California. Several themes emerged during the working group
meetings. Among the main themes are:

1) Because controllers and pilots cooperate to achieve safe, orderly,
and expeditious traffic flows through the National Airspace
System (NAS), designers of automated systems must consider the
tasks and goals of both the pilot and the controller.

2) The primary domain for air-ground communication involves
navigation on the pilot's side and traffic flow management on the
controller's side. Flexible, well designed communication
interfaces must be developed where sharing of information
between controller and pilot is easily accomplished without
increasing workload levels.

3) Cockpit automation development must evolve so as to achieve
compatibility with the NAS efficiency goals which relate to
increasing traffic density in available airspace. Cockpit situation
management must be matched to the traffic flow management
concems of the controller.

It should also be noted that the air/ground working group was tasked with
discussing some of the current issues in air/ground communication and therefore a
conscious effort was made to avoid any detailed discussion of topics explicitly

101

related to future planned datalink technology during this workshop. The
overwhehning feeling of the woiicing group members, however, was that it is in the
context of these future systems where new approaches to design are called for and
can be of most benefit A follow-up workshop on these issues was suggested. In fact,
it is anticipated that the interdependences noted below between pilot and controller
tasks will be even more closely coupled in the future and will likely include other
ground-based elements such as airline dispatch interfaces and/or airline
maintenance operations.

TOWARDS COORDINATED DEVELOPMENT AND DESIGN

The goals of automation in both the cockpit and control room environment have
traditionally been defined in isolation from one another. Because these
environments both function within the NAS, there exist strong interdependencies
which need to be taken into account in the definition and design of automation tools
for both pilots and controllers.

The interface between pilot and controller primarily supports communication
activities regarding an aircraft's safe progress through airspace populated with
many other aircraft. A high degree of cooperation between pilots and controllers
and adherence to commonly understood (standard) procedures is the foundation for
current safe operations.

It is important, however, to realize that the specific task goals of the pilot and the
controller differ in the following way: Pilots are in command of their aircraft only.
They are concerned with navigating through traffic and weather in their immediate
vicinity. Therefore, pilots can be said to be single-aircraft centered in their tasks
and goals.

Air traffic controllers, on the other hand, support the pilot's task of safe navigation
but share their attentional resources among all aircraft in a given sector for which
they carry responsibility. Controllers are increasingly concerned with system-wide
efficiency as it relates to traffic throughput and total number of aircraft served.
Controllers can be said to be multiple-aircraft centered or "distributed" in their
tasks and goals.

How does one then appropriately shape and coordinate the development of
automation tools in this interdependent context? Cockpit automation development
must be able to specify what the consequences of a particular interface design are
with respect to both the pilot's tasks and the controller's tasks within the operational
backdrop of the overall system.

102

SYSTEM-WIDE CONSEQUENCES OF COCKPIT AUTOMATION

The advent of automation in the cockpit (i.e., Flight Management Computers,
global navigation systems, sophisticated autopilots, etc.) assists pilots in complying
with standardized procedures and controller requests. With automated on-board
navigation equipment complex navigation and route structures can be adhered to
more precisely (as long as no unforeseen changes occur). Conceptually, at least, the
advent of cockpit automation should be beneficial to controllers as well as to pilots.

Automated navigation usually results in more precision and predictability of an
aircraft's position in the available airspace since maneuvers are executed
automatically and do not incur the common costs of pilot response or deviation
from prescribed routes or altitudes. Additionally, complex route structures with
many constraints (minimum or maximum altitudes or even time limits) present no
additional burden on the pilot, because they are executed by the on-board
computers.

If aircraft, because of their cockpit avionics, can adhere more reliably to complex
navigation structures, then controllers could have more flexibility in using such
structures and issuing clearances accordingly. Furthermore, controllers, in
addition to having a larger repertoire of possible altemative routes available, can
expect more consistent adherence to their instructions. A controller's expectation
about the possible range of deviations from the issued clearances is positively
affected, i.e., possible flight path deviations are minimized since navigation is
automated. This reduces the need for the controller to re-check and continuously
monitor an individual aircraft's progress since he/she can rely more strongly on the
execution of a particular flight path over time when dealing with an "automated"
aircraft.

Cockpit design at present occurs in isolation from the larger system context. For
example, the fact that controllers now have no explicit information available to
them regarding the type of automation equipment available on a particular aircraft
precludes their ability to make strategic use of this information. In other words, the
benefits of cockpit automation cannot be fiilly realized for the system as a whole. A
question that needs to be considered in this context is the following: Should a
controller's strategy for dealing with traffic consider the equipment available on
board the aircraft? Undoubtedly the answer for the future must be affirmative, i.e.,
the pattern of control and cooperation between pilot and controller must evolve
from common, basic assumptions about an aircraft's capabilities.

103

TRADEOFFS: PILOT VERSUS CONTROLLER CONCERNS

Flexibility in route assignments is necessary to move aircraft safely and efficiently
through die available airspace. A heightened emphasis on traffic "throughput"
makes strategic flexibility in traffic control a necessity. Pilots currently perceive
increased effects of the NAS demands primarily as increased woricload in the flight
phases occurring in terminal areas. To a large degree this occurs because of poor
interface design. Pilots in high density traffic areas must be able to respond quickly
to changes in ATC clearances. Older technology aircraft fare better under these
circumstances, presumably because the pilots themselves are processing and
integrating rapid changes in ATC clearances, and they do not need to engage in the
extensive re-programming efforts now required to keep the on-board systems of
the "advanced cockpits" informed of these changes. Thus pilots of non-automated
aircraft do not experience a noticeable increase in the level of workload as a result
of ATC instructions.

This paradox is also in part a direct result of control policies that deny controllers
the use of different control strategies for automated aircraft. The mixed traffic is
presently dealt with as if there were no performance differences among the
differently equipped aircraft. While cockpit automation should allow for more
flexible controller strategies in order to meet the ever increasing capacity demands
in addition to reducing the pilots workload during critical flight phases, just the
opposite occurs. Pilots of "equipped" aircraft experience increased workload levels
as a result of changes in controller issued clearances.

There exists, at present, a tradeoff between overall NAS efficiency, i.e., flexibility
of control strategies, and the pilot's ability to accommodate this flexibility in the
automated cockpit. The potential benefits of cockpit automation are clearly
compromised by the seemingly sluggish and isolated (from cockpit automation
development) air traffic control system. The solution to this state of affairs rests
with a joint commitment by airframe and avionics manufacturers, air carriers and
the FAA to develop air/ground automation tools that are explicitly designed to be
compatible with each 9ther.

DIRECTIONS FOR RESEARCH IN INTERFACE DESIGN

Among the specific complaints of pilots with respect to air/ground coordination are
the lack of adequate electronic map displays. The anecdotal evidence presented at
the woiking group points at both, an inadequate database in terms of completeness,
and at the inadequate current format of the interface to the database.

104

The entire concept of an "image assisted communication system" seems to have
eluded designers of current advanced cockpit systems. When the controller issues
clearances the pilot should not have to resort to paper charts to follow the
instructions. The electronic map display should be able to allow the pilot to easily
locate unpublished temporary fixes and provide an interface with which controller-
issued clearances for route structures can be stored and followed easily.

Questions of maintenance and integrity of the navigation database need to be
addressed. The potential benefits of a common pilot/controller geographical
navigation database should be explored and evaluated. Development towards these
jointly considered automation systems should be evaluated agamst the following
guidelines:

1) All aircraft in a given airspace can operate efficiently i.e., in a
safe, orderly, and expeditious manner.

2) The controller can maintain a high degree of confidence that an
aircraft will follow issued instmctions.

3) The pilot has available suitable information displays to comply
with controller-issued instmctions.

4) All humans in the system are comfortable with resulting work-
load levels.

5) Cooperation between pilots, controllers, and dispatchers is en-
couraged and enhanced via easy to use, image-assisted communi-
cation interfaces.

SUMMARY

Some major issues discussed by the working group are summarized in Figures 1
and 2. Figure 1 illustrates the coupling of computer processing and ease of using the
data. Figure 2 is a diagram of the information flow in a shared environment. The
exchange of information via voice-link, or in the future data link, is most critical.

Operational safety and efficiency goals demand full and adequate situation
awareness from both the pilot and the controller. While pilots are single-aircraft
centered in their tasks and goals, the controller's attention is distnbuted over
multiple aircraft. Hence there exist different needs which must be accommodated
by the interface, i.e., the joint concem for an aircraft's safe navigation must be
supported by features in the interface which are designed to support mutual

105

understanding and effective communication. The inherent trade-offs between pilot
and controller goals must be recognized and explicitly addressed.

Designers must specify the consequences of their design decisions in terms of the
communication interface between the aircraft and ground-based air traffic control.
Future displays for the controller need to include information regarding the
"automation status" of an aircraft and prediction functions for traffic paths which
allow him or her early conflict resolution planning. Well-designed displays for the
pilot must include easy to manipulate and easy to comprehend information
regarding area navigation and descent profiles, possibly including superimposed,
integrated weather information. Additionally, on the same display, the pilot could
be shown traffic that is directly relevant to his aircraft's flight path. The overriding
goal for the design of controller and pilot displays must be a shared frame of
reference for air/ground communication based on common geographical databases
which permit a high degree of cooperation towards meeting NAS efficiency goals.

106

Automation Development Goal:
Easy to Use Interfaces

HIGH

c
o
U

-§

LOW

"'^gggggSfc

Degree of Computer Processing

HIGH

Automation and Air-Ground Communication Working Group
Carmel, CA; August 1988

Figure 1

107

SHARED ENVIRONMENT (REAL SITUATION)
DATA
(SENSED-DIGITIZED)

[

Controller
Situation Awareness

]

Aircraft Centered

Data

Management

Activities

FORMATTING

Distributed

Data

Management

Activities

FORMATTING

DATA LINK

Automation and Air-Ground Communication Worlcing Group
Carmei, CA; August 1988

Figure 2

108

Flight Deck Automation: Promises and Realities

Final Report of the Working Group on

CREW COORDINATION

Chair: Al Ogden, United Airlines

Vice Chair: Barbara G. Kanki, NASA- Ames

Members: Renwick Curry, Tycho Systems, Inc.
Kenneth Malchow, Eastern Airlines
John Wilson, Air Line Pilots Association

INTRODUCTION

The design and implementation of increasingly automated systems on the flight
deck raises a variety of potential human factors issues relatmg to the work that
individual crewmembers perform. In addition to these concems, however, there
are issues which affect the crew as a whole; that is, the way in which crewmembers
coordinate their activities together. The most obvious, direct effects include
changes in task structure, changes in the interpersonal aspects of traditional and
standard procedures, and changes in information flow and communication chan-
nels.

There are also indirect effects (i.e., effects which are less specifically tied to flying
the aircraft). These include changes in the organizational structure of the crew
which can potentially create shifts in authority and the redistribution ot
responsibilities. Whether company policies and training programs mirror such
changes is a further consideration. Finally, there are indirect effects which are
related to the problems of transition from one technology to another, such as the
fact that proficiency must be maintained in manual backup systems in addition to
partially- and fully-operating automated systems.

The direct and indirect effects of automation listed above strucmred the working
group discussion of major issues, but once the issues were brought to bear on
specific operations, an immediate decision was made to discuss normal and
irregular operations separately. Because problems and the strategies used to cope
with each are quite different, we.also generated separate recommendations.

109

NORMAL OPERATIONS

Effects of Automation on Task Structure

In considering task changes from non-automated to automated systems (types of
tasks, distribution of workload, prioritization in normal operations), we first broke
tasks down into Pilot-Rying (PF) vs. Pilot-Not-Flying (PNF) roles. We did not feel
that the Captain (CA) and First Officer (FO) roles and responsibilities had been
altered, but that the PF/PNF task structure had changed considerably.

In general, in the automated cockpit, the PF (regardless of whether this is the
Captain or First Officer) has less direct control of the flight path, though potentially
more precise control, and must assume a greater managerial function. The PNF
who previously provided PF backup by waiting much of the time, and talking to air
traffic control (ATC), has become a more integral part of the PF's flight control
duties. In more automated systems, the PNF must provide a type of backup which
requires greater active participation in flight control and less systems monitoring.
In particular, flight path control is filtered through a communication chain that
involves both verbal and digital control display unit (CDU) inputs. For example,
one carrier's procedure for a simple altitude change requires both PF and PNF
participation; PF to command a CDU entry, and PNF to change the altitude setting
input into the flight management system (FMS). The airplane climbs or descends
appropriately or inappropriately and both pilots must observe carefully in order to
avoid gross errors of reception or data entry.

Systems operations: Largely a non-problematic area, the changes in
systems operations include a shift to more passive monitoring (normal
operations only); hence a decrease in workload related to monitoring
and control of subsystems. Although error messages via electronic
crew alerting devices such as the engine indicating and crew alerting
system (EICAS in the B757/B767) may occasionally be an annoyance,
these are not major problems in normal operations.

Primary flight control: A great number of changes are associated with
primary flight control, and these issues will be discussed separately
from navigational issues. Many flight path functions, such as
horizontal path control, are non-problematic, particularly in low
workload phases such as cruise. Vertical path control, however, is
affected both positively and negatively. Unfortunately, the times of
greatest benefit (airport traffic area and other high workload phases)
are also the times when some of the negative effects emerge. For
instance, ATC can issue a directive that makes vertical path adjustment

110

necessary. Although FMS operations, in general, create less of a
demand on mental arithmetic, vertical path adjustment can be more
difficult simply because of the laborious CDU data entry required of a
time-pressured PNF. In addition, feedback regarding these
adjustments can be frustrating because control is less direct than in
manual systems. A computerized system is more unpredictable simply
because response time may be slow or variable.

Thus, in spite of the fact that the FMS can provide greater vertical path precision
(previous systems did not provide vertical navigational capability), more advance
planning is necessary and greater PF/PNF coordination is required due to the
possibilities of unanticipated changes. In addition, the vertical path display can have
a mesmerizing effect on both pilots distracting them from other flight duties and
forcing their eyes inside. Frequently this occurs at a time when extra external
vigilance is required during climbs and descents. This may increase the risk of mid-
air collision. Whether by command, standard operating procedures, or simply
planning ahead, the PF must therefore assume a greater managerial role in order to
ensure smooth PF/PNF teamwork.

Less direct control of flight path operations also creates a need for a different type
of cross-checking and monitoring. Error analysis can no longer rely primarily on
immediate feedback via motor skills; rather, as in the case of cross-checking CDU
data entry, and "reading" the mode control display, checking and monitonng
procedures increase cognitive workload.

Navigation systems. Similar to flight path control, navigation systems
operation has, in general, benefited from increased automation.
Enroute navigadon radio tuning has largely become a flight
preparation data entry activity, reUeving both pilots of some degree of
inflight workload. Specifically, the entire route is entered into the
flight management computer on the ground, and the computer
automatically tunes each radio enroute, permitting precise lateral
flight path control. No further pilot action is required. However, the
benefits degrade when flight plan changes are required, and these cases
will be considered in the Irregular Operations section to follow. We
are again mindful of the irony, that the greatest benefit of automated
navigational parameters occur in the terminal area and this is very
often the area in which flight plan changes are requested by ATC*
Finally, it should be noted once again that monitoring tasks are

* Negative effects involving ATC are not intended to focus on flight deck automation problems
exclusively. The fact that ATC procedures arc not compatible witii the newer automated systems on
the flight deck is an important but separate issue (see Working Group report: Air-Ground Commu-
nications).

Ill

affected. Because much of the flight planning can be accomplished
prior to flight, there may be a decrease in situation (geographic)
awareness, if monitoring is not suitably adapted to this change in task
location.

Checklists. More highly automated systems operations in conjunction
with EICAS-type messages have allowed many of the routine checklist
procedures to become more efficient (e.g., dark cockpit design where
"no lights" means "no problem"). Two important benefits were
discussed: (1) Checklists may be shorter; therefore, each item takes on
increased significance and (2) it is possible to shift the items found on
the "moving" checklist (checklist performed while the aircraft is in
motion) to a "stationary" checklist (performed while standing still)
which is a definite safety benefit. In support of the sterile cockpit
concept, the elimination of unnecessary communication during taxi
prior to take-off may be an important deterrent to runway incursions.

Flight deck communication: Automated systems have changed the
typical communication patterns within the cockpit in a variety of ways.
First, as mentioned above with regard to checklists, electronic crew
alerting devices such as EICAS have benefited the sterile cockpit
concept because communication during this critical time has been
minimized and therefore made more meaningful. Another clear
benefit to crew coordination is that increased communication between
pilots is required for error-free flight path control. Because both
pilots participate in operating the FMS, greater communication, hence
greater awareness of tihe planned flight path results.

A more subtle example of how automation affects communication relates to the
availability of information for both pilots. There is no question that the new
automated graphic displays greatly increase flight deck information resources. It is
also generally the case that these displays are equally available to both pilots. Given
this situation, there is theoretically less need to share information by means of face-
to-face communication. On the other hand, die availability of information does not
automatically imply that both pilots are always attending to the same data, although
it is tempting to ASSUME that they are. TTius, communication may seem to be
redundant at times. However, there are also times when a false assumption would
not be tolerable. For example, when mode changes are made, the mode control
panel (MCP) is equally visible to both pilots. However, because we would not want
to falsely assume that all changes were noticed, mode changes should be announced
in spite of the fact that this might be viewed as redundant communication.
Verification of mode changes might be alternatively solved by the addition of an
annunciator signal on the attitude direction indicator (ADI) itself.

112

In short, automated systems do eliminate the need for some types of face-to-face
communication and this can be beneficial. However, there are other times when
communication may seem to be redundant but it would be incorrect and unsafe to
make that assumption. Certainly in some situations, the possibility of error would
be unacceptable and the formalized sharing of information may be warranted, as
well as increased cross-checking procedures. For example, setting the mode control
panel inputs to permit descent away from the altitude window setting is particularly
dangerous since it can result in controlled flight into terrain. Operation in split
mode during descent presents a situation where autopilot altitude capture engaged
with autothrottles disengaged could result in a stall.

Summary of Task Structure Changes

Decreased mental arithmetic

More cognitive, less motor skill

Less active systems monitoring

Increased cross-check workload

More evenly distributed workload PF/PNF

More flight path control coordination

(formalized crew coordination)

PF more managerial function

PNF has increased CDU flight control participation but less

systems monitoring

Tiiltural Changes

Again, we wish to reiterate that Captain (CA) and First Officer (FO)
responsibilities have not changed; that is, the Captain still holds the final authority
and responsibility.

However, the FO as PNF is now in control of more information than formerly and
the CA must modify his/her team management to accommodate to this change.
There are two major areas which should be addressed.

Redistribution of responsibility for traditional tasks: First, insofar as
the tasks for PF and PNF are redistributed more evenly, task allocation
should reflect these changes. More important, both Captain and First
Officer must be equally proficient in handling the increased PNF
responsibilities. In particular, both pilots must be well-practiced in all
areas of automated systems operations; from handling the entry and
operations of ACARS data to CDU entry in making flight plan
changes, navigational changes and vertical path modifications.

113

Direction of information flow: In less automated aircraft, crew
members were frequently able to accomplish their tasks
independently. Given the greater coordination necessary to operate the
FMS, however, lines of communication are created which represent a
different flow of control. (Note again that this refers to a change in the
transfer of information, NOT a change in authority structure.) For
example, when CA is the PF, a simplified conceptualization of the flow
of information follows [CA "*- FO -•' machine], where flight path
control is being accomplished through the FMS and affected by CDU
data entry. However, when the CA is PNF, the reverse is true: [FO -*
CA "*• machine]. Since this sequence must flow in both directions, this
reverses the traditional system in which the unilateral direction [CA '■*-
FO], or simply CA and FO working independently was more common.
It is important that manuals and procedures reflect these changes and
that the PF/PNF terminology is supported in conjunction with the
CA/FO role distinctions.

Summary of Cultural Changes

• More even division of responsibility between PF/PNF

• "Role reversal" in terms of information flow

• Increased responsibility of PNF

Recommendations for Normal Operations

Training to address workload distribution

Formalized crew coordination

Formalized FMC checking process

Design changes to minimize entry errors

Procedures design to follow cockpit design

Use of plan-ahead procedures (take advantage of automation)

IRREGULAR OPERATIONS

It became clear to our working group, in the course of discussion, that none of the
negative effects was really very serious in normal operations. Rather, increased
automation on the flight deck has been successful in reducing workload, increasing
the amount of informational resources available, and permitting greater precision
in several technical performance areas.

114

Areas of concern emerged only when normal operations were interrupted or no
longer possible. Since some of these times occur fairly frequently and are not
"abnormal" in terms of malfunction, we defined "irregular operations" as
unanticipated deviation from intended operations with respect to a range of possible
events. At the least extreme end of the range, irregular operations could be
instigated by internal events such as minor system malfunctions or external events
such as unusual ATC requests or new weather conditions. At the high end of the
range, irregular operations would be brought on by hard failures which prescnbe
the use of formalized, written procedures (e.g., loss of pressurization or AC
power).

Minimum equipment lists: The purpose of minimum equipment lists
(MELs) has not changed in more highly automated aircraft. However,
the criteria for designing MELs now need to consider the implications
of degraded automated capabilities. When irregular operations require
a decreasing shift from a fully-operating automated system, there is an
accompanying shift in workload and task structure which must be
relieved by the appropriate level of resources designated by the MELs.

Task structure: At the onset of "irregular operations," the degree to
which tasks must be reassigned will depend on the severity of the
problem and the degree to which die automated systems will need to be
shut down. In almost all cases, however, the PNF will change from
being a passive systems monitor to an active systems controller. Other
task reassignments would need to be based on the particular
"irregularity" although there should be pre-determined rules
(supported by company policy and training) that provide guidelines
for switching from a fully operating automated system to a partial
system or a total reversion to manual control. It was felt in general,
that (1) there should be no arbitrary reversion and (2) the highest level
of automation should be maintained, where possible, consistent with
safety and tasks required. For example, because vertical path
constraints and manual approach building are not in the database, these
operations in a completely automated system result in an increase in
the number of tasks and workload and such an increase might not be
tolerable during "irregular operations." These kinds of considerations
must be weighed in selecting the level of automation that can be
realistically and safely maintained.

Pre-established priorities: Priorities for the task reassignments
required by irregular operations also need to be pre-established and
supported by company policy and training. Embodying the notion of

115

"A" tasks and "B" tasks where tasks are unambiguously prioritized, the
switch from normal to irregular operations should be associated with a
comparable task priority list as well. As an example: PF will fly the
aircraft and handle ATC communication while PNF handles secondary
communication and becomes active systems controller. As systems
controller, PNF will either initiate a procedural worklist (irregular
procedure checklist) or begin a situation assessment of the system. The
PNF must be able to invoke rules for interpreting the system of alerts
and cautions during critical phases of flight. Whether "full mode" or
"split mode" switching is used will create a need for different rules.

Recommendations in Irregular Operations

• Minimum equipment list design

- accommodate the implications of degraded automated
capabilities

• Task reassignment

- establish priorities

- rules for assignment

- determine level of automation operation

SUMMARY

In sunmiary, there are four major areas in which crew coordination is affected by
increased fight deck automation. To take full advantage of the benefits of
automation, the following elements are essential in the coping strategy:

1. There must be an increased emphasis on crew concepts in both
training and operations areas.

2. There must be better pilot-to-pilot communication during ALL
phases of flight.

3. There must be a complete understanding of tasks and responsibility
for task accomplishment in the more highly automated
environment.

4. Aircrews must know when and how to transfer from automated to
semi-automatic to manual operations as the situation dictates. This
implies both systems and interface knowledge as well as altemate
courses of action available when normal operations are interrupted
or no longer available.

116

Flight Deck Automation: Promises and Realities

Final Report of the Working Group on
UNDERSTANDING AUTOMATED SYSTEMS

Chair: Rolf Braune, Boeing Commercial Airplane Company

Vice-Chair: Alfred Lee, NASA-Ames Research Center
Members: Robert Cavill, Northwest Airlines

B. S. Grieve, Britannia Airways, Ltd.

Charles Knox, NASA-Langley Research Center

David Woods, Ohio State University

INTRODUCTION

Since the introduction of the autopilot into aircraft more than a half century ago,
automation has taken over many of the tasks which were once the exclusive
domain of the human pilot. When machines control or perfonn tasks and pilots
are relegated to a monitoring or supervisory role, questions arise as to the extent
pilots need to understand these systems. Understanding a system means not only
being aware of what it is (or is not) doing, but also knowing the reason for a
system's action and anticipating what the system is going to do next. In general,
the need to understand a system is closely related to die need for intervention by
the pilot if the system fails to operate as designed or, in the opinion of the pilot, is
operating in a manner which compromises safety.

This report is intended only as a summary of the discussions conducted by this
working group. A comprehensive review of known or of potential operating
problems with automated systems is beyond the scope of this report. Problems
which exemplify more fundamental issues in training, design, or operating
procedures are provided for illustrative purposes only. Likewise, the solutions to
these problems which have already been implemented or are recommended
should not be construed as exhaustive of possible alternatives.

CURRENT OPERATIONAL PROBLEMS

For current operational aircraft, problems involving the pilot's understanding of
automation can occur in two areas: flight path management and aircraft
subsystems management. To date, automation of subsystems management does not
appear to present a problem for aircrews. However, the current state of affairs
may change as aircraft age increases and subsystem failures occur more
frequently.

117

The more immediate problems are associated with flight path management.
Examples in this category are usually associated with Right Management Systems
and related areas of computer control of the aircraft flight path. Problems of
standardization of mode control panels are repeatedly cited by aircrews. The
problem appears particularly acute in the area of status annunciation where
confusion may arise as to what is and is not under automatic control. The
problem of standardization of mode control interface design has been aggravated
by the mixing of aircraft fleets resulting from the large number of recent airline
mergers. Lack of pilot-system interface standardization increases the cost of
traming and increases the potential for pilot errors in line operations.

Distinct from these problems of standardization in design are problems which
arise from inadequacies in the pilot-system interface of an automated system.
Errors can and do result when system status annunciation is unclear or
ambiguous. For example, if a Flight Management System can engage an autopilot
independently of an autothrottle, the system should make the pilot aware that the
system is operating in this "split mode." Lack of pilot awareness due to pre-
occupation with other duties can result in changes of aircraft attitude in the
absence of coordinated throttle inputs. If the pilot, for whatever reason, is not
aware of the status of the system, intervention may well occur too late.

Annunciation of gradual or "soft" failures of autoland systems is another example
where pilot-system interface design is particularly important. With the aircraft in
close proximity to the ground and configured for landing, awareness of an
autoland system's impending loss of function becomes critical as rapid and precise
pilot intervention may be needed. As increasingly sophisticated automated systems
are introduced in areas involving the operating limits of an aircraft, addressing
the problem of soft failure annunciation will become more important.

Closely allied to the problem of status annunciation is the need for pilots to
understand the design intent of an automated system. Currently, this receives
little, if any, attention in the pilot training process. The design intent underlying
an automated system can often help the pilot understand what such a system can
and can not do during actual operations. Windshear-induced autopilot hysteresis
is an example. Sudden changes in the direction and speed of the wind can result in
autopilot-induced pitch control oscillations. These excursions can be quite large,
and if occurring close to the ground, catastrophic. Understanding system design
should cause the pilot to disconnect the autopilot immediately. However, pilots
may be reluctant to do so if they attribute capabilities to the system that it does
not actually possess, e.g., that it has the intelligence to adjust to unusual
operational conditions. Failing to understand the capabilities and limits of
automated systems are persistent problem areas in operating such systems.

118

COPING STRATEGIES

For current aircraft, coping strategies adopted to overcome operating problems
with automated systems faU into three areas: training, operatmg procedures, and
after-market design changes. As is typical with any design problem, trammg takes
on a disproportionate role. Unfortunately, pilot training on automated systems
has been recognized as being less than adequate in both areas of systems
understanding and use of the system in line operations. Incorporation ot
automated system operation into existing Line Oriented Flight Training (LUM)
is occurring, although the cost of this type of training is high. Alternative
strategies of systems training employing computer-based training systems are also
being considered.

The second means being used to cope with automation problems is to change the
procedures for using the system in line operations. For example, operating an
autopilot separately from the autothrottle is now prohibited by some earners as is
the operation of autopilots in windshear and severe turbulence. Most, if not aU, ot
these changes have resulted from an operational incident or accident. This tnal
and error coping method is inherently undesirable as it may mcur enormous costs
in lives and property. Operating limits of systems should be clearly defmed pnor
to line operation as should the potential of these systems for design-mduced
human error.

Finally after-market design changes can have Umited use in ameliorating
operating problems with automated systems. Improving status annunciation
symbology and operating interfaces have some value in this regard. Improving
Control Display Unit design to provide easier re-programming of the FMC is one
such example as is enhancing the speed with which the FMC will accept pilot
inputs However, such changes are often difficult and expensive. In some cases,
annunciation displays and associated interfaces that could enhance pilot awareness
of an automated system's status cannot be accommodated in the cockpit without a
major re-configuration.

ON THE TECHNOLOGICAL HORIZON

Obviously, the time to address design issues is during the design process.
However, aircrew training to understand and operate future automated systems
will always be necessary and should be factored into the cost of operating any
advanced system. Fundamental design and training philosophies for automated
systems need to be established for future advanced technology aircraft if
operating problems with these aircraft are to be avoided. Advances m computer
technology will almost certainly result in even more complex levels ot

119

automation than are currently available. The result will be increased demands on
limited training resources.

Issues that are on the technological horizon are varied and far reaching in their
potential impact on pilot interaction with automated systems. Examples of these
mclude the incorporation of ground-air-ground data link systems which will
make possible the automation of clearance delivery to the FMC. Automatic
warnings of altitude deviations and of descents below minimum safe altitudes,
automated altimeter settings, and many other services are possible. Increasingly
sophisticated flight control systems are also on the horizon including gamma
flight path control,* envelope protection, and relaxed static stability. Airframe
subsystem management will become more sophisticated with the introduction of
intelligent systems and the use of decision aiding technology to facihtate failure
diagnosis. Clearly, determining the extent of pilot understanding needed to
effectively control automated systems will become even more important in future
aircraft than it is today.

RECOMMENDATIONS

It is evident that the knowledge required to fly automated aircraft requires more
than simply knowing which button to push and when. However, it is important to
emphasize that the fundamental role of the pilot has not changed (but see
Addendum). This role is made expUcit in FAR 91.3: "The pilot in command of an
aircraft is directly responsible for, and is the final authority as to, the operation
of that aircraft." As part of this role the pilot has in the past, and continues to,
perform the function of systems monitoring and flight path management. The
machinery by which an aircraft is operated does not fundamentally alter this role
though it may smiplify or eliminate the need to perform certain tasks. This leads
to the working group's first recommendation: That a philosophy of flight deck
automation be adopted which assures that the pilot plays an active role in the
management of the aircraft flight path and that any information which affects that
management should be provided either in the training process or as an integral
part of flight deck design.

Secondly, the development of flight deck information management principles are
needed to support the integration of automated systems in future aircraft. Know-
ing what kind of infomiation is needed, when it should be provided and in what
form it should be presented are essential to the design process. Standardization of
functional requirements for automated systems interfaces, particularly in mode
control panel design, is also needed as are guidelines to minimize the possibility
of design-induced errors on the part of aircrews. Such standardization require-

* Editor's note: Either automatic or pilot control of aircraft inertial velocity vector.

120

ments will require a more active role on the part of regulatory agencies than
currently exists.

Thirdly, the integration of automated aircraft into the Air Traffic Control (ATC)
system and the eventual automation of that system suggest that problems of
aircraft-ATC integration will increase unless a comprehensive systems analysis
effort is undertaken. A key element in that effort should be the delineation of the
role and responsibilities of humans (pilots and controllers) in an automated air
traffic control system.

Finally, the training of aircrews of automated aircraft must emphasize the
understanding of automated systems and how these systems can and cannot be
used in hne operations. The design intent underlying an automated system should
be an important ingredient in training program development.

ADDENDUM

The definition of the pilot's role in automated systems is not without controversy.
It should be understood that others have taken the position that the role has been
altered by the tasks, i.e., it is a role that is more passive, less manual, etc. It may
be that these differing viewpoints result from focusing attention on different
levels of the pilot's task hierarchy. In any case, this issue undoubtedly deserves
more extensive consideration than is possible within the scope of this report.

121

Flight Deck Automation: Promises and Realities

Final Report of the Working Group on

TRAINING FOR ADVANCED TECHNOLOGY AIRCRAFT

Chair: Frank Tullo, Continental Airlines

Vice Chair: Harry Orlady, Aviation Safety Reporting System
Members: Earl Wiener, University of Miami
Steven Alvania, FAA-ATC
Wendell Dobbs, American Airlines
Rod Lalley, FAA-Aircraft Evaluation
Grace Pariante, San Jose State University

INTRODUCTION

Airline pilot training provides the interface between transport aircraft and the
pilots who operate them in day-to-day line operations. Training is obviously
important and, despite the use of sophisticated simulators and other advanced
training aids, it continues to be very expensive. While there is no disagreement
within the aviation community regarding the importance of effective pilot
training, there is considerable disagreement on the kind and amount of training
that is required to enable pilots to operate new and different airplanes safely and
efficiendy within the aviation system.

Today, there is a consensus among training experts, both within and without the
industry, that regulatory requirements (and those training practices that are based
solely upon them) have not kept up with advancing cockpit technology. It is not
surprising that such training is not efficient and, perhaps because of an apparent
excessive preoccupation widi automation, it has not always been sensitive to the
wide gamut of operating needs of the pilots who routinely fly these aircraft.

There is, therefore, a very clear need to review all of the factors involved in
contemporary airline pilot training. It is particularly important to review the
training from a systems perspective because of the interaction of the many factors
that are involved.

Airline pilot training, even without the complication of advanced cockpit
technology aircraft, is a very complex subject. Its complexity is exacerbated by
such factors as some very basic differences in airline operations, the widely
varying training resources of airlines that range from the established trunks to
newly-formed commuters, the varying needs of pilots with a wide range of
established skills and experience, a broad range of aircraft and aircraft systems

123
PRECBMNG PAGE BLANK NOT FILMED

and an ATC system badly in need of modernization. In addition, all of these
aircraft must be operated safely and efficiently in day-to-day line operations in a
dynamic variable operating environment. It was not possible to fully cover all
aspects of airline pilot training in the time allotted to us.

Therefore, the Working Group did not deal with those specific areas nor with a
host of traditional generic training issues such as the kind and level of fidelity
needed in cockpit procedures or limited part- task trainers, the role of "motion" in
instructional simulators with full motion capability, the optimum amount and
kind of feedback for computer-based instruction (CBI), the design of training
programs based on present regulatory restrictions, adapting training programs to
the varying needs of a diverse pilot population, problems in "differences"
training, the effectiveness of the variety of teaching methods and teaching devices
currently available, etc. Because it did not deal with these and similar items, the
Working Group does not mean to imply that they are not important.

However, in order to take advantage of the current general industry consensus
that reexamination of airline pilot training principles, practices, and regulatory
procedures is sorely needed, the Working Group concentrated on identifying
general areas it believes should be included in the current reexamination of
airline pilot training. It identified seven general areas it believes should be
included in such a reexamination. The seven general areas are listed below and
will be discussed in the following paragraphs.

1) Review of FAR 121 Appendix E (initial) and F (recurrent)
training requirements

2) Human factors training for "Decision Makers" in the industry

FAA

Manufacturers and their vendors

Airlines

Pilot associations

Airline trade organizations (e.g., ATA, RAA, and

lATA)

Intemational organizations (e.g., ICAO)

NTSB

Specialized training organizations

3) Pilot training in automation

4) The role of the manufacturer and its vendors in training

124

5) Standardization and simplification in automated aircraft

6) Workload management in the 2 person crew automated flight
deck

7) The potential role of digital flight data recorders in training

REVIEW OF FAR 121 APPENDIX E AND
APPENDIX F TRAINING REQUIREMENTS

The requirements specified in Appendix E (initial, transition, and upgrade flight
training) and Appendix F (pilot proficiency checks) apply to all airlines operating
under Part 121. They, in conjunction with the training requirements of FAR Part
121 Subparts N and O, effectively control air carrier pilot training in the United
States for all airlines other than those commuter airlines that operate under FAR
Part 135.

The Working Group believes that present regulations are not fully responsive to
the technical and operational requirements of contemporary air carrier
operations, and that a full review of FAR 121 training requirements is urgently
needed. It fully supports the training objectives of the Administrator's Task Force
on Flight Crew Performance in this area and believes this subject should continue
to receive a very high priority.

In addition, the Working Group believes that the review process should be
formalized to ensure that similar reviews are made periodically on a
predetermined schedule or can be made in response to technological advances.

HUMAN FACTORS TRAINING FOR
"DECISION MAKERS" IN THE INDUSTRY

Over the years there has been growing recognition that human factors should be
considered a "core technology" in all parts of the air transport system including
the air traffic control system, the design, manufacture, and operation of transport
aircraft, and the regulation of these basic elements. All of these affect training
requirements.

The growing recognition of the importance of human factors, like so many
aspects of this dynamic industry, has been evolutionary. Not surprisingly, its
growth has not proceeded at an equal pace among all elements. The consensus that
"human factors" should be considered a core technology on a system basis has
been only recently achieved.

125

The Working Group believes at least two things are required to take full
advantage of the human factors potential to improve the safety and efficiency of
air transport operations:

a. First, individuals responsible for decision-making at all levels
affecting design, training, operations, and regulation (and this
includes those with responsibility for the allocation of funds)
should have training (or indoctrination) in human factors. This
should be at a level that ensures awareness of the importance of
human factors, and in particular, its relevance to air transport
operations. The training (or indoctrination) for these decision
makers should be of sufficient depth to enable them to recognize
human factors needs within their area of responsibility, and to
recognize the need for additional expertise in specific areas when
such a need arises.

Organizations with such responsibilities include the FAA, aircraft
manufacturers and their vendors, the airlines, pilot associations,
the NTSB, and specialized training organizations.

b. Second, it is equally important that members of the scientific
community interested or involved in air carrier operations
receive sufficient training or indoctrination in those operations to
ensure that their recommendations and research are responsive to
real- world needs and problems.

PILOT TRAINING FOR INCREASINGLY AUTOMATED AIRCRAFT

[Note: All of the recommendations in this section apply to both FAR Part 121 and
Part 135 carriers]

Basic Airma nship Skills and Knowledge

The Working Group believes that the addition of sophisticated cockpit automation
systems has not reduced the need for or the level of basic airmanship skills and
knowledge which have traditionally been required of airline pilots. Therefore this
discussion assumes that pilots transitioning to advanced cockpit technology
aircraft already possess those skills and that knowledge. In addition, the
importance of the extension and application of diose fundamentals to the advanced
technology aircraft should be emphasized in the early phases of both ground
school and simulator training. General aircraft familiarization should always
precede detailed instruction in automatic features.

126

Monitoring of Automatic Systems

Effective monitoring of the operation of automatic systems is an increasingly
important responsibility of the flight crew. The development of methods to
increase monitoring effectiveness should be given a high priority. Cockpit
resource management (CRM) courses that emphasize the importance of
monitoring and the role and increased responsibUity of the pilot-not-flymg (PNF)
are needed. In addition, the importance of monitoring activities should get
greater emphasis in both training and checking activities.

It is particularly important that transition training includes not only the operation
of the automatic systems and their limitations, but also their "design mtent. It is
not reasonable to expect pilots to effectively monitor the operation of automatic
systems without providing them a clear understanding of how the system they are
monitoring is planning to accomphsh its specific task.

LOFT

One of the major advances in the trainmg of airlme pilots during the past decade
has been the development of line-oriented flight training (LOFT). However,
because LOFT is still a relatively new concept there have been wide vacations m
both its use and in the quality of the training provided.

Despite these difficulties, the Working Group believes there is a need for greater
use of LOFT in initial training in order to better prepare pilots for line
operations. There was a weaker consensus that in recurrent training, the primary
use of the simulator should be in a LOFT environment. It is important to
recognize that recurrent LOFT can be conducted in the more elementary visual
simulators as well as in Phase I, H, and IH simulators.

Cockpit Resource Man agement (CRM)

There is a growing consensus within the aviation community concerning a
pressing need to improve cockpit management and cockpit crew coordination.
Although a variety of CRM training programs have been developed, the possible
need for CRM programs modified or tailored for the pilots of advanced cockpit
technology aircraft has been essentially ignored.

The. Potential Use of Home Co mputers in Training

The sensitive and intelligent use of home computers to fulfill training
requirements and for voluntary self-instruction should be explored. While there

127

are obvious potentials for misuse, there is also a considerable potential for
fulfilling the needs and desires of all of the parties involved — air carriers, pilots,
and the FAA. Implementation, however, can be a particular challenge for air
carriers and the representatives of their pilots.

The Role of the Ma nufacturer and its Vendors in Training

Determination of the general training requirements needed to enable pilots to
operate new equipment safely and efficiently should be considered an integral
part of the design process. Determination of training requirements at the design
stage of any changes or updates developed subsequent to the original design are
equally important. These requirements need not be, and probably should not be
specific, e.g., at the SBO (specific behavioral objective) level, but should clearly
indicate what the designer of the system believes the pilot should know in order
to operate that system safely and efficiently.

After the initial design and the inevitable compromises and tradeoffs inherent in
the manufacturing process have been completed, it is a logical extension of this
philosophy to have the manufacturers of transport aircraft and their vendors play
a larger role in two important training areas. The first training area is in the
development of the specific training objectives required to satisfactorily operate
their products. The second training area is in the development of the training
aids, techniques, materials, etc. needed to achieve those training objectives.

STANDARDIZATION AND SIMPLIFICATION IN
2PC AUTOMATED AIRCRAFT

There is a great need for more emphasis on standardization and continued
emphasis on the simplification of all aspects of the design and operation of 2PC
(two-person crew) automated aircraft. It should be given a very high priority by
both the manufacturers and the purchasers of their aircraft. This problem has
been exacerbated by the increasing number of aircraft leasing organizations,
airline mergers, consolidations, etc. that are a part of the contemporary airline
scene. Different names for the same item, different procedures to operate
essentially the same system, and different symbology to display identical
information can create very real problems for the crews who have to cope with
them. Unfortunately, this frequently happens under less than optimum conditions.

This by no means should be construed as a restriction on the development of
improvements in transport aircraft. However, it seems very clear that a great deal
of the lack of desirable standardization in current aircraft has little to do with
improvements in the aircraft, their systems, or their cockpit symbology.

128

For the same reasons, this emphasis on standardization and simplification should
be extended to flight operations and equipment manuals, operating procedures,
and checklists. This is primarily a responsibility of the airlines and, to a
somewhat lesser extent, the regulatory authorities. It should be given a high
priority. However, it should be noted that, particularly in the case of operating
and equipment manuals, this emphasis on simplification does not imply that these
documents should not contain the often detailed information and data required to
fulfill their basic functions.

WORKLOAD MANAGEMENT IN THE 2PC
AUTOMATED FLIGHT DECK

Air carriers are urged to take a new and creative view of flight crew workload in
automated 2PC aircraft. 2PC operational procedures and checklists should be
carefully reexamined with particular attention to the workload required to
perform them. Many carriers have a strong history of 3PC operations and there
is considerable evidence that their operation of 2PC automated aircraft does not
reflect advances that have been made in cockpit technology and in our
understanding of flight crew behavior. The problem has been exacerbated by the
large number of flight crew members who have transitioned to these aircraft
from a 3PC airplane.

Properly developed LOFT scenarios can be used to illustrate heavy workload
conditions and identify problem areas. For example, a prominent problem area in
current operations, and one that present training (and/or procedures) has not
dealt with effectively in many cases, is whether or not to continue to use
automated navigation systems when ATC clearance changes are received during
the final stages of an approach. In these cases, the important decision is often
simply whether or not to continue to use automation and reprogram or to simply
turn the automation off. If flight crew workload is further increased by
inappropriate policies or procedures, the problem can be clearly identified during
the LOFT exercise.

Considerable flight crew workload can be created by the requirement to perform
non-operational, but important, tasks at particularly inopportune times. For
example, calls for passenger connections, meal requirements, wheel chairs, and
other passenger service items can be more than just a nuisance to the flight crew.
This is by no means a new problem, but is becoming more critical because of the
proliferation of high-density operations. In many cases these flight crew tasks
may be either further automated (as in the case of many ACARS functions),
eliminated or reassigned.

129

THE POTENTIAL ROLE OF DIGITAL FLIGHT
DATA RECORDERS IN TRAINING

This is an admittedly controversial item. The Working Group believes considera-
tion should be given to the system-wide use of digital flight recorders to identify
areas needing training emphasis. It can also be used to identify those that are not
creating problems. This is not a new idea. It has been used successfully by several
foreign carriers with the sanction and complete cooperation of their pilot unions.
The key provisions have been a clear recognition by all parties that the sole pur-
pose of the program has been to improve the safety of tiieir operations and that
the rigid restriction on the use of this data has been honored.

The sensitivity of the recommendation creates a formidable challenge for all of
the parties involved. The challenge is to develop procedures that permit taking
advantage of state-of-the-art technology for their mutual benefit. The abiUty of
data from digital flight recorders to improve safety and, in some cases, to justify
a reduction in training requirements and training costs requires a cooperative
intelligent utilization of that data. If this can be achieved, problem areas can be
identified early, and safety can be improved. The reduction of training
requirements and therefore training costs is an additional potential benefit.

SUMMARY

Historically, airline pilot training has developed in a relatively piecemeal and
unexamined manner. While a great many changes in the industry have been
essentially evolutionary, the cumulative magnitude of these changes has created a
pressing need to reexamine current operating procedures and training
requirements in light of automation's demands and in tiie opportunities it presents
to the aviation community.

The recognition of human factors as a core technology has been a major
breakthrough. Another has been recognition of the need to reexamine our
training needs from a total system perspective. The challenge, and it is a
challenge to both the operational and scientific community, is to take full
advantage of our new-found knowledge.

130

Flight Deck Automation: Promises and Realities

Final Report of the Working Group on

ERROR MANAGEMENT

Chair: David Nagel, Apple Computer

Vice Chair: Everett Palmer, NASA-Ames Research Center
Members: Earl Wiener, University of Miami
Steven Alvania, FAA-ATC
Wendell Dobbs, American Airlines
Rod Lalley, FAA- Aircraft Evaluation
Grace Pariante, San Jose State University

INTRODUCTION

The objective of this working group was to identify the influences, both positive
and negative, of cockpit automation on the occurrence and detection of error on
the flight deck.

A kev goal in the design of aircraft cockpits, aircraft operating procedures and
crew training is the reduction of incidents and accidents attnbuted to human
error Some have claimed that automation can elimmate human error by
removing the pilot from the control loop. Others have claimed that while some
types of error may be reduced the automatic equipment itself introduces
opportunities for new types of human error. The new equipment may elimmate
smaU errors but introduce the possibility of large errors. These new error forms
seem to be particularly difficult to anticipate during the design phase.

The group discussed: the changes in cockpit systems that have affected the type
and frequency of crew errors; examples of types of human error that have been
reduced; and examples of new types of human error.

The key output of this working group was a list of "high" priority and "medium"
priority automation issues and recommendations that relate to errors and error
detection in current and future advanced technology cockpits.

131

HIGH PRIORITY ISSUES

Industry Wide Error Data Rasp

The design of aircraft cockpits is an evolutionary process. Each new cockpit
design is an attempt to improve on the past designs. If the cockpit designers know
that pilots systematically make specific errors in operating a piece of equipment,
they can often design the new equipment so that an error is either impossible or
much less likely. To make this process work the designers must know about the
types of error that are occurring. Currently there is a large body of operational
experience which is not known to the flight deck designers. An industry wide data
base should be established to record errors and incidents that can be used for
design of future systems, upgrades to current avionics software or changes in
current training courses and procedures. The lATA has an incident data base that
might be adapted to this use.

Training for Highly Automated Aircraft

Training should be organized so that the pilot can always answer the following
questions about an automatic system: What is it doing? Why is it doing it? and
What will it do next? The pilot needs to know the system well enough to be able
to predict what it will do in different contexts. Training should contain
infonnation on how the designers intended the system to operate (e.g. FMS and
autoflight). This information is often lost in the long chain between designer and
operator.

Error Detection ^ r orrection: Self and Automatir

Humans make and usually detect errors routinely. The same mental processes that
allow humans to cope with novel problems can also lead to error. Bill Rouse has
argued that errors are not inherently bad but their consequences may be. He
proposes the development of "error-tolerant" systems that detect errors and take
steps to prevent the consequences of the error from occurring. Research should
be done on self and automatic detection of random and unanticipated errors. For
self detection, displays should be developed diat make the consequences of errors
immediately apparent. For example, electronic map displays graphically show the
consequences of horizontal flight plan entry errors. Vertical profile displays
should be developed to make apparent vertical flight planning errors. Other
concepts such as "energy circles" could also help the crew detect gross flight
plannmg errors. For automatic detection, systems should be developed that can
track pilot activity, infer pilot intent and inform the crew of potential errors
before their consequences are realized. Systems that perform a reasonableness
check on flight plan modifications by checking route length and magnitude of

132

course changes are simple examples. Another example would be a system that
checked the aircraft's planned altitude against a data base of world terram
elevations.

.Sitiiation/Svstems/Confi^urat i nn Awareness

Comprehensive knowledge of current status is necessary to make appropriate
error-free decisions. In autoflight control systems, the pilot should know how
close the system is to its performance limits. Trend information should trigger
annunciations of potential loss of control authority problems. For example the
message, "You are using up your control authority," might have been helpful to
the crew of the China Air flight that lost control of a 747 on a flight to San
Francisco after a single engine failed. Similarly after a subsystem failure, the
pilot should be able to call up displays showing the consequences of the failure m
terms of remaining subsystem functionality and any new operational limitations.

D^c i ^isinn Support and Inform ation Management

Systems should provide information appropriate for the current flight situation.
This could include suppression of non critical information during critical phases
of flight. The system should also be capable of answering WHAT, IF and WHEN
questions to support the pilot in exploring options and deciding on a course of
action.

MEDIUM PRIORITY ISSUES

Cockpit Resource Managem ent (CRM) and
Crew Coordination in Advanced Technology Aircraft

Good cockpit resource management (CRM) is an important element in the
detection of crew errors. The CRM concept was developed with older lower
technology and may need to be updated for the new two-crew advanced
technology cockpits.

Standardization

Standardization of equipment would reduce errors due to transitioning between
cockpits but it may have a negative impact on progress. We do not want to
standardize on an error-prone and difficult to use design. Unfortunately,
standardization is most important on complex systems like the FMS that most
need to be improved. In addition, renewed emphasis should be placed on
standardization of fundamentals which affect human performance.

133

The Minimum Equipment T js^

Operational decisions as to what equipment is on the minimum equipment list
(MEL) should consider the human role. It was felt that the designer's concept of
how the aircraft should be operated was sometimes compromised by decisions
which allow the aircraft to operate when certain equipment is not functioning.
Flying with MEL items should not result in operation below a minimum level of
capability. This minimum level should include the normal displays. It was felt
that much valuable training time could be saved if training for operations in very
rare backup configurations was not required.

Error Management: Inform vs. Protect

Should an automatic system inform the crew that they are about to exceed the
performance envelope of the aircraft or should it unilaterally prevent the aircraft
from exceeding its performance envelope? This issue is closely related to the
larger issue of the proper role of automation in the cockpit. It is a very important
issue but there may not be any empirical way to address the problem. It is also
not an "all or none" issue. An "inform" cocoon could be inside of the "protect"
cocoon. A "protect" cocoon could be turned off in certain situations at the pilot's
discretion.

On-Line Help

Help functions should be provided on the control display unit (CDU) for
nonroutine operations to reduce the need for in-flight consultation of the manual.
This will help insure the optimum use of equipment and help prevent errors.

Workload Manageme nt rBoredom and High Workload)

Resolution of the role of automation in the cockpit should enhance workload
management. In addition, during long flight legs, consideration should be given
to on line training of complex systems such as the flight management system
(FMS). Other possibihties are interactive electronic flight manuals.

Error-Focused Design Methods

Theories of human error and design guidelines developed by cognitive scientists
should be applied to the design of new cockpit systems. For example. Professor
Donald Norman's new book, "The Psychology of Everyday Things," describes a
theory of human action and error and offers numerous guidelines on how to
design systems that are easy to use and less error-prone.

134

Human Factored Certifir atinn Methods

The certification process should include explicit human factors criteria.
Numerous methods have been developed in the field of human-computer
interaction for evaluating the adequacy of an interface design. These methods
should be adapted to provide more objective evaluation methods and guidelmes
for human factors certification criteria.

SUMMARY

In addition to these specific automation issues and recommendations, the group
members felt that defining the proper role for the automation in a human-
centered aviation system was of fundamental importance in the prevention and
detection of crew errors.

135

Flight Deck Automation: Promises and Realities

Final Report of the Working Group on

DESIGN AND CERTIFICATION

Chair: Richard Gabriel, Douglas Aircraft

Vice Chair: William Reynard, NASA-Ames Research Center

Members: Donald Armstrong, FAA-Aircraft Certification

Nomian Geddes, Search Technology

Al Mattox, Allied Pilots Association

Samuel Morello, NASA-Langley Research Center

Kenneth Waldrip, Air Line Pilots Association

William White, FAA-Washmgton, DC

Fred Womack, Piedmont Airlines

INTRODUCTION

The issues of design and certification of transport category aircraft are both
complex and interrelated. Certification regulations, to some extent, do effect
design decisions. But the current certification criteria, relating primarily to
workload and automation factors, are not specifically identified The working
group on design philosophies and certification was charged with identifying the
major factors underlying the effective use of automation technology, its
introduction into an operational environment and directions for the future in
design and certification.

A major question which must first be addressed is, "Why should automation be
introduced onto the flight deck?" Although many answers could be given to this
question, among the most important are:

1) It allows the flight system (crew + aircraft) to attain a broader
operational capability.

2) Economic efficiencies can be more easily obtained. The most
notable example is in fuel management.

3) System consistency is improved. The introduction of automation
allows for the reduction of operational variability.

4) Automation has clearly enhanced safety.

137
PRECEDING PAGE BLANK NOT FILMED

ttajjLj^^

With these positive aspects of automation identified, it is now possible to proceed
with a discussion of how the design and certification of automated systems might
be improved and what are the major issues surrounding such an improvement.

DESIGN

The issues surrounding the design of automation for the flight deck are complex
and interrelated. The discussion of this topic is organized into the following six
subjects: the design of the interface, the pilot's role, evaluation criteria, current
performance, unpact on the NAS as a whole, and redesign/retrofit issues.

Design of the Int erface to the Automated System

Among the most central of the issues is the design of the interface to the
automation since the crew must communicate with the automation through this
mterface. This communication wiU be required regardless of the ultimate role or
responsibility assigned to the crew. "Unfriendly" interface designs seem to be
among the most common complaints about automatic systems. This is particularly
true in the design of those automatic systems which are affected by the ATC
environment. Unfriendly interfaces are a bigger problem when there is a
requirement to modify the data/instructions to the automation under the pressure
of time and/or short notice.

Interface designs must also assure appropriate situation awareness. Careful
attention must be given to the effects of consistent presentation of status
information and prioritization of cautions and warnings. Preservation, where
possible, of tactile cues to maintain awareness is also important.

For computer controlled systems, consistency in data entry, information
retneval, and procedural aspects are very important. Page seeking should be
mmimized and on-line "help" should be available as necessary. Systems such as
the PMS, FMS, INS, OMEGA, ACARS, TCAS, and Mode S were among the
systems felt to require the most careful interface construction.

Automation, particularly that associated with the CRT or glass cockpit, may also
be a distraction in the cockpit. The extent, or even whether this is a serious issue
is not known. The issue of concern is the time and/or attention dedicated to
adjusting or changing the automatics as opposed to flying the aircraft.

Role of the C rew and Automation

A critical question both in this working group and throughout the conference are
the roles of the automation and the crew. The preponderance of opinion was that

138

the automation should take a greater role in the basic stability and control of the
aircraft, particularly for aircraft which have relaxed and/or unstable flight modes
(e.g. tilt rotor concepts). Higher level functions, however, such as flight
planning/replanning, system status management and decision making, should not
be completely relegated to the automation.

This view is primarily based on the fact that the fundamental pilot functions have
not changed and are not expected to change regardless of future improvements to
aircraft and/or the NAS system. In particular, the basic flight crew functions are
to:

aviate, navigate, communicate, and operate.

The goal of automation should be to aid the crew as they participate in the task
and manage the aircraft as a system.

With respect to operating philosophies, there is a difference between automated
and non-automated aircraft. The responsibility of the pilot is the same in both
types of aircraft but the actual activities required to accomplish these tasks are
very different depending on the level of automation. In recognition of this fact,
several carriers have elected to divide their fleet so that pilots do not fly both
automated and more manual aircraft in the same time frame.

One central issue regarding the role of the pilot and automation is that of designs
which try to provide error protection. Envelope protection has been cited as an
example of this form of error proofing. On the positive side of this issue, this
trend is consistent with the historical trend toward increasing automated
intervention, particularly in the guidance and control area. On the negative side,
however, such error proofing is not as reliable as hoped. These systems have
failed and will continue to fail. Although the design goals hoped to achieve failure
rates of less than 10 to the minus 9, the actual operational experience has been
(and will continue to be) much lower than this number, because designers are also
human. They are also subject to error and, in particular, are not able to anticipate
every possible operational situation, particularly in the complex aviation
environment.

It is, therefore, very important to provide a manual override for automated
systems and to provide the pilot with the mechanisms for a greater role in
decision making. In particular, it is important to distinguish between those
automated systems which completely perform the task and those which aid the
pilot so diat the crew is a participant in the system.

139

Another question is whether a high level of automation would violate FAR 91,
Automation should not take away the Pilot-In-Command's authority and
responsibility. In short, the design should not isolate the human from the ability
to intervene and manage the aircraft at all times.

The role of the crew is summarized in Figure 1. The central circle, situation
dominance, indicates that the flight crew must maintain "legal" awareness and
control/dominance over the aircraft status. The crew must have the information
and ability to exercise operational judgement, contingency management, systems
management, and flight planning and replanning. In addition, all of these
functions must have an appropriate "manual" capability. The role of the
automation should be to support the crew particularly in guidance and control,
navigation, and system monitoring.

Evaluation Criteria for Automated Svstems

Several characteristics were suggested which could be used to define an
appropriate implementation of automation versus a poor implementation. As
discussed above, it is very important that the implementation be user friendly.
Ease of training is a natural corollary to such an implementation, since a good,
self explanatory design can reduce die need for extensive training. In addition,
functional adaptability is important. More definitive guidelines or criteria need to
be established so that designers can evaluate systems as early in the design phase
as possible.

Performance of Current Automated Svstems

The general discussions regarding current automated systems indicated that
functional systems such as the FMC (flight management computer) are the ones
with the principal issues. Other automated systems such as those for fuel and
pressurization work well. Again, the central issue seems to be that of designing
interfaces which can facilitate and support the active role of the crew.

More specific information could also be used in this area. To what degree, if any,
are the operational crews uncomfortable with the kind or degree of automation?

IMPACT OF AUTOMATED FLIGHT DECKS
ON OTHER PARTS OF THE NAS

The National Air System (NAS) is composed of many interacting elements. A
new technology implementation in one part will necessarily impact the other
elements. The introduction of new automated technology onto the flight deck has,

140

however, been incremental. This is, of course, the usual method for introducing
new technology. It can, however, cause problems to occur particularly where the
older technology must interface with the new.

• Maintain operationai safety

• Goai setting

• Situation assessment

• Systems management

• Operationai Judgement

• Maintain "iegai" status

. Contingency management

Figure 1

For example, the ATC system is currently in the process of being updated, but
automated flight decks must currently interface with the existmg, less-automated
ATC technology. Although there is clearly an impact from the mcremental
introduction of automation, the magnitude is not understood. Future
implementations would do well to study the impact of a new implementation on

141

the total system, before it is introduced so that potential system-wide anomalies
could be evaluated before the operational phase begins.

Redesign/Retrofit Tssnt^s

The current organizational process for designing and approving systems does not
adequately address the issues associated with redesign and retrofit. In the
introduction of a new technology, lessons are frequently learned or apparent only
during the operational phase. This may necessitate a redesign and/or a retrofit
into the existing fleet. However, the cost of redesign, certification, and retrofit
often make the actual implementation impractical and operations may continue
for years using the existing system supplemented only by aircrew training. The
panel report on operational experiences cited some examples where this type of
operational "crutch" had not been effective.

Summarv of Design Directions for the Future

The major issues associated with the design of fumre automated aircraft are
summarized below.

1) ROLE OF THE FLIGHT CREW: SITUATION DOMINANCE

- Pilot's Role Must not be Compromised

- Tasks can be Appropriately Delegated to Automation

2) ROLE OF AUTOMATION: AIDING THE FLIGHT CREW

- Optimize Resource Efficiency & Economy

• permit crew to do more in a complex environment

• achieve better perspective of "big picture"

- Reduce Operational Variability

- Enhance Safe Operations

3) DESIGN PHILOSOPHY: HUMAN-CENTERED AUTOMATION

- Achieve Objectives Cited Above

- User Friendly

- Adaptable to Function/Option

- Support Crew in Functions which do not Compromise Crew's
Role of Situation Dominance

142

4) INTERACTION OBJECTIVE: CREW INVOLVEMENT &
PROFICIENCY

- Use a Systems Approach to Assure Flight Crew Manual & Cognitive
Proficiency

- Designs with Crew as Backup should Permit Ease of Task Recognition
& Performance without Confusion, Hesitancy, or Lack of Skill

- Eliminate Growing Tendency to Peripheralize Crew Involvement and
Function

5) CRITICAL DESIGN NEED: DETERMINATION OF BENE-
FITS AND LIMITATIONS OF STANDARDIZATION

CERTIFICATION ISSUES

The certification issues are broadly grouped into 3 categories: Certification
policy, validation of automated systems/software, and human factors criteria.

Certification Policy Issues

The current FAA philosophy regarding certification generally does not attempt to
influence the introduction of new technology in any way. The position is largely
one of allowing industry to move as necessary to build the best possible aircraft.
The FAA role is simply one of evaluating the "safety" of the resulting aircraft.
As we have seen, however, the introduction of new technology itself can
contribute to operational safety issues. This is particularly true due to the rapid
introduction of advanced, computer based technology onto the flight deck.

The philosophical issues surrounding the question of constraining die introduction
of new technology are complex. No specific recommendations regarding this
issue were made by the group, but it is a factor which should receive further
discussion.

Another philosophical issue involves certificating to standards versus certificating
to guidelines. Most other aircraft systems such as structures, are primarily
hardware based and specific numerical standards can be developed. Automation,
however, is a combination of many disciplines including computer hardware,
software and human factors. The complex interaction of the components does not
always lend itself to specific standards. In addition, the technology is changing so
rapidly that it would be difficult to develop exacting standards which could be
effective over the next decade. In view of these factors, the use of guidelines for
automation as opposed to standards should be discussed.

143

A last policy issue, but one of primary importance, is the way that the
certification process considers (or does not consider) the trade-off between
understandability and training. Since the design of the interface to an automated
system was an important issue in the previous section, the philosophy of
certificating to the "understandability" of an interface is most important.
Currently such trade-offs between the need for training and the complexity of the
design are not usually apparent. Perhaps they should be.

Validation Qf Automatgd Systems/Sgftware

Software validation is not usually an easy process and yet it is very important to
the proper functioning of most automated systems. The certification process must
insure that the software will function as expected under all conditions. Yet, is it
possible for the certification process to adequately make this assurance? How can
this best be accomphshed?

Human Factors Evaluation Criteria

The hardware components of an aircraft receive substantial testing before they
are certificated. This testing typically involves stressing the structure to the
failure point in a "shake and bake' manner. The automated systems aspects,
particularly those associated with human factors, do not receive such thorough
testing. This is partially true because such tests and procedures are only now
becoming available. An issue of concern is the extent to which more and better
human factors testing of systems is required prior to certification.

Sunmiary of Future Directions for Certification

The major issues associated with the future directions for certification of
automated aircraft are summarized below.

1) RE-EVALUATION/DESIGN REVffiW REQUIREMENT

- No Certification Standards can be Appropriate over the Entire
Life of an Aircraft

- After a Reasonable Service Time, Operational Shortconiings
Should be Addressed

2) FULL/PART MISSION SIMULATION TO TEST DESIGN

- To Help Prevent Costly Redesign

- To Permit Certification Staff to Observe and Prepare for the
Flightiest

144

3) REVISE AND STRENGTHEN CERTIFICATION PROCESS

- Crew Interaction

- Automation Interface

- Human Performance

4) ESTABLISH INSTITUTIONALIZED COMPREHENSIVE FEED-
BACK SYSTEM

- To Identify Opportunities for Subsequent Design Review

- To Anticipate Next Generation of Aircraft

5) ESTABLISH MULTI-DISCIPLINARY CERTIFICATION DESIGN
REVIEW

- At Beginning of New Aircraft Development to Learn, Review,
Critique, etc.

6) ESTABLISH CRITERIA FOR ASSESSMENT OF HUMAN
PERFORMANCE DESIGN AND ENGINEERING

- Provide Design &. Engineering Guidelines

- Assist Certification & Review Processes

7) FLIGHT DECK AUTHORITY FOR FAA CERTIFICATION
AND FLIGHT TEST PERSONNEL

8) ESTABLISH BETTER COORDINATION AND COMMUNICATION
WITHIN FAA RE: AIRCRAFT DESIGN/CERTIFICATION

145

SUMMARY

AND

CONCLUSIONS

mtCEDim FAGt BLANK NOT FILMED

Miijii^^*'"*'*^

» .^-•S,v

SUMMARY and CONCLUSIONS

Susan D. Norman
Chair

INTRODUCTION

The material presented at this workshop provided a particulariy comprehensive
and broad overview of automation in the air transport system today. It is,
therefore, not easy to select the most important points from the workshop and
prepare the conclusions. This section will focus on the major, global themes.

SUMMARY

A very condensed summary of the major ideas and concepts presented in the
panels and papers is given here in order to form a basis for the conclusions. A
brief section on common themes of the Working Groups is also included.

Design Issues

Although specific data regarding accident/incident rates for automated versus
non-automated aircraft are not readily available, it is clear that automated aircraft
have a good, operational record. For example, one carrier stated that they have
been flying the Boeing 767, one of the first aircraft to employ substantial
automation, since 1982 and they have never had an accident or incident resulting
in damage to the aircraft.

One manufacturer cited principles for effective system design in the following
priority: simplicity, redundancy, automation. The goal is to use automation when
necessary, but automation should not be a goal in itself. Design issues are first
solved with simplicity, then redundancy, and finally automation.

Certification Issues

A fundamental issue with respect to certification is the lack of comprehensive
human factors requirements in the current rules. Although the reasons for this
situation are complex, the result is that design engineers must necessarily make
critical design decisions long before the flight tests occur. In the absence of rules
which incorporate human factors criteria, the question is whether or not the
concems of the FAA test pilots and the line pilots he represents are adequately
considered.

149

With respect to certifying automated aircraft in today's environment, several
factors were cited as important. These are:

• Crew alerting systems

• Manual operation of an automated aircraft

• Crew over-confidence in automation

• Identifying circumstances where automatic protection is a clear
benefit (i. e. angle of attack protection, etc.)

Operational Considerations

Although automation has been a clear benefit, some factors were cited which have
been involved in incidents with automation. These include:

• Inadequate operational knowledge on the part of the crew

• Inadequate cockpit discipline and allocation of responsibilities
between the pilot-not-flying and the pilot-flying

• Inadequate cockpit resource management

• Operation in split mode (e. g., operation of autopilot without
autothrottle)

• Poor switch discipline

Lessons Learned from Non-Aviation Automation Experiences

Numerous examples have been cited throughout this document of lessons learned
from our current understanding of designing and operating automated systems.
Dr. Woods summarized these as follows:

• Shifts in automation have changed the human role in unforeseen
ways.

• Critical human role is to adapt to unanticipated situations.

• Automation changes required human skills, but does not eliminate
them.

• Automation introduces new error forms and types of system
breakdown.

In designing for human centered automation, the machine/automated system
should always provide support for the human. It should display information on
what it is doing, what it will do next and why. Provision of support for the
human role in error detection and recovery is also most important with
computerized systems.

150

Field Studies in Aviation

Dr. Wiener reported his interim findings of a study of two air carriers and flight
crew evaluation of the B-757.

• High enthusiasm for the B-757

• Training is good, overall, but there is too much emphasis on
automation rather than the basics.

• ATC limits the use of some features ( e. g., VNAV)

• Workload may be increased or decreased.

Captain Peter Heldt, chief technical pilot for Lufthansa, reported the results of a
pilot questionnaire on the A310-200. Overall, the pilots liked the automation, but
"keeping the pilot in the loop" was cited as mandatory for automated systems.

/\ f ^y^pr.ftd Automatio n System TAAS) for ATC

The schedule for implementation of the AAS calls for the first site delivery at the
Seattle Center in April, 1992, with the equipment expected to be operational at all
sites by June, 1995. The system will provide:

New automatic separation-assurance techniques

More direct, conflict-free routing

Better traffic metering techniques

Increased controller productivity

Capacity to handle projected air traffic growth

Tie together all FAA primary enroute and terminal ATC facilities

Provide greater system reliability {max. down time - about 2.5

minlyr)

rnmmon Themes in the Work ing Group Reports
Several themes were repeated in the working group reports. Some of these are:

• Design of the pilot/controller interface is crucial and it must provide support
such as status annunciation and display of an appropriate level of situational
awareness for their intended roles.

• The air-ground interface design is critical and yet there is an apparent lack of
coordination/research in this area.

• Definitive human factors criteria need to be developed.

151

• Since aircraft automated systems must support the role of the pilot, it is most
important to understand this role and to develop some level of consistency for
this role among the various components of the aviation industry from
airframe manufacturers to operators.

CONCLUSIONS

Automation is a Clear Benefit

Although most of the text of this report relates to issues regarding improvements
to our design, operational use of and training for automation, an important fact to
remember is that automation has substantially improved the operational safety
and efficiency of our air transport system. As with the introduction of any new
technology, there are components and factors which cannot be clearly and
completely understood without the severe and challenging test of day to day
operations in a real environment.

Aviation is perhaps one of the most demanding, real-time environments for use
of any new technology due to the vast number of daily operations and the
complexity of the ATC interaction, weather, etc. The safety record for automated
aircraft, however, is very good. The relatively smooth introduction of cockpit
automation into the existing system must be taken as a tribute to the technical
ability of the aviation community.

Even though the safety record for automation appears to be good compared to the
previous generation of aircraft, the potential for improvement in our ability to
design, certificate, use and train for automation was apparent at the workshop. In
particular, the aviation environment is not one where flights always proceed as
planned. Weather problems, ATC clearance revisions, emergencies and
equipment malfunctions, and even combinations of these events, occur more
frequently than we would like.

Yet automation is a technology which works best in predetermined situations such
as those which can be planned and programmed ahead, either at the time of
design or on the ground before take-off. Automated systems, because they are
really machines, are designed or programmed primarily to handle the "normal"
situations which account for most of the flying hours. However, the technology is
not yet well enough developed to provide quick and easy flexibility when the
external situation changes.

This phenomenon has been called "brittleness" and it is a characteristic which has
been associated with automation (particularly expert systems). It is not wrong in

152

itself; it is merely a limitation of the technology which must be considered in the
design, certification, training and procedural use of these systems. Non-aviation
industries have experienced the same phenomenon and much can be learned from
their experiences (refer to the Woods paper in this document).

As a result of factors such as brittleness, irregular operations in an automated
environment can become the most troublesome. Irregular operations, discussed in
more detail in the working group report on crew performance, are flights in
which normal operations have been interrupted or are no longer possible. In
these situations, the pilot's operational understanding of the system, and how it
will perform under the varying conditions, becomes crucial. The design must
support this pilot role by providing appropriate display information. In addition
under these circumstances, the human role is often to maintain the boundaries of
the automated system, perhaps by methods such as inserting an erroneous wind
vector, so that it can function effectively.

Understanding (Figure 1) thus becomes a key concept. The ability of the flight
crew to understand the automatics must be supported by all phases of the aviation
process from design through training and operations.

IlJNlE)]SIRS'irANIlDIING Ss n IKsy C(J)in(e®[pa

Of the Way it Works

Of the System Intent

Of Control Laws

Of Normal Versus Irregular Operations

Of the Implications of System Status

Figure 1

A point must be made regarding the actual systems which have been automated. It
was generally agreed that the automation of aircraft subsystems has been much
less troublesome than other systems such as those which support navigation. For
example, autofuel and autothrottle systems have functioned very well and their
operation appears to be easily understood in the cockpit. These systems, however,
do not depend on external information sources such as pilot data entry and
ground transmitting stations. As such, they do not always have the inherent
interface complexity of other automated systems, such as those used for
navigation.

153

Because the factors which affect the performance of an automated system can be
complex and may not always be immediately apparent, the crew communication
in the cockpit and with ATC about the operation of the aircraft must compensate
for this situation. In addition, actions on the part of the automatics may
sometimes be transparent to the crew. In older technology aircraft, these actions
are performed and checked by a crew member. In summary, the use of
automatics necessitates closer crew communication as weU as a closer interaction
in all phases from design to operations (Figure 2).

(CL(DSIEE UNTrHRACTIKDMS
A Close Interaction is Required in All Elements of:

• Design

• Training

• Operations
. ATC

Examples:

Between the Automatics and the Flight Crew

Between Crewmembers on the Flight Deck

Between Flight Deck Design and Operational Procedures

At the System Design Level (System Level Perspective)

Between the Flight Deck Crew and ATC

Figure 2

Comments from the Participants

The following comments from verbal remarks and anonymous evaluation forms
may help to provide a basis for understanding the essence of this document.
Participants were asked the most important new idea or issue learned at the
conference:

154

"The basic human factors associated with automation are generic and
are Ml being adequately addressed by either ATC or cockpit
automation designers."

"Man must not be replaced by but rather learn to make
effective/ efficient use of automation in the aircraft environment."

"The problems generated by introducing automation in the cockpit
are very similar to the problems encountered in automating the
control tower..."

"Automation is working very well in many applications and users
want more, not less; but there are problems that need work."

"The automation problem is very broad and it is very important to
attack it systematically."

A valuable part of the conference was the exchange of ideas on
... "how others are coping (training and operational tips), what design
changes and enhancements are in store and the concepts of automatic,
semi-automatic, and manual operation when control is necessary in a
time constrained or irregular operation."

Direction for the future is also an important issue and several pertinent comments
were made regarding it.

"The role of the pilot should not change. Automation has to be added
carefully. There needs to be early design guidance to accomplish
good automation."

"We must find ways to introduce the equipment more effectively. In
my training experience, a number of pilots were very reluctant when
first introduced to the new equipment."

One of the most critical areas to emerge as a need for the future is
the "development of improved interaction between the air and
ground sides of the National Air System.... I am concerned that the
cockpit is expanding beyond the ATC capability. This may cause the
airborne side to be incompatible with the ground operations and
pilots may not use new systems."

155

"The industry is trying to run with its brakes on. Information dissemination,
particularly to the higher levels of manufacturing and operational management, is
essential.

The Role of fhft Pilot

One important issue to emerge from the conference concerns whether or not the
role of the pilot has changed as a result of automation. Several controversial ideas
were discussed and I have elected to quote from several letters written to me as
chair after the close of the conference. Drs. Rolf Braune of Boeing and Earl
Wiener of the University of Miami most articulately presented the relevant issues
in this debate and their views are cited here.

The definition of temis becomes crucial in this discussion and "goal," "role" and
"functional level" are all important aspects. Regarding the goal of the pilot/ flight
crew, Dr. Wiener writes that the goal of the pilot is:

"....to fly the aircraft from A to B with maximum safety, minimum
cost, and maximum passenger comfort. This strategic goal is
unchanged, but in order to achieve this goal, tasks and subtasks are
carried out, and these, of course, have been altered by automation
and other cockpit equipment..."

As stated, this goal (to fly the aircraft from A to B ) for the flight crew is

relatively uncontroversial; however, the issues of role and the impact on the
specific tasks which the flight crew must now perform in an automated
environment are more difficult to assess.

Regarding the role of the pilot. Dr. Rolf Braune pointed out that:

"...the role of the pilot has not changed and probably will not change
until Federal Air Regulation (FAR) 913 is changed. This regulation
states that the pilot in command of an aircraft is directly responsible
for, and is the final authority as to, the operation of that aircraft.
This is a clear definition of a role. As part of this role, the pilot
always performed the functions of monitoring and managing. What
the automation technologies are attempting to do is to provide the
pilot with the tools which will simplify necessary tasks or eliminate
unnecessary tasks."

156

Dr. Wiener, however, concludes that the role of the pilot has changed. He writes:

"Role refers to a more global construct than a list of tasks. It refers
to the function of the crew member in realizing his strategic goal.
This function, or role, has been altered by the tasks. The pilot' s role
is more passive, less manual, more cognitive, etc. To fail to
recognize this is a serious impediment to our understanding and
eventual resolution of the ' automation problem' .

The role of the pilot is changing and our job is to understand the
implications of this change."

Although the above basic statements of the issues are equally valid, both points of
view draw very different conclusions regarding the apparent change in the role
of the pilot. Clearly, FAR 91.3 gives the pilot in command ultimate authority and
in this sense, the pilot's role has not changed. But the methods which pilots use to
perform their duties have been dramatically altered in the automated
environment.

Dr. Braune further clarifies the important difference here when he writes,

"The apparent disagreement over whether the role of the pilot has
changed or not may arise because we are looking at different
functional levels of the pilot's task hierarchy. Afunctional hierarchy
of the commercial pilot's task environment should be developed.
This would help to identify:

• those functions which may have changed,

• effects of the change,

• ways to counter those with negative effects,

• level of these functions.

In addition, there is a limited amount of objective data to help focus
on the real issues. We may be confronted with over-generalizations
which do not help understand the real causes of the issues we have
discussed, particularly in light of the highly favorable accident
statistics for advanced technology aircraft."

In summary, the consensus at the conference was that the role of the pilot should
not change. The flight deck crew must be actively involved in the operation of the
aircraft and the substance of FAR 91.3 should remain in effect. However, as the
tools and methods used to fly aircraft change, the choices which a pilot can

157

realistically exercise may also change. It is in this sense that a fundamental change
in the pilot's ability to accomplish the intended role may occur.

Unfortunately, as Dr. Braune has correctly pointed out, there is a paucity of data
regarding the actual, operational task responsibilities of the flight crew. Due to
this lack of data, only opinions can be given. As chair, it is my opinion that the
intended role of the pilot has not changed, but that the actual abihty of the flight
crew to exercise much of the assigned role/authority has indeed been changed by
many factors including the ATC environment as well as the implementation of
current flight deck automation.

Dr. Braune indicated the need to understand the pilot's task hierarchy more
clearly and Dr. Wiener suggests that it is our job to understand the magnitude, as
well as the implications, of these changes. Both of these needs are important
topics for future research, particularly if we are to understand clearly how to use
automation technology more effectively.

DIRECTIONS FOR THE FUTURE

Although much has been accomplished regarding automation of the flight deck,
there is still much to do. The workshop provided a valuable opportunity to
exchange experiences regarding operations and training for automated aircraft.
Continuation of such opportunities needs to be provided by the relevant
organizations and government agencies.

The need for quantifiable data regarding the implementation and interface designs
for automated systems is also very apparent. Full mission simulations which
compare alternate displays, etc. and test the performance of the entire system
(ATC + flight crew + aircraft) would be very helpful.

Improvement in the human factors aspects of the certification process is needed.

Additional research is needed to better understand how to support the human role
in an automated environment. This involves interface design, training and
operational procedures.

Improving our understanding of the air-ground interface is crucial for the future.
This development is necessary if coordination of the planning and implementation
of advanced automation for both the flight deck and the air traffic controller are
to proceed in a integrated manner. Automation provides the potential for one part

158

of an automated system to impact another, sometimes without direct human
intervention. As a result, it is important to examine the implementation of
automation as an overall svstem so that the design implications can be made
visible before the operational phase commences.

159

p(<£C£DU>iu k-v-^o

'■- t::.-^-^ P! ^?'''i HOT tiLMED

4£

IHTIMIW**^^^

WJ^t«'

APPENDIX A

AIRCRAFT AUTOMATION PHILOSOPHY:
A SOURCE DOCUMENT

This report was prepared to provide

an initial basis for discussion for the participants in

the NASA/Industry/FAA workshop,

"Flight Deck Automation: Promises and Realities,

Carmel Valley, California, August 1-4, 1988.

NASA Ames Research Center
July 1988

PREGEL^i^^L: f.

163

t?A6iiiil-WttNnOII*Ul HAM

This report was prepared based
on contributions by the following participants

at a preliminary workshop on
Philosophy of Flight Deck Automation

held at Carmel Valley,
April 25-26, 1988

Susan Norman

Charles E. Billings

David Nagel

Everett Palmer

Earl L. Wiener

David Woods

with contributions by:

Ren Curry

Norm Geddes

Grace Pariante

Automation: "automatic operation or control of a
process, equipment, or a system" (American Heritage
Dictionary, 1976)

Automatic: "acting or operating in a manner essen-
tially independent of external control; self-regulating" —
but also, "lacking volition, intention, or conscious plan-
ning" (American Heritage Dictionary, 1976)

164

AIRCRAFT AUTOMATION PHILOSOPHY:
A SOURCE DOCUMENT

CONTENTS
1 . Introduction

167

2.0 The Evolution of Automation in Civil

Transport Aircraft J°^

2.1 The Beginnings }^°

2.2 Early Autopilots j^°

2.3 The Second Generation Jo^

2.4 The Third Generation j^^

2.5 The Next Generation 172

2.6 The Historical Implications: A Summary 1/4

3.0 Theoretical Effects of Introducing Automation:

Examples from Non-Aviation Industries 1/4

3 . 1 Implications of Technology Centered

Develqpment j^^

3.2 Peripheralization J^^

3.3 Introduction of New Error Forms 1/6

4.0 Effects of Automation Technology on Aviation 177

4. 1 Problems with Technology Centered

Approaches 1'°

5.0 Technology-Centered versus Human-Centered

Automation 1°"

5 . 1 Toward a Working Philosophy J » }

5.2 A Conceptual Framework jol

5 . 3 Emergent Principles for Design j °^

5 . 4 Emergent Principles for Operation 1 o4

6.0 Toward Implementation 1^"^

7.0 The Role of the Human in the Future System 186

165

AIRCRAFT AUTOMATION PHILOSOPHY:
A SOURCE DOCUMENT

1.0 Introduction

Assessing the impact of automation on civil transport aviation is difficult and
complex. Although many major benefits have been reaUzed from the mtroduction
of new technology onto the flight deck, there is an underlying concern about its
implications and long term effects.

One recent report which reflects broad aviation concerns states, "The recent and
rapid introduction of advanced computer-based technology onto the flight deck of
transport category aircraft has been associated with a dramatic change m both
aircraft operations and the role and expertise expected of the flight crew.
Although no specific accident statistics are available, there have been a number of
serious incidents, which necessitate development and testing of a critical,
scientifically based philosophy of automation" (Norman, et al., 1988 p. 1).

A discussion of the framework for a "Philosophy of Automation" and the need
for its development form the basis for this paper. An automation philosophy can
provide guidelines and constraints for answering design questions and a
methodology to evaluate both individual design decision and the overall utility of
the automation.

Beginning with a historical context for issues related to automation, this paper
explores the consequences of current automation and describes a direction for the
future which could lead toward a Human-Centered Philosophy of Automation.

2. The Evolution of Automation in Civil Transport Aircraft*

This section describes the historical evolution of automation technology, in the
hope of disceming trends in the respective roles and functions of automation and
of the humans who will operate these aircraft.

* This section is reprinted from an unpublished manuscript entitied, "Toward Human Centered
Automation", with the permission of the author. Dr. Charles E. Billings.

167

^^'J^iiL^aMIKMIUI HAM

2.1 The Beginnings

In the earliest days of manned flight, automation technology was needed to
stabilize the aircraft attitude by directly controlling the aerodynamic surfaces.
Gyroscopic devices were well suited to tfiis task.

In 1891, Sir Hiram Maxim secured a British patent for a gyroscopic stabilizer. Five
years later, it was installed and tested on a steam-powered flying machine (Pallett,
1983). Orville Wright was more successful with an automatic stabilizer that
activated the wing- warping and elevator mechanisms associated with control of roll
and pitch. The device was patented in 1913 and later won a Collier Trophy for its
inventor (Prendergast, 1980).

The following year, Lawrence Sperry received a French safety award after the
successful flight demonstration, in Paris, of a two-axis system (HeinmuUer, 1944).
In 1918, H. J. Taplin patented a system that relied on aerodynamic pressures. His
device was successfully flown in 1926 in the United States, but is not known to
have been used thereafter (Taplin, 1969).

2.2 Early Autopilots

Gyroscopic devices have dominated aircraft inner loop control (maintenance of
attitude for all spatial axes) since that time.

A Speny "automatic pilot" was installed in the Winnie Mae in 1932 and was used
extensively by Wiley Post in his successful circumnavigation of the globe in 1933.
A later model Sperry gyro pilot was installed in the Lockheed Electra whicti
Howard Hughes used for his global flight in 1939.

By the beginning of World War II, autopilots were installed in a variety of long-
range aircraft. With vacuum-driven gyroscopes (which often provided cockpit
attitude and heading reference as well), these devices were the comerstone of
automatic flight throughout the war and for several years thereafter. They
alleviated fatigue and freed pilots from the burden of constant manual aircraft
control.

Following the war, autopilot technology advanced rapidly. Vacuum-driven gyros
were replaced by more effective electrical systems and processing of autopilot
error signals was taken over by electronic amplification.

The transition of radio navigation from low frequency ranges and non-directional
beacons to very high frequency omnirange (VOR) transmitters simplified the
navigation process, insulated it from electrical storm interference, and provided
more precise directional data. The output of these radio aids, when coupled to
autopilots, provided the ability to track radials with respect to a VOR station. The
output of the VHF instrument landing system (ILS) was also used to provide

168

precise tracking of localizer beams, while the corresponding glide slope
transmitters were coupled to pitch axis control for vertical guidance.

Precise data regarding magnetic heading, altitude and position with respect to
external references, when integrated into the autopilot system, enabled
considerable outer loop, as well as inner loop, control. When commercial jet
aircraft were introduced in the late 1950s, this state of the art prevailed. SoUd-
state electronics had not been applied to aircraft. Autopilots were still large,
bulky and temperamental, but when they worked they provided the pilot with:

— two or three axis inner loop control of aircraft heading

— the ability to capture and maintain a magnetic heading

— the ability to capture and maintain a VOR or ILS track

— altitude hold and the ability to track a glide slope

— altitude select and capture ability, in sonw cases.

2.3 The Second Generation

Turbojet transport aircraft (the de Havilland Comet in 1952 and the Boeing 707
in 1958) provided substantial increases in speed and altitude capability, but
required more precise inner loop control, particularly at high altitudes. Newer,
more precise flight instruments were required by the pilots of these aircraft.

Pallett notes that "Flight instrument evolution has followed a pattern of divergent
display complexity with advancing technology followed by consolidation of the
displays as human capabilities of data interpretation were exceeded." (1983)

Flight directors were mtroduced to provide command information for both the
pilots and the automatic devices. Integrated with altitude indicators, they directed
pilots to climb, descend and turn to new headings or radials; thus they enhanced
inner loop control of the less stable swept-wing aircraft configuration, especially
during low visibility and high precision maneuvers. Pilots came to rely
increasingly on these flight directors, although substantial concerns were
expressed at the time regarding "losing sight of the raw data" from which the
flight director derived its information.

The tendency of swept-wing aircraft to yaw away from banked turns (Dutch roll)
prompted the introduction of automatic devices which counteracted this tendency
(yaw dampers). Fast jet aircraft were similarly equipped with pitch trim
compensators which counteracted the tendency to pitch down at high Mach
numbers. These devices could be turned off, but in practice they were used
without crew intervention.

During the 1960s, advances in solid state electronics made newer, more
competent autopilot/flight director systems possible. These systems incorporated

169

complex control laws including flare logic for automatic landings and provided a
stepwise increase in automatic flight capability. As a further improvement, the
automatic throttle logic integrated control of power and flight path. These
systems were introduced in the DC- 10, L-lOU, and other aircraft.

The initial impact of these new types of automated systems led to flight crew
reports of difficulty in learning to operate the more complex system aspects.
Training was modified to emphasize system operation. Pilot certification began to
require a full demonstration of the pilot's abihty to use the new systems to full
advantage under a wide variety of circumstances where previous requirements
had emphasized the ability to operate without such aids.

In 1975, after a series of "controlled flight into terrain" accidents, Congress
mandated the installation of ground proximity warning systems (GPWS) in trans-
port aircraft. These devices used radar altimetry to evaluate height above ground
and its rate of change; they provided light and voice warnings/commands to the
cockpit crew when predetermined parameters were violated. Crew responses to
the GPWS warnings were mandated by aircraft operators, but no attempt was
made to implement these automatically (i.e., without crew intervention). Such
systems did, however, represent a further extension of the "automated command"
philosophy first embodied in the flight director systems, because they annunciated
a command for the pilot to maneuver the aircraft. This was a significant exten-
sion from the previous use of automatic devices which only maintained stable
aerodynamic or navigational control. Today, wind shear advisory and collision
avoidance systems are further extending this philosophy.

In the late 1970s, the introduction of area navigation systems (first Doppler and
then inertial) and their integration with the autopilot, added another dimension to
the level of automation complexity in some cockpits. By the beginning of the
1980s, airline cockpits in the operational fleet ranged from ahnost fully manual
jets which had been manufactured during the 1960s to newer, highly automated
aircraft. This mix of variously configured automation types has in itself caused
incompatibilities.

2 A The Third Generation

The next major steps in cockpit automation were the sweeping changes associated
with the introduction of both the more flexible electronic CRT displays, the
"glass cockpit," and the automated system management devices. This automation
was motivated in part by economics as well as the desire to reduce the workload
to a level which permitted the aircraft to be safely operated with only two cockpit
crew members.

170

Aircraft such as the DC 9-80 (now the MD-80) incorporated simplified, but
sophisticated, aircraft management and electronic flight control systems while
retaining the electromechanical instruments. Systems control was now affected
through computers such as the flight management computer and alphanumeric
control display units (CDUs) used for data entry.

About this same time, Boeing introduced the 757/767 aircraft which used color
cathode ray tubes (CRTs) for primary flight information and engine and aircraft
system data. The format of the primary displays, however, retained the
conventional, electro-mechanical presentation. One exception was a new and highly
capable map display format which presented an effective alternative to die horizontal
situation indicators that had been used since the early 1960s.

These systems, along with the slightly later introduction and retrofit of new flight
and performance management systems, completed a major revolution in the
cockpit. Yet the implications of this new technology were only beginning to be
understood. While manual flying was still possible, it was not encouraged or even
practical under some circumstances. Pilots were evaluated largely on their ability
to use fully the vastly more capable integrated flight path and aircraft
management systems. Operationally, the new systems enabled vertical as well as
horizontal automated navigation and guidance. Under normal conditions, thrust
management as well had become ahnost completely automatic. In the final
evaluation, pilot and crew workload for routine situations was considerably
reduced. Other implications were, however, yet to be explored.

The new systems provided pilots with the capability to increase flight path
precision, and equally important, to do so with maximum fuel economy. They
enabled fully automatic routine flight virtually from takeoff to rollout, regardless
of weather while reducing inflight workload.

Unfortunately, it also became evident that the air traffic management system was
not sufficiently adaptive to permit full use of the capabilities of the newer aircraft
flight management systems.

Changes in ATC instructions required cumbersome reprogramming of flight paths
through the CDUs, a task requiring considerable attention inside the cockpit
perhaps during the descent or approach phase when attention outside die cockpit
was most crucial. Such changes actually increased crew workload substantially and
diverted them from managing and monitoring their aircraft's progress and
environment. It has been said, by both human factors researchers and operational
experts, tiiat die new devices tended to increase the workload when it was highest
(climbs, descents and approaches) while decreasing it when it was already boringly
low (cruise and high altitude flight).

171

2.5 The Next Generation

The next generation of aircraft about to enter service (747-400, MD-11, etc.)
have continued to introduce new forms of automation which attempt to alleviate
some of the issues previously cited. However, this automation has also brought
with it new forms of potential error. Some examples are discussed here to
illustrate important trends.

Concern about increased workload during off-nominal operations and the
consequent diversion of crew attention from outside to inside the cockpit
(especially at low altitude) have given rise to questions about the complexity of
current CDUs. The newest aircraft now entering the production cycle incorporate
attempts to simplify CDU operation by making available more comprehensive
navigational data bases and by the addition of software to make reprogranmiing
simpler. Despite these attempts, elimination of the CDU keyboard data entry has
been seriously discussed.

Another technological advance about to enter service is the introduction of
advanced control systems ("fly-by-wire") incorporating logic to prevent the
aircraft from exceeding its safe operating envelope. Automatic limitations to
angle of attack, bank angle as a function of air speed and configuration, and other
control laws will insure that a pilot cannot exceed predetermined parameters
estabhshed by the designers. These limitations apply regardless of the real time
control inputs provided to the vehicle.

Aircraft subsystem management will also be drastically simplified to reduce
workload for die smaller flight crews. Microprocessor technology has been used
to automate navigational tasks as well as management of electrical, fuel, hydrauUc
and pneumatic systems in normal operations.

Each of these technological advances, included for a variety of reasons, has not
only enabled new forms of errors, but also has had the effect of making the flight
crew more peripheral to the actual operation of the aircraft (Figure 1). Whether
this was intended or not is not within the purview of this paper, and is probably
irrelevant. Nevertheless, the pilots who formerly exercised direct authority over
all aspects of aircraft control and management, now have become responsible
primarily for the management and direction of extremely complex hardware and
software interfaces through which (and only through which) they may direct the
operation of their vehicle.

172

Figure 1 : Evolution of Transport Aircraft Automation

AIRCRAFT

CONTROLS

PILOT

AIRCRAFT

CONTROLS

AUTOPILOT

AIRCRAFT

COrfTROL
SYSTEMS

AIRCRAFT

AUTOPILOT

CONTROLS

CONTROLLER

AUTOPILOT

FMS

CONTROLLER

PILOT

CDU

NAV
AIDS

PILOT

NAV
AIDS

INS

CADC

PMS

. hv

fmwa^ngPBfiphmBitmtton of the PIM >

173

2.6 The Historical Implications: A Summary

Three major conclusions are apparent from the discussion of the history of the
introduction of flight deck automation.

• The implementation of the technology has been incremental in
nature (a component level approach was implemented as opposed
to a systems level design strategy). This is common with a
technology-centered philosophy.

The incremental development of independent, component level systems began with
the gyroscopic stabilizer technology for aircraft attitude control. Flight control
including fuel management became feasible and soon automated navigational
control was enabled by the development and installation of ground based systems.
Microprocessor and CRT technology signalled the advent of glass cockpits and
automatic systems which directly commanded the pilot to maneuver the aircraft (i.e.
GPWS).

• The technology has resulted in an increasing peripheralization of
the pilot. The flight crew has been gradually removed from direct
control of the aircraft to control of systems which in tum control
the aircraft.

• Third, new types of errors have been introduced such as data
entry errors and software engineering errors.

Before proceeding with aviation examples, it is important to note that these same
issues have been associated with the introduction of automation in other industries
and a brief discussion of the implications is instructive.

3.0 Theoretical Effects of Introducing Automation:
Examples From Non-Aviation Industries

The effects of advanced technology have been widely studied in other industries
and a wide body of literature regarding automation's theoretical basis and
practical application has been collected. Potential solutions to problems with
automation in the aviation domain may be placed in perspective based on a review
of the broader experience with these technologies.

A basic challenge to the implementation of any new automation capability is to
predict accurately the effects of the introduced changes on other system elements.
New automation has a large range of possible effects and ramifications that go
well beyond the specific task addressed by the new technology. The problem is
how to achieve the potential benefits from the new technology while finding and

174

correcting deficiencies in its utilization. Careful examination of the history of
automation reveals that shifts in automation have system-level effects which may
create new difficulties because the entire human-machine system has been
changed in unforeseen ways (e.g., Noble, 1984; Hirschhom, 1984; Adler, 1986).

3.1 Implications of Technology Centered Development

One assumption often made is that automation will not only reduce the number of
workers but also reduce or even eliminate worker skill requirements. However,
actual studies indicate that this is not always the case. Automation changes the
human role and this, in turn, changes the required skills. These skills are
frequently more demanding. For example, more diagnostic and fault-finding
tasks occur (Miller & Bereiter, 1986; Bereiter & Miller, 1986).

Examples of unintended and unforeseen negative consequences that have followed
purely technology-driven design and implementation of new automation
capabilities include:

A failure occurred during the introduction of a computer based power plant control
room alarm system because the fault management information associated with the
older annunciator alarm systems was simply no longer available (Pope, 1978; Kragt
& Boten, 1983).

Shifts ftom paper based procedures to computerized procedures have collapsed due
to unforeseen disorientation problems by the human operator who exhibited
different information needs with die new environment (Elm & Woods, 1985).

These examples illustrate that increasing levels of automation create the need for
new kinds of feedback regarding the system status, its processes, and its control
functions. Implications for the automation designer are substantial and will be
discussed in a later section.

3.2 Peripheralization

Adler (1986) describes the process of "peripheralization" that accompanies
increased levels of automation where the human's role is shifted away from direct
contact with the process and becomes one of system manager, system maintainer,
and/or machine prosthesis (provide eyes and hands for the system). Human
peripheralization can produce the boredom/panic syndrome:

Boredom — Situations which are within the range of competence (performance
envelope) of the automation and there is littie for die operator to do;

Panic — Situations which approach or exceed the limits of the automation and the
operator is suddenly thrown into a highly active, crucial role.

175

Problems have been noted in domain environments where there are extremely
negative consequences to relatively rare events (e.g, in nuclear power plant
settings).

3.3 Introduction of New Error Forms

The human's role, by default, is to make up for shortfalls in the automated
system's ability to cope with the level of variety demanded by the operational
environment (Roth, Bennett, & Woods, 1987). But, in contrast to this, the usual
design principle is to attempt to anticipate all situations with the implication that
designs which do not anticipate everything are bad. However, in any complex
environment, mcluding the National Airspace System, it is clearly impossible to
anticipate all situations. Instead, designs should be valued for being robust in the
face of violations of the design assumptions.

A new form of error can also result from the tendency of automation to increase
(or explode) the number of alerting signals that an operator must monitor. This
has several implications:

• Designing a supervisory control role involves the design of how the attention of
the human operator should be distributed in different contexts and states.

• Designs of the human interface to automated systems should facilitate proper
distribution of attention and not degrade performance or lead to a loss of
situational awareness.

• There is a need to know how to design environments to support effective
human monitoring.

Shifts in the level of automation can effect the type, consequences and frequency
of error occurrence. Multiple, interacting factors can produce special situations.
Breakdowns can occur due to ambiguous instructions, special conditions or
contexts, or erroneous assumptions about the expected operation. Frequently the
only way to control automatics is by controlling the input. The pilot or operator
may need to work around the automatics or may enter erroneous input with
possible negative consequences and/or unusual system responses.

Novel errors may be introduced when a change in one system element impacts
other elements in unexpected ways. This may occur because automation usually
increases the degree of coupling within the entire system. System coupling refers
to the interdependence of subsystem elements. Error forms associated with highly
coupled systems will increase, particularly when failures produced by several
small, seemingly unrelated factors occur at the wrong time. For example, a

176

momentary power failure or surge can result in unintentionally reinitializing the
system, leading to a misperception of the current system status.

4.0 Effects of Automation Technology on Aviation

Automation has both delivered on old promises and introduced new problems.
Safety of flight has clearly been improved and the ability to fly in all weather and
manner of flight conditions has been an improvement to the safety, cost
effectiveness and utility of aviation commerce.

Cockpit crew size has been reduced by one-third, without the imposition of
unacceptable levels of workload on the remaining two crew members; the
economic payoff of this change has been considerable. Area or pomt-to-pomt
navigation, now commonplace, has shortened flight times and decreased fuel
consumption without detrimental effects on the air traffic management system.
The availability of automatic landing guidance has improved flight reliability,
particulariy in regional areas prone to severe weather. Flight crews, m general,
give high marks to the automated systems in the newest aircraft m service.

As is always the case with the introduction of new technology, automation has had
unanticipated effects as well. However, specifics of these effects are not weU
understood, primarily because there is a paucity of quantifiable data. But, some
of the major issues are articulated in the National Plan to Enhance Ayiation
Safetv Throi i ph Human Factors . Potential issues arising around the new,
automated aircraft include:

• Substantially increased head down time.

• Difficulty in recovering from an automation failure.

• Reluctance of fUght crews to take over from a malfunctiomng automated
system.

• Degradation of pilot skills.

• Introduction of unanticipated failure modes.

. Difficulty in detecting system errors. .

• Incompatibility between advanced automated aircraft, existing ATC capability
and the rest of the fleet. (ATA, 1988)

This has necessitated a reevaluation of the implications of this technology
particulariy with respect to the current operation and future design of transport
category flight deck automation.

A complete account of the effects of advances in aviation automation technology
is beyond the scope of this paper. However, a variety of problems, often only
apparent after the new technology has been introduced and is being operated by
real operators in real situations, have been documented in civil aviation.

177

4.1 Problems with Technology Centered Approaches

The impact of the technology centered approach is difficuh to quantify but is
readily apparent in examining occurrences of accidents/incidents with the newer
technology aircraft. Systems level effects pervade most of these examples and this
category is therefore not specifically discussed.

Examples related to peripheralization of the pilot are:

7. Loss of Situation Awareness: Loss of situation awareness occurs when the pilot
develops, and fails to detect, an erroneous perception of the state of the vehicle
and its relationship to the world. As an example, a number of authors have
suggested that the pilot of flight KAL 007 may have:

a) mis-set his flight management computer,

b) failed to detect this eiror,

c) remained unaware that this error would result in eventual positioning of
the aircraft in tenninally hostile airspace.

2. Loss of System Awareness: Loss of system awareness occurs when operators of
complex systems are fundamentally unaware of automated system capabilities and
limitations or develop erroneous ideas of how the system performs in particular
situations. For example, the flight crew of the China Air 747 which entered (and
recovered from) a serious stall at about 41,000 feet over the Pacific Ocean
several years ago was apparently unaware that:

a) the autopilot was progressively increasing angle of attack in a futile
attempt to maintain constant altitude when the aircraft is not capable of
maintaining altitude on only 3 engines at 41000 feet (because of failure
of an outboard engine to spool up following engine idle conditions, also
produced by an automatic system).

b) the automatic yaw damping system was incrementally increasing aileron
control power to compensate for asymmetric yaw conditions (or
unaware, functionally, that the automation would eventually reach limits
in the ability to perform such compensation).

3. Poor Interface Design: Although in some instances it is difficult practically to
separate the interface from the automation, numerous problems have been
attributed to poor interface design.

For example, reprogramming the flight management system to accommodate a
change m assigned runway during the approach phase often represents such high
'head down" workload levels that pilots are unable to rely on the automated system
to guide this phase of flight. The automated systems under these circumstances can
be turned off and a manual approach is flown.

178

Thus, even though the autopilot fundamentally has the capability to adapt to a
change in clearance, the actual interface with the system is so complex and time
consuming that its usefulness is limited when it could be most effective. Good
interface design is regrettably still an art whose successful practice is limited by
the experience and ability of the designer.

4. Reversion to Manual Control: Pilots of advanced technology aircraft
understandably fear the loss of basic flight skills and many manually fly their
aircraft in order to maintain this skill (Wiener, 1988; Curry, 1985). While it is
difficult to find documented accidents/incidents where skill erosion has been cited
as a contributing factor, it could be argued that the reluctance of flight crews to
take over from automatic systems when there are good indications that something
is wrong results in part from skill erosion.

5. Automation Induced Crew Coordination Changes: Foushee (1988) has
described anecdotal evidence of a particularly subtle effect of highly automated
environments. Because much of previously observable human behavior has been
replaced by hidden (or hard to observe) machine behavior, one can argue for the
importance of increased (and improved) crew conununication. However, at least
one major study of crew behavior in advanced technology aircraft suggests that
crew members may actually do just the opposite. More data are needed to
evaluate this claim.

Those issues related to the introduction of new error forms are:

1. Systematic Human Procedural Errors. In a recent industry study of accident
causation (Sears, 1986), it was asserted that fully 35% of civil accidents may be
attributed to procedural mistakes by the flight crew. While the reasons for such
lapses are complex and incompletely understood, as an approximation they can be
attributed to human limitations associated with working memory, with
inappropriate attention allocation, with information processing limitations under
stress of various types (e.g., time, workload, etc.), and with human tendencies to
make performance "slips," which have been documented most recently by
Nomian (1981) and Reason (1987).

2. Systematic Decision Errors: A group of psychologists and cognitive scientists
collectively known as "Subjective Decision Theorists" has, over the past two
decades, done a revealing job of cataloging systematic biases which limit the
ability of humans to make optimal decisions. Woods (1988), and others have also
shown that joint or cascaded machine-human decision systems often exhibit
suboptimal performance, relative to what might reasonably be expected on the
basis of individual competencies. In the aviation context, the Air Florida Flight

179

90 accident at National Airport has been asserted (Nagel, 1988) to illustrate a
variety of classical human decision limitations.

5.0 Technology Centered versus Human Centered Automation

We have argued, although it is somewhat difficult to find explicit supporting data,
that the introduction of automation has been technology driven. That is, most
automation technology has been implemented without a clear understanding or
perhaps explicit statement of the actual problem that the technology is supposed to
solve. A recent industry report states that:

"There are many problems associated with the introduction of advanced technology
onto the flight deck of transport category aircraft. Many of these arise from the lack
of a consistent 'philosophy of automation'. To date, the designer has largely not
been constrained by human factors considerations nor guided by a global approach
to iJie introduction of automated systems and procedures. This has resulted in
designs where the crew has been forced into the system almost as an afterthought
and frequently is outside the system control loops. It also results in operations in
which the human is primarily a monitor of the automated systems and yet data
indicate that humans are totally miscast in this role." (Norman, et al., 1988)

Such implicit automation philosophies which have guided much of advanced
systems development in aviation and elsewhere, have tended toward
unplementations which replace human function with machine function — the
technology centered approach. As a result, the human has not always been left
\yith a task environment that is fiilly compatible with human capabilities and
limitations. In this sense, the technology driven approach also has had the effect
of leaving the human out of meaningful control and/or active participation in the
flight operation. This "Human-Out" phenomenon need not necessarily be
associated with technological advances, but unfortunately it is normally what
happens.

A logical contrasting philosophy to that of slowly removing the human from
substantive task responsibilities is to design systems such that the human is the
central element in control and management of the system — a "Human centered"
approach. Implicit in this approach is the development of designs which:

• fully utilize and enhance the unique human capabilities of pattern recognition,
information integration, learning and adaptation, etc.

• protect the system from human limitations such as systematic human error
tendencies, unreliable monitoring skills, decision-making biases and limitations
of working memory and "processing speed".

Note that this human centered philosophy, as stated, is mute on the topic of
whether the flight crews are allowed to choose to do anything they desire.

180

Additional operational procedures are required which, depending on the situation
(system context), would state the rules and circumstances for use of specific
automated capabilities.

5.1 Toward a Working Philosophy

It is not the purpose here to define a human-centered philosophy of automation.
Instead we attempt to prepare the foundation for such a philosophy by describing
its basic concepts and developing a conceptual framework for design. Toward this
end, the realities of the aviation environment must form the basis for any
philosophy which is expected to be actually implemented.

Realities:

• The civil aviation system is complex; this complexity is increasing.

• The National Airspace System (NAS) is heterogeneous; it is a highly coupled
system involving a mixture of humans and machines, each with a very broad
spectrum of capabilities.

• The flight environment is highly unpredictable (e.g., weather). Problems in one
part of the system have effects at distant times and places (e.g., traffic delays).

• The flight crews and air traffic controllers who operate the system, possess
(now and for the foreseeable future) unique perceptual and cognitive abilities
which are as yet unmatched by artificial systems.

• These same human operators have performance limitations which are
increasingly well known and understood in the context of predictive modelling.
For example, humans need help with very fast or very slow event sequences.

5.2 A Conceptual Framework

As we have noted, a working philosophy must be able effectively to consider
three important concepts: 1) a system level perspective, 2) peripheralization
issues, and 3) management of new error forms.

1) System Level Perspective: Aviation system design must reflect
global systems engineering approaches and practices. It can no
longer be designed and developed incrementally using a piecewise
integration of independently designed components.

The global questions which should be directed to die automation designer and
operator are best summarized by the following:

181

a. Is there an explicit understanding of the implications of the in-
troduction of the new, automated technology on ±e total system,
particularly the role of the human?

b . Have the reverberations throughout the total system been antici-
pated?

c. Has the new human role been properly supported?

These questions revolve around the central issues of function allocation between
the human and the machine. Such choices need to be decided both on the basis of
overall system effects including the performance/cost improvements and with
consideration for the potential of introducing new error forms. The system
perspective is most crucial.

2) P eripheralization: In order to counteract the effects of
peripheralization, human-centered automation systems must be
designed which:

a. allow for human interaction and involvement with the system
which is consistent with human intellectual abilities, skill level,
and responsibility;

b . allow for the joint and collaborative interaction and responsibili-
ties of flight crews, controllers, and ground personnel; and

c. enhance unique human capabilities.

The question of providing proper support for the new role of the human after the
introduction of new automation is important. Automation, as previously stated,
increases the peripheralization of the human. This frequently means that the
human role is changed from one of direct control to one of issuing coordinating
instmctions. Automation designs, however, do not always allow the operator to
intervene. In these cases the person has to find ways to "get around" or "trick"
the system to get it to perform correctly, particularly under special
circumstances. This situation may have resulted from a form of "designer
overconfidence." However, it is difficult both to design an interface so that
intervention is possible and to analyze all the possible circumstances which could
require the operator to intervene. But these issues are crucial to the successful use
of automation and they form the basis for the design principles which foUow.

3) New Error Forms: Aviation system elements should be designed
so that they are effectively protected from systematic human
limitations analogous to designs which have been used to protect
against machine failures.

182

In the system level view, the introduction of new error forms are interpreted as
symptoms of deficiencies or flaws in the underlying automation design or
architecture (Hollnagel and Woods, 1983). Such errors point to the areas where
improvements in the architecture of the automation are needed.

This is in stark contrast to the view that the human is an independent source of,
or contributor to, errors. Yet the technology-centered approach often assumes
that the pilot is an independent source of errors so that such events are often
remedied by increasing the level of the automation. We have seen this frequently
introduces new forms of errors.

5.3 Emergent Principles for Design

1. Decision Support: Joint cognitive systems should be designed to avoid
problems associated with human/machine decision systems. Decision support
should take the form of machine supported human decision making paradigms.
They could be designed so as to reduce decision biases and provide both action
sequences and their consequences. An attempt should be made to create systems
that are both error tolerant and error evident. Waming and alerting systems
should be: intent-driven, based on strategic goals selected by the crew; predictive,
able to forecast critical conditions; intelligent, able to recognize inconsistent
input; and able to give explanations if necessary. System designs should allow
maximum discretion of the crew in decisions to employ or not to employ cockpit
devices.

2. Interface Design: A most critical element is the task representational aspect of
interface design. Available evidence suggests that cognitive/perceptual tasks are
best mediated with information systems and displays which minimize the kind and
type of mental "transformations" required to translate between physical and
cognitive representations. Automated systems designers should evaluate the way
the pilot would perform the same function. Inmitive design allows the pilot to
understand more easily the automated system and if necessary take over from it.
Additionally, consideration should be given to annunciate to the pilot how well
the automatics are performing their function and how close the automatics are to
their own performance limits. This clear annunciation of the status of the
automation is termed transparency. In other words, the pilot should be able,
easily and quickly to understand and check machine status and parameters.

3. Procedures Support: Humans are poor and unreliable monitors (Wiener,
1987). They often fail to follow established procedural protocols under actual
operational conditions. New technologies (intent inferencing, activity tracking,
etc.) promise systems which have the capability of "intelligently" monitoring

183

humans. Procedural aiding systems, which have been used successfully in process
control settings, offer promise for reduction of procedural errors in aviation. In
system design, procedural aspects should be considered so that lower priority
pilot tasks do not interfere with the satisfactory performance of higher ones.
Additionally, consideration should be given to reducing the time to switch
between tasks.

4. Crew Coordination: Foushee (1987) has pointed out the importance of
effective communication practices between multiple crew members in highly
automated cockpits. Fonnerly visible actions of human crews have in many cases
been replaced by the invisible actions of imbedded machinery.

5. Workload Management: The strategic management of workload consists of a
redistribution of tasks, reducing workload where it is high or possibly increasing
it where workload is low and the flight regime is not critical. Cockpit tasks,
supporting materials, and ATC procedures should be designed to minimize head-
down time during critical phases of flight. If workload is properly managed,
situational awareness will not suffer due either to very demanding task
requirements or low task priority.

5.4 Emergent Principles for Operation

A more complete treatment of this topic is required, but this brief statement is
included to indicate that principles of operation are as (or perhaps more)
important than those applied for design.

Training: Initial emphasis should be on the "basic airplane" before introducing
the automation. LOFT exercises should allow flight consistent with the principle
of machine-supported human decision making.

Procedures and Policy: Company policy and procedures should consider the
impact of additional cockpit duties on high workload activities (e.g., making
company radio calls). Careful consideration should be given to operational
procedures particularly for crews flying derivative type airplanes, i.e., the fleet
contains both traditional and advanced technology derivative aircraft. In these
instances, consideration should be given to training issues as well as the
development of procedures for the advanced technology derivative.

6.0 Toward Implementation

Much work needs to be completed before the emergent principles and other
factors described here can be translated into a useful working philosophy for the
design and operational environment. It is possible, however, to describe some of

184

the realities of implementation as well as the components of a philosophy which
can then form the basic framework for further work.

Current Implementation Practices:

• A new cockpit is rarely designed from the top down. Most are incremental.
Design limitations must be recognized and accounted for in a philosophy.

• Cuirendy the only human factors issue in the certification process (part 25) is
workload System level effects, peripheralization issues and potential for new
errors need to be considered.

Components of a Philosophy:

A philosophy of automation specifies the crucial interactive nature of the
relationship between the human and the machine.

Any design philosophy of automation should have the following
characteristics:

ExpliciUy describe desi^ and op erational principles.

fYf^icrive ability to sort out improvements versus potential hazards in design
and procedures at a gross level.

A procedure for analvsis which will enable the consequences of design
decisions to be described.

A svstem level not a component level, perspective.

/V<;sftssment criteria which allocate functions between the human and machine
and a process for their application.

An analytical ISSI to identify inconsistencies in the application of the philosophy
or problems in its implementation. The test environment should be a systems
level, operationally oriented test, that would describe a non-standard but
frequently occurring situation such as a change in assigned runway on a close in
approach. The performance of specific data entry systems and procedures could
then be effectively evaluated in these well defined, benchmark situations.

A philosophy of automation is needed to provide consistency in design and
operation. It should provide a view which is consistent and, therefore, supports
system reliability. A philosophy provides design constraints on human/machine
interaction and is needed to guarantee that the role of the human will not be
unduly compromised by design or procedural expediency, cost or simply lack of
awareness.

185

An automation philosophy provides guidelines and constraints for answering
design questions, especially where experimental data are not available or not
possible to obtain. It provides a methodology to evaluate both individual design
decisions and the overall utility of automation technology.

A crucial element in any philosophy is the role of the human. The difficulty is
one of making a conscious choice to define and implement this role and to use
technology in its support. This leads us to the last and perhaps most important
questions surrounding aviation automation — the role of the pilot in the future
system.

7.0 The Role of the Human in Future Systems*

It is not unreasonable, given the capabihties of automated aircraft about to enter
service, to ask whether in fact the human crew could not be eliminated in routine
operations. Given that air traffic control were to gain a higher bandwidth means
of communicating control instructions to aircraft, would it not be possible for
those instructions to be translated directly into commands for flight path
management? The answer to this question, given full implementation of
technology currently available, is an unequivocal "yes." If the automated aircraft
systems can become sufficiently reliable (and the newest systems have been
designed to be very reliable indeed), there is no reason why ground-directed
flight, from takeoff roll to landing rollout, cannot be fully automated.

The fact that fully automated flight except for taxi and "unusual circumstances" is
possible with at most minor improvements in current technology gives rise to the
concern voiced by Wiener and Curry in 1980: "The question is no longer
whether one or another function can be automated but, rather, whether it should
be."

For many reasons, including reUability and passenger acceptance, it is extremely
unlikely that unmanned, fully automatic aircraft will be introduced into air
transport at any time in the near future. That statement, however, begs the
question of whether manned, fully automatic aircraft should be introduced; that
is, aircraft that require perhaps some monitoring, but no human intervention
during normal operations, except perhaps during taxi and ground operations.

The question posed by Wiener and Curry is very nearly academic. Ahnost fully
automated aircraft are about to be introduced into air transport operations,

* This section is reprinted from an unpublished manuscript entitied, "Toward Human Centered
Automation", with the permission of the author, Dr. Charles E. Billings.

186

whether they should be or not. Perhaps a more appropriate question would be
"Given the level of automation now available in transport aircraft, what should be
the role of the pilots?"

Automation could be designed to keep the pilot closer to the control of the
vehicle, while providing an array of information management and aiding
functions designed to provide the pilot with data regarding flight replanning,
degraded system operation, and the operational status and limits of the airplane,
its systems and the physical and operational environment. Outer loop functions,
including monitoring of operator performance, could be components of such a
philosophy of automation. The pilot could caU on automation modules to assist in
problem diagnosis, in evaluation of available altematives, and in execution of
alternative plans. The automation would serve as the pilot's assistant, providing
and calcullating data, watching for the unexpected, and keeping track of resources
and their rate of expenditure.

Is such "human-centered automation" possible? The answer is certainly "yes." Is
it likely? Unfortunately, it is exceeding unlikely unless serious thought is given to
the direction of our past and current automation development, and unless a new
automation philosophy is adopted prior to the design of the next generation of
transport aircraft.

We do not suggest that it is a conscious aim of the designers of current transport
aircraft to eliminate the flight crew from a central role in aircraft and aviation
system management. The direction of the trend in automation technology,
however, may well have precisely that result. If this is not to happen, we may
have gone as far or farther than we should go without making explicit our
philosophy of automation and examining the directions in which our automation
technology development are leading us.

187

REFERENCES

Adler, P. (1986). New technologies, new skills. California Management Review.
29, p. 9-28.

ATA, (1988). National plan to enhance aviation safety through human factors
improvements. Flight Systems Integration Committee Memorandum No.
88-06. May 1983.

Bereiter, S. & Miller, S. (1986). Investigating downtime and troubleshooting in
computer-controlled production systems. Fourth Symposium on Empirical
Foundations and Software Sciences. Atlanta, GA, 1986.

Curry, R. E. (1985). The Introduction of new cockpit technology: A human fac-
tors study. NASA technical memorandum 86659, Moffett Field, CA.

Ehn, W. C. & Woods, D. D. (1985). Getting lost: A case study in interface de-
sign. Proceedings of the Human Factors Society. 29th Annual Meeting.

Foushee, H. C. and Helmreich, R. (1988). Group interaction and flight crew per-
formance. In Wiener, E. L. and Nagel, D. C. (Eds). Op. Cit.

Heinmuller, J. P. V. (1944). Man's Fight to Fly. New York: Funk & Wagnalls.

Hirschhom, L. (1984). Beyond Mechanization: Work and Technology in a
Postindustrial Age. Cambridge, MA: MIT Press.

Hollnagel, E. & Woods, D. D. (1983). Cognitive systems engineering: New wine
in new bottles. International Journal of Man-Machine Studies. 18, p. 583-
600.

Kragt, H. & Bonton, J. (1983). Evaluation of a conventional process-alarm sys-
tem in a fertilizer plant. IEEE Transactions on Systems, Man, and Cyber-
netics. SMC-13, p. 586-600.

Miller, S. & Bereiter, S. (1988). Impacts of automation on process control
decision making. Robotics and Computer-Integrated Manufacturing. In
press.

Morris, W. (1976). American Heritage Dictionary. (Ed.). Boston: Houghton
Mifflin Co.

188

Nagel D C. (1988). Human error in aviation operations. In Wiener, E. L., and
Nagel, D. C. (Eds.), Human Factors in Modern Aviation. New York: Aca-
demic Press. In press.

Noble, D. F. (1984). Forces of Production: A Social History of Industrial Au-
tomation. New York: Alfred A. Knopf.

Norman, D.A. (1981). Categorization of action slips. Psych. Review, 88 (1), 1-
15.

Norman S., et al. (1988). Man-machine interface working group summary re-
port. Joint Government/Industry Task Force on Flight Crew Performance
Man-Machine Interface Working Group. June 1988.

Pallett, E. H. J. (1983). Automatic Flight Control, Edition 2. London: Granada
Publishing Ltd.

Pope R H (1978). Power station control room and desk design: Alarm system
' and experience and the use of CRT displays. International Symposium on
Nuclear Power Plant Control and Instrumentation. Cannes, France, 1978.

Prendergast, C. (1980). The First Aviators. Alexandria, VA: Time-Life Books.

Reason J (1987). A framework for classifying errors. In J. Rasmussen, K.
Duncan, & J. Leplat (eds.), New Technology and Human Error. New
York: John Wiley & sons.

Roth, E., Bennett, K., & Woods, D. D. (1987). Human interaction with an
"intelligent" machine. InternationalJournal of Man-Machine Studies. 27.

Sears R L. (1986). A new look at accident contributions and implications of
'operational and training procedures. Unpublished report. Seattle: Boemg
Commercial Airplane Company.

Taplin, H. J. (1969). "George," an experiment with a mechanical autopilot. Jour-
nal of American Aviation Historical Society. 14(4), p. 234-235.

Wiener, E. L.(1988). Cockpit automation. In Wiener and Nagel (Eds)., op. cit.

Wiener, E. L. & Curry, R. E. (1980). Flight deck automation: Promises and
problems. NASA Technical Memorandum 81206.

189

Woods, D. D. (1988). Coping with complexity: the psychology of human
behavior in complex systems. In L. P. Goodstein, H. B. Andersen, and S.
E. Olsen, editors. Mental Models, Tasks and Errors, Taylor & Francis
London, 1988.

190

APPENDIX B

WORKSHOP PARTICIPANTS

Participant Address List:

Dr. Kathy Abbott

Mail Stop 156-A

NASA Langley Research Center .

Hampton, VA 23665-5225

Mr. Steven Alvania

ADS 100

FAA

800 Independence Ave., S.W.

Washington, DC 20591

Mr. Donald Armstrong

FAA

3229 E. Spring Street

Long Beach, CA 90806

Dr. Rolf Braune

Mail Stop 9606

Boeing Commercial Airplanes

P.O. Box 3707

Seattle, WA 98124-2207

Captain Robert Cavill
F7400

Northwest Airlines
Minneapolis-St.Paul Intl Airport
St. Paul, Minnesota 551 1 1

Dr. Renwick Curry
Tycho Systems
2133 Webster Street
Palo Alto, CA 94301

Dr. James Danaher

Human Performance Division (TE-50)
National Transportation Safety Board
Washington, DC 20594

Mr. T. A. Demosthenes

ALPA

1 149 Snowberry Court

Sunnyvale, CA 94087

Captain Wendell Dobbs
American Airlines, M.D. 843
P.O. Box 619617
Dallas/Fort Worth Airport
Texas 75261-9617

Dr. Richard Gabriel
Dept. E-20, Mail Code: 72-15
Douglas Aircraft Company
3855 Lakewood Blvd.
Long Beach, CA 90846

Dr. Norman Geddes

Search Technology

4725 Peachtree Comers Circle •

Suite 200

Norcross, GA 30092

Captain B. S. Grieve
Britannia Airways Ltd.
Luton Airport
Bedfordshire
England LU2 9ND

191

Captain Peter H. Heldt

Lufthansa German Airlines

FRANP

0-6000 Frankfurt/Main

Germany

Dr. Barbara Kanki
Mail Stop 239-1
NASA-Ames Research Center
Moffett Field, CA 94035

Mr. Charles Knox

Mail Stop 156- A

NASA Langley Research Center

Hampton, VA 23665-5225

Mr. Rod Lalley

FAA

26026 S.E. 159th Place

Issaquah, WA 98027

Dr. Alfred Lee
Mail Stop 239-1
NASA-Ames Research Center
Moffett Field, CA 94035

Mr. Alden Lemer

ATR150

FAA

800 Independence Ave., S.W.

Washington, D. C. 20591

Captain Kenneth Malchow
Eastem Airlines
Building 30, ML\FV
Miami International Airport
Miami, Florida 33148

Captain Al Mattox, Jr.
Allied Pilots Association
Route 1, Box 258
Weyer's Cave, VA 24486

Mr. John I. Miller

Dept. C1-E62, Mail Code: 105-11

Douglas Aircraft Company

3855 Lakewood Blvd.

Long Beach, CA 90846

Dr. Samuel Morello
Mail Stop 153

NASA-Laiigley Research Center
Hampton, VA 23665-5225

Dr. David Nagel
Mail Stop 22-C
Apple Computer
20525 Mariani Avenue
Cupertino, CA 95014

Ms. Susan Norman
Mail Stop 239-21
NASA-Ames Research Center
Moffett Field, CA 94035

Captain Al Ogden

United Airlines

Right Center, B-767 Fleet

Stapleton International Airport

Denver, CO 80207

Captain Harry Orlady
Orlady Associates/ASRS
16188 Escobar Avenue
Los Gatos, CA 95032

Dr. Everett Palmer
Mail Stop 239-1
NASA-Ames Research Center
Moffett Field, CA 94035

Ms. Grace Pariante
Mail Stop 239-19
NASA-Ames Research Center
Moffett Field, CA 94035

192

Mr. William Reynard
Mail Stop 239-21
NASA-Ames Research Center
Moffett Field, CA 94035

Dr. Renate Roske-Hofstrand
MaU Stop 239-21
NASA-Ames Research Center
Moffett Field. CA 94035

Mr. Steven Rothschild
Federal Aviation Administration
800 Independence Avenue, SW
Washington, DC 20591

Mr. William Russell
Air Transport Association
1709 New York Ave., N.W.
Washington, D. C. 20006

Dr. Michael Shafto
Mail Stop 239-1
NASA-Ames Research Center
Moffett Field, CA 94035

Mr. George Steinmentz
Mail Stop 156- A
NASA Langley Research Center
Hampton, VA 23665-5225

Mr. Harty StoU

Mail Code: 77-35

Boeing Commercial Airplane Co.

P.O. Box 3707

Seattle, WA 98124-2207

Mr. William Syblon
American Airlines, M.D. 843
P.O. Box 619617
Dallas/Fort Worth Airport
Texas 75261-9617

Captain Frank J. Tullo
Continental Airlines
Post Office Box 92044
7300 World Way West
Los Angeles, CA 90009

Captain Kenneth F. Waldrip

ALPA

8550 Grand Avenue

Bainbridge Island, WA 98110

Mr. William White

"APS 430

FAA

800 Independence Ave., S.W.

Washington, D.C. 20591

Dr. Earl Wiener

Dept. of Management Science

Box 248237

University of Miami

Coral Gables, FL 33124

Mr. John Wilson

ALPA

Post Office Box 2005

Appomatox, VA 24522

Captain Jack D. Wisely

TWA

Flight Crew Training

P.O.Box 20051

Kansas City, MO 64915

Captain Fred Womack
Piedmont Airlines
Box 2720
Winston Salem, NC 27156

193

Dr. David Woods

Industrial & Systems Engineering

290 Baker HaU

The Ohio State University

1971 Neil Avenue

Columbus, OH 43210

194

APPENDIX C

INSTRUCTIONS TO WORKING GROUPS

The most important elements of the workshop on Flight Deck Automation:
Promises and Realties are the discussions which will occur in the small workmg
groups. Each participant has been assigned to a specific workmg group. These m-
depth discussions are critical because they provide the basis for the final products of
the workshop.

The panels and invited papers were selected to prepare a foundation for discussion
by describing ideas, concepts and terminology. The application of these concepts
and ideas will hopefully lead to new, more creative ways of understandmg the issues
associated with flight deck automation. Since the intent of the woiicshop is also to be
of practical value, identification of current approaches or coping strategies used
with the current generation of aircraft is also important.

Although the "Philosophy of Automation" is a major topic, the conference is not
intended to be theoretical in nature. It is, however, important to understand and
assess the existing situation before any changes, future research programs or
philosophies are developed. Obviously a view toward the fumre is important and
should be included in the discussion, but a critical, exact understandmg ot the
current situation must form the basis for any discussions of the future.

The time aUocated to these working groups by each of the participants is very
valuable and it is a rare opportunity for a broad representation of the aviation
community to be able to come together to discuss complex issues.

Because of the importance of these working groups, considerable time and effort
has been taken in the development of their goals and objectives. Careful
consideration has been given to developing workable, productive objectives. Ihe
information attached has been prepared to initiate discussion and to assist the
working group chairs, the NASA vice chairs and the individual participants.

Objectives

• Identification of the issues regarding flight deck automation

• Prioritization of these issues

195

• Identification of coping strategies used in current operations
training, and ATC interface.

• Identification of design and operational guidelines.

Expected Products of the Working Groups

1 . An informal, interim verbal report for Wednesday morning, and
a final verbal report for the closing session on Thursday.

2. A final written report will be prepared (format is described
below).

3. A list of terms which serve to describe automation related issues
(an initial draft has been prepared by Ev Pahner).

Procedures:

Each working group has an assigned chair from the industry, and a NASA vice
chair who will serve multiple functions, including host, resource person,
technical and scientific advisor, and recording secretary. Detailed working
procedures have been left to the discretion of each chair.

Working group assignments were made by NASA on the basis of several
considerations. Please note that although we have assigned one of six specific
areas to each working group, we do not mean to limit the scope of any group's
discussion to the assigned area. Instead, consider the assigned area to be a focus
for the primary area of consideration. If you wish to make recommendations in
other areas, you certainly may do so after a thorough development of the
primary area.

The working group's discussions are intended to be informal. The idea is to
describe the benefits and issues associated with flight deck automation and to
prioritize their importance. This prioritization should (where applicable) rank the
issues by their frequency of occurrence (i. e. the most common to the least
conmion). In addition, areas with a low frequency' of occurrence, but high
consequence, should be identified. Other factors such as ease of change also may
be considered. It is important to note that a consensus at this point is not
necessary.

Please note that two NASA participants, Susan Norman and Michael Shafto, do
not have working group assignments. They will circulate among all the groups
throughout their discussions.

196

Working Group Reports

It is the responsibility of your NASA vice chair to help the chair draft the
working group report which wiU be submitted to the general assembly. To msure
a uniform format, we ask that these reports foUow the format suggested below.

Each report should consist of three major sections: (1) an Introduction; (2)
Recommendations; and (3) a Summary. The Introduction should descnbe general
features of the approach taken to the assigned issue by the workmg group, and
other materials of a general, introductory nature.

The Recommendations section is the heart of the report. This section should
directly address three topics: (1) Issues Identified and Priontization of Issues; (2)
Guidelines and Current Coping Strategies; and (3) Recommendations for the
Future.

Any major areas of disagreement, minority opinions, and other similar
information should be placed in the summary section.

The interim status report is intended to be informal. It should be a short
verbal report of about 10 minutes and is intended to foster discussion among the
other working groups.

The final report should be written (viewgraph or text ) but wiU be presented
verbally on Thursday morning. Resources such as Macintosh computers are
available.

We realize that the issues described here are complex and that a full description
of each issue may not be possible or desirable. Our intent is to provide a basis for
understanding how these issues relate to one another and not necessanly to
understand any one issue completely. With this in mind, it is more important to
focus on the broad, interactive aspect of automation than to fully descnbe specific
elements

Thank you for your participation in this workshop. It is through your efforts that
we can obtain a better understanding of the complex issues surrounding flight
deck automation.

197

Flight Deck Automation: Promises and Realities

Working Group

CREW COORDINATION

Chair: Al Ogden

Vice Chair: Barbara Kanki

The design and implementation of increasingly automated systems on the flight
deck brings raises a variety of potential human factors issues relating to
individual crewmembers. In addition to these concems, however, there are issues
which may affect the crew as a whole; that is, the way in which crewmembers
coordinate their activities together. The most obvious, direct effect include
changes in task structure, changes in information flow and communication
channels, and changes in the interpersonal aspects of traditional and standard
procedures.

There are also indirect effects (i.e., effects which are less specifically tied to
flying the aircraft). These include changes in the organizational structure of the
crew such as shifts in authority and responsibility as well as effects related to the
problems of transition from one technology to another and training.

I. Direct effects of flight deck automation on crew coordination

A. Changes in task structure

i. type of tasks, e.g., active vs. monitoring

ii. distribution of workload, "who does what"

B. Changes in information flow/communication channels
i. face-to-face information transfer may decrease

ii. information may be maximally available but "who knows what"

no longer public knowledge
iii. changes in ground-based support role

C. Changes in interpersonal aspects of standard procedures

II. Indirect effects of automation on crew coordination

A. Social/organization issues

i. who is in authority/who holds the information
ii. who is responsible for shared info

B. Effects of transition

i. must pilots be fluent in both systems for backup
ii. what are the rules for switching (an additional

interface task)
iii. training must incorporate changes

198

UNDERSTANDING AUTOMATED SYSTEMS

Chair: R. Braune

Vice-Chair: A. Lee

The purpose of this working group is to provide a forum for an interdisciplinary
discussion on the need for, and implications of, operator understandmg of
automated systems. Automated system is broadly defined as any self-operatmg
machinery which controls or performs tasks (e.g., an FMC). Issues mcluded m
this group discussion is the extent to which operators need to know the operating
principles of automated systems and the need for maintenance of operator
awareness of the statiis of such systems. The extent to which operators require
explanations of system actions and the need for operators to anticipate actions of
the system will be addressed, finally, the implications of these and other related
issues on operator training, a system design, and operating procedures will also
be discussed.

199

WORKING GROUP: AIR-GROUND
COMMUNICATIONS (ATC) OBJECTIVES

Chair: Jack Wisely

Vice Chair: Renate Roske-Hofstrand

Among some of the issues to be discussed in this working group are how
increased cockpit automation affects the pilot's interaction with ATC. We should
discuss both negative and positive effects. Possible sample issues include the
following:

1. Air-Ground Matching (Mis-matching)

2. Flight Plan Changes and CDU re-programming

3. Cooperative Behavior (Responsibility and Rehance)

4. Pilot perceptions regarding new control procedures such as:

• Flow Management

• ATC intervention only to prevent conflicts

• Communications Management

• Enroute delays

If we were tasked to establish guidelines for designers of cockpit automation what
would we have to say?

What should we know about the FAA's Advanced Automation System that could
or should influence the design of on-board automation tools?

To stimulate our discussions, I have attached a brief article entitled "The Quest
for Airspace Safety and Capacity" which reports on the UK's National Air
Traffic Services' (NATS) attempt to deal with increased demands.

200

Flight Deck Automation: Promises and Realities
Working Group: Error Management Issues

Chair: David Nagel

Vice Chair: Everett Pahner

The objective of this working group is to identify the influences, both positive
and negative, of cockpit automation on the occurrence and detection of error on
the flight deck.

A key goal in the design of aircraft cockpits, aircraft operating procedures and
crew training is the reduction of incidents and accidents attributed to human
error Some have claimed that automation can eliminate human error by
removing the pilot from the control loop. Others have claimed that while some
types of errors may be reduced the automatic equipment itself introduces
opportunities for new types of human error. Like the digital watch it may
eliminate small errors but introduce the possibility of large errors. These new
error forms seem to be particularly difficult to anticipate at design time.

We will discuss:

• The changes in cockpit systems that have affected the type and
frequency of crew errors.

• Examples of types of human error that have been reduced.

• Examples of new types of human error.

• Methods for anticipating the impact of new technology on human
behavior and on the namre of human errors.

The key output of this working group will be a prioritized list of automation
issues and how they relate to errors and error detection in advanced technology
cockpits.

201

Flight Deck Automation: Promises and Realities

Working Group
Training and Operational Procedures

Chair: Frank Tullo

Vice-Chair: Harry Orlady

Operating procedures and basic training curricula are developed by the
manufacturer in order to operate their aircraft safely and efficiently under all of
the conditions they may be exposed to in transport operations— i.e., standard
operating conditions, abnormal conditions, and emergencies. They are then, with
the approval of the airline's FA A Principal Inspector (PI), frequently modified to
be consistent with the general operating practices of the airline. After the
procedures have been developed and approved pilots must be trained to use them.

Pilot training for all airlines is both important and very expensive. While there is
no disagreement regarding its importance, there has not always been agreement
on the kind and amount of training that is required to enable pilots to operate new
and different airplanes safely and efficiently. As an entirely separate issue, there
is also controversy regarding the effect of automation on training requirements.

The training issues are complex. They can vary between aircraft and among
airlines and over time. They can be identified and categorized in many ways. The
following issues and sub-issues have been discussed at considerable length in the
literature — often with considerable disagreement. Here they have been divided
into four entirely arbitrary categories: 1) Conceptual or Generic Issues; 2)
Implementation, Company Policy, and Procedural Issues; 3) Transition Training
Issues; and 4) Recurrent Training Issues. There is considerable overlap among
the categories.

I. Conceptual or Generic Issues

- The "Changing Role" of the Pilot

- Effective Crew Coordination in ADVTECH Aircraft

- Monitoring and Vigilance in ADVTECH Aircraft

- Understanding System (including Software) Logic

- Low-time Pilots in ADVTECH Aircraft

- Cabin Crew/Cockpit Crew Coordination

- Ab Initio Training for ADVTECH Aircraft

- Instructor Training for ADVTECH Aircraft

- Short vs. Long-haul Operations

202

II. Implementation, Company Policies, and Procedures

- Utilization of Advanced Technology

- Maintenance of Manual and Cognitive Skills

- Emphasis on "Automatics" vs. Basics

- Heads-Down Time in Traffic

- Contracted Training

III. Transition Training Issues

- Adequacy of Transition Training Program

- Sensitivity to Varying Pilot Training Needs

- Sensitivity to Line Operating Needs

- Appendix E Maneuvers and Procedures

- Initial Operating Experience

- "Differences" Training with "Common Type"

- Computer-Aided Instruction for ADVTECH Aircraft

- Effectiveness of Flight Management System Trainers

- Specific Transition Training Issues

• Transition to EFIS

• Aircraft Systems

• The Autopilot/Autothrottle, FMS, and MCP

IV. Recurrent Training Issues

- Adequacy of Recurrent Training Programs

- CRM (Cockpit Resource Management) for ADVTECH
Aircraft

- LOFT (Line Oriented Flight Training for ADVTECH
Aircraft

- Appendix F Maneuvers and Procedures

203

APPENDIX D

AUTOMATION TERMS AND ACRONYMS

TERMS

Note: Terms in italics are defined in this appendix.

actual complexity - The actual complexity of the system is generally of interest
only to the designer and the maintenance personnel. It has to do with how the
system actually functions.

automated aircraft or ADVTECH aircraft - Advanced technology aircraft
with CRT displays and Flight Management Systems, such as the Boemg /5 / &
767, 737-400, 747-400, MD-88, and Airbus A-310, A-320.

automatic - "acting or operating in a manner essentially independent of extemal
control; self-regulating"— but also, "lacking volition, intention, or conscious
planning" (American Heritage Dictionary, 1976).

automation - a. "automatic operation or control of a process, equipment, or a
system" American Heritage Dictionary, 1976) b. any computer processing ot
information displayed to the pilot or of control inputs from the pilot, c. usmg a
computer to accomplish a task that was or could be done by the pilot, d. using a
computer to make decisions that were previously made by the pilot.

backgrounded tasks - Tasks that have been delegated to another agent, either
machine or human. A key factor is that the person who delegates the task
remains responsible for its successful accomplishment.

breakdown - An automated system is in breakdown when its current operational
environment was not anticipated by the designers of the automated system and it
cannot cope with the environment to achieve its goals.

breakdown of the system image - The actual behavior of the system is
inconsistent widi the system image. This could be due to problems m the design
of the system or due to hardware or software failures.

brittleness - A characteristic of some forms of automation if they cannot cope
with unanticipated situations. The system performs correctly only for situations
that were anticipated during the systems design.

205
FMLvM"..-:. i-.. -... ' ■ PAGi,^£ii_IN7EMTI0IIM«[ SUM

bumpless transfer of control - Smooth transfer of control between manual
and automatic control modes; for example, if an autopilot is disengaged while
it is holding full right rudder, the transfer to manual control is "bumpy" if the
rudder correction is suddenly removed.

clumsy automation - Automatic systems equipment which are difficult to use
and understand. It may perfomi the necessary functions but is difficult to use.

complacency - A false sense of security induced by tlie reliable functionmg of
automatic equipment. Failing to maintain situation awareness because of a
rehance on automatic equipment.

coupling - A system is highly coupled when a failure of one component of the
system affects the functioning of other system components. These effects occur
quickly.

Error Terms

slip - An error in which the person has the correct intent but errs in
executing the action.

mistake - An error in which the person forms the wrong intent.
This wrong intent may be executed correctly.

error displacement - a. Errors made by the system designer
which are not evident until the system is in operation, b. The
consequences of an operator error is not evident at the time the
error is made. The error only becomes apparent later in the
flight.

error evident - The system is designed so that errors are detectable
by the operator in a reasonable time period.

error propagation - Errors in one part of the system affect the
functioning of another part of the system.

error tolerant - Errors are detected by the system and annunciated
before their consequences degrade system performance.

fixation - a. A human operator makes a situation assessment and
then fails to change it in the face of new evidence that is contrary
to the initial situation assessment, b. Cognitive hysteresis.

206

envelope protection - Does not allow any pilot inputs which exceed the
performance envelope of the aircraft. An example is "alpha floor or alpha
limit." The pilot cannot increase the pitch angle to the stall angle-of-attack.
This is possible on fly-by-wire control systems.

event oriented procedures - Procedures that attempt to locate the event which
is causing a system malfunction.

function oriented procedures - Procedures that attempt to maintain and
restore the critical safety functions of a system. They are not necessarily
concerned with diagnosing the event which caused the system malfunction.

glass cockpits - Advanced technology flight decks that use CRT (cathode ray
tube) (glass) displays for the primary flight displays.

mediated - a. Information to the pilot and controls from the pilot are processed
by a computer, b. The user works through a computer to accomphsh a task.

Model Terms

conceptual model - The users' model of how the system works.
The user can form expectations about the behavior of the system
in new situations based on his or her conceptual model of the
system.

design model - The designers' model of how the system works.
This is the system model that the designer would like the user to
have. It is also how the designer hopes the system will actually
operate.

system image - The image the system presents to the user. The
designer should attempt to make the system capabilities and
operations evident from this image. The system image includes
the design of all of the visual and interface parts of the system.

user model - This is the model of the system that the user actually
develops. It can be based on the system image, training materials,
knowledge of task or knowledge of other similar system.

perceived complexity - The users' view of the complexity of the automatic
system. This is the complexity of the "user model."

207

performance envelope - The boundary conditions within which a system can
perform correctly. For example, aircraft performance envelopes define power
and altitude limits as a function of aircraft configuration. Conceptually,
automated systems also have performance envelopes, but they are often not
well understood by system operators.

redundancy - More than one independent method exists for accomplishing a
function or task.

reversion - The process of switching from automatic to manual control.

supervisory control - The supervisor is responsible for successful completion
of the mission. Low level tasks are delegated to other machines or human
agents. Generally the automatic equipment exercises the "manual" control,
while the human sets goals and monitors the system as a whole.

transparency - a. A computer system is transparent if the user feels that he/she
is operating directly on the task and not using a computer to accomplish the
task, (example: many Macintosh applications.) b. A computer system is
transparent if it is clear how it operates.

208

ACRONYMS

AAS Advanced Automation System for Air Traffic Control

ACARS Automatic Communication and Recording System

ADF Automatic Direction Finder

AFS Autoflight System

AP Autopilot

APU Auxiliary Power Unit

ASRS Aviation Safety Reporting System

ATC Air Traffic Control

Basic "T' Traditional spatial configuration for display of primary flight
instruments.

CAV Caution/Waming

CAT n Category n approaches have weather minimums of a 200 ft ceiling and
2600 ft RVR.

CAT in Category HI approaches have weather minimums of RVR 700 ft and
no minimum ceiling for landing. Subcategories of CAT III have even
lower minimums.

CDU Control Display Unit - the pilot's interface with the FMC

CMl Captain

CM2 Copilot

CRT Cathode Ray Tube

CWS Control Wheel Steering

ECAM Electronic Centrahzed Aircraft Monitor

EEC Electronic Engine Control

209

EFIS Electronic Flight Instrument System

EGT Exhaust Gas Temperature

EICAS Engine Indicating and Crew Alerting System

F/D Flight Director

FMA night Mode Annunciator

FMC night Management Computer

FMS Flight Management System

G/A Go-around

GPWS Ground Proximity Waming System

ILS Instrument Landing System

INS Inertial Navigation System

IRS Inertial Reference System

IRU Inertial Reference Unit

LNAV Lateral Navigation

LRU Line Replaceable Unit

MCP Mode Control Panel

MODE S: When implemented, Mode S will provide the medium for a digital data
Hnk which will be used to exchange information between aircraft and
various ATC functions and weather bases.

NAS National Airspace System

NDB Non-Directional Beacon

OMEGA Enroute long-range radio navaid of very low frequency (VLF)
hyperbolic type

210

C%^

PF Pilot-Rying

PNF Pilot-Not-Flying

PROF Profile - used in vertical navigation

RVR Runway Visual Range

TCAS Traffic Alert and Collision Avoidance System

V/S Vertical Speed

VHF Very High Frequency

VNAV Vertical Navigation

VOR Very High Frequency (VHF) Omnidirectional Radio Range

211

f\JASA

NalonaJ Aaron aubcs wtd
Spfto* Admni«lr«aon

Report Documentation Page

1 . Report No.
NASA CP- 10036

2. Government Acx»ssion No.

4. Tide and Subtitle
Right Deck Automation: Promises and Realities

3. Recipient's Catalog No.

5. Report Date
September 1989

6. Performing Organization Code

7. Author(s)

Susan D. Norman and Hairy W. Orlady (Orlady Associates,
Los Gatos, California), editors

9. Performing Organization Name and Address

Ames Research Center
Moffett Field, CA 94035

12. Sponsoring Agency Name and Address

National Aeronautics and Space Administration
Washington, DC 20546-0001

8. Performing Organization Report No.
A-89196

10. Work Unit No.
505-67-21

11. Contract or Grant No.

13. Type of Report and Period Covered
Conference Publication

1 4. Sponsoring Agency Code

15. Supplementary Notes

Point of Contact; Susan D. Norman, Ames Research Center, M/S 239-21, Moffett Field, CA 94035
(4 1 5) 694-57 1 7 or FTS 464-57 17

16. Abstract

Issues of flight deck automation are multifaceted and complex. The rapid introduction of advanced computer-based
technology onto the flight deck of transport-category aircraft has had considerable impact both on aircraft operations and on
the flight crew. As part of NASA's responsibility to facilitate an active exchange of ideas and information among members of
the aviation community, a NASA/FAA/Industry workshop devoted to flight deck automation, organized by the Aerospace
Human Factors Research Division of NASA Ames Research Center, was held in California in August 1988. Participants were
invited from industry and from government organizations responsible for design, certification, operation, and accident
investigation of transport-category, automated aircraft. The goal of the workshop was to clarify the implications of automation,
both positive and negative. Workshop panels and working groups identified issues regarding the design, training, and procedural
aspects of flight deck automation, as well as the crew's ability to interact and perform effectively with the new technology. The
proceedings include the invited papers and the panel and working group reports, as well as the summary and conclusions of the
conference.

1 7. Key Words (Suggested by Author(s))
Automation
Cockpit

ATC automation
Air transport operations

19. Security Classlf. (of this report)
Unclassified

18. Distribution Statement
Unclassified-Unlimited

Subject Category - 06

20. Security Classif. (of this page)
Unclassified

NASA FORM 1626 OCT 86

21. No. of Pages
215

22. Price

AlO

For sale by the National Technical Information Service, Springfield, Virginia 22161

Internet Archive Audio

Featured

Top

Images

Featured

Top

Software

Featured

Top

Books

Featured

Top

Video

Featured

Top

Mobile Apps

Browser Extensions

Archive-It Subscription

Save Page Now

Full text of "Flight deck automation: Promises and realities"

See other formats