Reprint: 17^th Digital Avionics Systems Conference, Seattle, WA, November 4-6, 1998.

A two-part survey of pilots is summarized in which the effects of pilot reports on decision making behavior was examined. The survey involved a questionnaire on the world wide web and a more focused series of personal interviews. Replanning decisions in response to turbulence and weather were examined.

Air carrier flight plans are the result of a careful balance between flight schedule, environment (winds, weather, and traffic congestion), and aircraft performance, and they are designed to optimize the route so as to deliver the aircraft on time and with minimum fuel burn. However, these flight plans are based on forecast conditions and ideal flight patterns, and it is not uncommon for pilots to encounter unforeseen circumstances such as inclement weather, traffic conflicts, or a change in winds which may require a replanning of the original route.

When these in-flight replans are required, pilots, together with Air Traffic Control (ATC) and the Airline Operations Center (AOC), decide upon an alternate course. To make an informed decision, pilots need access to relevant data such as weather location and intensity and turbulence.

This paper describes two studies which solicited current preferences and decision-making behaviors. The focus was on determining the effect of pilot reports (or lack thereof) on decision-making.

Some in-flight replanning issues have been raised and explored in previous studies [1-11]. Several of these studies focused on eliciting pilot behavior and preferences for replanning[1-3] and on developing prototype expert systems to act as decision aids in the cockpit or for traffic management [5-8]. Other studies have focused on modeling the replanning process and on outlining the major research issues [8-10].

Still other studies have focused solely on the amount and types of information available to pilots and how it affects their decision-making process. These studies looked at "Party Line Information" (PLI) that pilots receive by overhearing communications addressed to other aircraft on shared VHF voice frequencies. This information includes elements such as pilot reports, traffic avoidance, and weather deviations. One finding which was common to all of the PLI studies was the perceived importance of weather information reported by other pilots. According to one study, pilots appear to use PLI as a supplemental information source[2]. This study performed a survey of active commercial pilots and asked them to rate different PLI elements in various stages of flight. For the cruise portion of flight, ride reports and weather information received the highest importance ratings. In fact, across all phases of flight, weather situation information and ride reports were rated first and third in terms of importance, respectively. According to further results of the study, weather information and ride reports were two of the most accurate and available PLI elements. Also, in a following simulation study it was found that turbulence and weather deviations are extremely effective in inducing action responses from pilots.

Another survey of pilot perceptions of PLI information emphasized the pilots’ need for specific traffic and weather information[4]. Still other research concentrated on the methods used by pilots to make decisions in high-risk situations[3]. According to this research, the majority of high risk situations reported by pilots were weather related. This research also highlighted the need to understand a pilot’s information needs when designing decision aids. Understanding these needs will allow for the improved design of decision aid systems. If pilot reports and weather scenarios have a large influence on the in-flight replanning decisions then it is important to understand their influence so that it can be incorporated into a decision aid.

What was not clear from previous work, however, was the degree to which pilot reports induce or inhibit replanning decisions. Accordingly, this paper presents preliminary results of two survey efforts designed to investigate how turbulence and weather-related pilot reports affect pilot decision-making.

To better understand how pilot report information is used in the replanning process on the flight deck, two surveys of active pilots were conducted. The first survey was written in HyperText Markup Language (HTML) and was conducted via the world wide web. Users could fill out the survey at their leisure using a web browser, and their responses were delivered via anonymous email to the experimenters. Additionally, the online format allowed for efficient distribution to pilots around the world. The second, "follow-on" study was a more focused version of the online study and further explored pilots’ preferences and decision-making behavior through personal interviews.

A hypertext link to the online survey was posted at several popular aviation websites, including a general-interest web site that produces weekly newsletters to aviation enthusiasts (Avweb[14]) and a site devoted to users of Flight Management Systems (Bluecoat[15]). Because of the inability to control and verify the background of respondents over the internet, several open-ended questions on the survey were used to screen out non-pilots.

Over a three-week period, from mid-January to early February, 1998, a total of 309 valid survey responses were received from the online study. These surveys came from a variety of pilots with different levels of flight experience. The respondents were grouped into four categories according to the highest level of flight rating obtained: Private, Instrument, Commercial, and Air Transport Pilot. Among the 309 respondents, 91 (29%) were Air Transport Pilots. Because the focus of this effort was on airline in-flight replanning, the data discussed in this paper include only the responses from the Air Transport Pilots. These pilots had an average of 9,678 total flight hours with an average age of 44 years. A complete summary of the survey responses for all pilot types is available in Reference 16.

At this point the follow-on study has been completed by four pilots. The results of the follow-on study can not be statistically significant with such a small sample size, however, they can provide preliminary insight as to the use of pilot reports in in-flight replanning.

In the first part of the survey, pilots were presented with scenarios involving the prospect of turbulent weather. In each scenario, pilots were told that there was a region approximately 20 minutes ahead along their route of flight in which moderate chop (turbulence) had been reported earlier. Moderate chop is a state of continuous, rapid turbulence, which may be dangerous to unbelted passengers but is not dangerous for flight.

The scenarios were designed to test the following parameters: presence of a lead aircraft, turbulence on own aircraft, pilot report of moderate chop from a lead aircraft, and an altitude change performed by a lead aircraft in response to turbulence. These scenarios are shown in Table 1. Only scenarios 1-5 were presented in the online survey. However, all 8 scenarios were presented in the follow-on study.

In each scenario, respondents were asked to choose one of three options: proceed using the current flight plan, change the flight plan, or request more information. In addition, there was space for pilots to specify the type of information they would request and to give general comments with regard to the scenario.

In each of the first three scenarios, there were clearly-preferred courses of action that received a majority of support by the respondents (Fig. 1). Whether this preferred course of action was to continue with the current plan, replan, or to request more information depended on the specific scenario, as discussed below.

The results of the scenarios are reported below. Only results which are statistically significant at the 1% level are reported (p < 0.01).

Scenario 1 was a control case, in which the subject’s aircraft was the only aircraft in the area. Pilot decision-making was therefore based solely on the prior report of turbulence 20 minutes ahead. In scenario 1, 63% of the pilots surveyed online indicated that they would request more information before deciding whether to continue with the current flight plan. In the follow-on study, all of the pilots also chose to ask for more information. This likely results from the ambiguous baseline condition in which the recency and scope of the turbulence report was not explicitly mentioned to the subject.

In scenario 2, the pilots heard via radio that another aircraft was flying approximately 5 to 10 minutes ahead of them on the same route and at the same flight altitude. However, this aircraft had not made a turbulence report. This scenario was designed to determine whether a lead aircraft could have an effect on pilot decision-making. In this scenario, pilot preference shifted significantly such that 83% indicated that they would continue on course. In the follow-on survey, one of the pilots chose to continue on course while the other three chose to ask for more information. From the online survey, it appears that the presence of a lead aircraft can have a significant influence on the pilots’ replanning decisions to proceed as planned. Additionally, from talking to the pilots who completed the follow-on study, the information that they would request would be a ride report from the lead aircraft. This implies that the pilots are reassured by the presence of another aircraft, presumably because they expect the lead aircraft to report turbulence when and if any is encountered. Thus, the lead aircraft acts as a surrogate ride quality sensor.

In scenario 3, the pilots heard a report that the lead airplane was experiencing moderate chop. Comparing scenario 3 with scenarios 1 and 2 allows evaluation of the effect of a pilot report from a lead aircraft. As in scenario 2, scenario 3 resulted in a significant change in pilot preference. In this case, slightly more than half (53%) indicated that they would request a change in flight plan. The remaining pilot preference was split approximately equally between requesting more information and proceeding as planned. In the follow-on survey, two pilots choose to change the flight plan while the other two asked for more information. This scenario shows that a pilot report of turbulence from a lead aircraft shifts pilot preference toward changing the flight plan. However, approximately 25% of the pilots indicated that they would request more information, suggesting that pilot reports, though useful, do not provide all of the information that is desired.

In scenario 4, the lead aircraft, in addition to the pilot report of moderate chop, also requested a change in altitude in order to improve ride quality. In this case, the additional impact of a lead aircraft that changes its flight plan can be determined. In this scenario, the fraction of pilots requesting a change of plan was slightly less than in scenario 3 (46%) for the online survey, as shown in Fig. 2. In the follow-on survey, two pilots changed course and two asked for more information, the same as in scenario 3. Apparently, a request for deviation due to turbulence does not carry much additional information over a turbulence report alone, and therefore does not significantly change a decision to reroute. It may, however, factor into the type of rerouting that is requested.

Finally, in scenario 5, the lead airplane was proceeding straight and had not reported any turbulence, but the subject’s airplane was experiencing moderate chop. This allowed for the exploration of the effect of actual (as opposed to predicted) turbulence on decision-making. A somewhat surprising result was obtained in response to scenario 5 (Fig.2). Fewer pilots (33%) indicated that they would request a change in their flight plan in this case than had indicated they would do so in scenarios 3 (53%) or 4 (45%). The results of the follow-on study were less clear, mainly due to the small sample size. In the follow-on study one pilot indicated that he would proceed as planned, another chose to ask for more information, and the other two indicated that they would change course immediately. From the online survey it appears that subjects were less willing to change the flight plan while they were experiencing turbulence than when the subject’s ride was smooth. The reason for this apparent contradiction is likely due to the presence of the lead aircraft. Namely, a lead aircraft that is not reporting turbulence is interpreted to indicate that there is no turbulence at its location. Thus, in scenario 5, the subjects likely decided that the turbulence would abate in the near future, and thus a change of altitude might not be warranted. In contrast, although the subject’s ride was smooth in scenarios 3 and 4, the prospect of turbulence ahead, due to the pilot report, was sufficient to lead many subjects to request a change in flight plan.

The remaining three scenarios were not presented in the online survey; however, they were examined in the follow-on study. Scenario 6 is the baseline condition for all of the cases in which turbulence was acting on the pilot’s own aircraft. In this scenario all of the pilots chose to ask for more information. As in scenario one, this likely results from the ambiguous baseline condition in which turbulence is forecast but no updates have been presented.

In scenarios in which the lead aircraft reports turbulence (scenario 7) and then requests a change in altitude (scenario 8), three out of the four pilots changed course immediately. The fourth pilot requested more information. This suggests that the presence of turbulence on their own aircraft causes pilots to give more weight to the reports of the aircraft ahead of them, especially when the report confirms what the pilot is experiencing.

In addition, several subjects commented on additional information they would have liked to have had in the scenarios. This additional information included: alternate altitudes or flight paths with smoother air (mentioned by 13 pilots); type of aircraft reporting the turbulence (10 pilots); spatial extent of the problem, or how long turbulent conditions would last (9 pilots); and the ride quality of other aircraft in the area (7 pilots). Other factors that would also be considered by the air transport pilots (as expressed in the additional comments for this question) include cabin concerns (i.e., whether meals are being served, mentioned by 5 pilots), the recency of the pilot report (3 pilots), fuel (mentioned twice) and penalty of diversion (mentioned twice). In the general comments section, diversion penalties, weather conditions, wind profile, airborne traffic, and cabin concerns (mentioned once or twice each) were also listed as other factors influencing the replanning decisions.

The effect of turbulence on the pilots’ own aircraft appears to make the pilot more cautious. As suggested in the online study, it is as if the pilot report is a surrogate weather information source. The information from the follow-on study further adds that if the pilot’s own aircraft is in calm air then turbulence reports from a lead aircraft are taken more skeptically. However, when there is turbulence acting on their own aircraft, the pilots react immediately to the reports of the lead aircraft.

The second part of the follow-on study was a study of the impact of pilot reports on replanning choices around a weather system. In the study, pilots were presented with a weather radar display and three route options (Fig. 3). The choice of Route A was to remain on their current path at 31,000 ft (FL310), which led directly into the weather, Route B was a climb of 4000 ft to FL350, and the third choice, Route C, was a deviation to the North around the weather system at FL 310. Figure 3 is an example of a test scenario in which a report of moderate chop is being given by a lead aircraft on route A (as shown by the diamond symbol).

Fourteen separate scenarios were tested, and the order was counterbalanced with each pilot in order to compensate for learning effects throughout the study. The scenarios went through the following situations on each route: absence of a lead aircraft, presence of a lead aircraft, lead aircraft reporting light chop, and a lead aircraft reporting moderate chop (Table 2).

The scenarios were presented with a light weather picture which showed an entirely green cell. Additionally, the scenarios involving Route A were repeated with a slightly worse weather display reading in order to explore the effect of harsher weather on the replanning process. This weather cell included some areas of yellow along the route of flight.

	Route A	Route B	Route C
No lead
Lead; no report
Lead; light
Lead; moderate

Analytical Hierarchy Process

For each scenario in the weather radar case study, the pilot was asked to use the Analytical Hierarchy Process (AHP) to rate the route choices that were presented. The Analytical Hierarchy Process is used to obtain subjective preferences of multiple options. It provides a means of evaluating multiple design options using weighted ranking scales [17]. The process breaks up multiple options into a series of paired comparisons which are then recombined to produce an overall weighted ranking. Most importantly, AHP retains information about the relative size of the intervals between the rankings. So it is possible to know how much better one option is than another. The final result of AHP is a pie chart in which the dominance of an option is given by its area in the chart; the larger the area, the more dominant the option.

For most scenarios, route C was the most preferable option with routes A and B taking up smaller slivers of the pie chart as in Figure 4. What these results mean besides the fact that Route C was the most preferable option of all three, however, is that route B was the second most preferable, and was preferred about three times as much as route A.

While Figure 4 shows the results for most of the scenarios, it is especially interesting to look at the results of the scenarios which involve a lead aircraft on route C. The results for the case in which a lead aircraft is present on route C without any pilot reports is shown in Figure 5.

Although it seems strange that Route A would have such a high preference rating when the lead aircraft had chosen to fly on Route C, one pilot explained that since the lead aircraft was close to the weather system and not reporting any turbulence it was reasonable to assume that the ride through the weather system would probably be acceptable.

Understandably, when the lead aircraft was reporting moderate chop on Route C, the preference for Routes A and B fell to levels only slightly higher than the ones held without a lead aircraft, as shown in Figure 6.

Pilot reports appear to play a significant role in decision-making related to turbulence. A pilot report of turbulence from a leading aircraft along the route of flight alters the following-pilot’s preference from "request more information" to "change flight plan". The lack of a pilot report from another aircraft in an area of previously-reported turbulence likewise significantly alters pilot preference from "request more information" to "proceed as planned". An aircraft that issues a pilot report for turbulence and also requests a change in altitude does not significantly alter following-pilot preferences to change course over that which occurs when the pilot report alone occurs, suggesting that the deviation request does not provide much additional information over the ride report itself.

The results from the weather scenarios also support the conclusions drawn in the turbulence scenario. Pilot reports from other aircraft have a strong influence on which route a pilot is willing to take. If there is a report of turbulence within the weather system then the pilot is not willing to follow the route into the system. However, if a pilot is near a weather system with a lead aircraft not reporting a lot of turbulence, then the pilot may be willing to follow the shorter route.

The authors are appreciative of the support provided by the NASA Langley Research Center, including Sally Johnson, Mark Ballin, and Terry Abbott. We are also appreciative of the collaboration with Bill Corwin at Honeywell, and Geoff Gosling at the University of California, Berkeley. The pilot respondents to the survey also deserve thanks for their time and input.

This research was supported by the National Center of Excellence for Aviation Operations Research, under Federal Aviation Administration Research Grant Number 96-C-001. This document has not been reviewed by the Federal Aviation Administration (FAA). Any opinions expressed herein do not necessarily reflect those of the FAA or the U.S. Department of Transportation.