Analysis of Pilot Landing Control in Crosswind using Neural Networks

Ryota Mori and Shinji Suzuki University of Tokyo 7-3-1, Hongo, Bunkyo-ku Tokyo, Japan, 113-8656 +81-(0)3-5841-6568 tt077065@mail.ecc.u-tokyo.ac.jp, tshinji@mail.ecc.u-tokyo.ac.jp AbstractHuman pilot control at visual approach has been analyzed using a neural network modeling technique. Neural network models simulate the relationship between control (e.g., elevator, throttle, etc.) and human input (visual information) and can be analyzed mathematically. In previous research, only longitudinal operation was analyzed, because the characteristic flare maneuver is said to be one of the most difficult maneuvers in normal operations. However, lateral control is also difficult especially under crosswind conditions. In a crosswind approach, crab control is applied first, and then winglow sideslip control is applied. The transition process of these two controls is called decrab, which is recognized as quite a difficult maneuver. Moreover, the longitudinal control also depends on the lateral control, and the influence of this coupling deserves some interest, too. In this paper we focus on the lateral control analysis. As in the previous research, neural network plot models are investigated with sensitivity analysis. Several simulator experiments were conducted with different pilots and under various wind conditions, and the analysis results clarify the differences in control strategies. 12 TABLE OF CONTENTS 1. INTRODUCTION .................................................................1 2. PILOT MODELING USING NEURAL NETWORK.................2 3. ANALYSIS RESULTS OF SIMULATOR EXPERIMENTS .......4 4. CONCLUSIONS ...................................................................8 REFERENCES ........................................................................9 BIOGRAPHY ..........................................................................9 Another characteristic control at landing is the so-called decrab control, which is executed under crosswind conditions. Under crosswind, the aircraft approaches the airport with the nose turned a little windward. However, if the aircraft lands with its nose still turned that way, the lateral load on the undercarriage increases which leads to damage of the aircraft. Therefore, in a crosswind landing, the aircraft is slightly banked windward and then straightened in front of the runway shortly before landing. This control is called decrab, and because of its complexity is believed to be rather difficult. Thus, it should be considered in the analysis of landing control. Moreover, the decrab control results in high pilot workload, which can affect the performance of the flare maneuver, consequently leading to a coupling effect. Therefore, in the current study, the proposed pilot analysis method is adopted to lateral control.[3] First of all, the NN modeling method for lateral control is established the visual cues which are needed for lateral control are defined, and the lateral NN structures are decided. In order to verify the adequacy of the network, Monte Carlo simulations are used, and it is confirmed that the obtained NN can imitate the characteristics of the pilots lateral control including decrab control. 1 have developed an evaluation method for pilot control[1]. Furthermore, the knowledge of human pilot control strategy can also improve the performance of automatic control systems. The subject of the analysis is the final landing control, because it is known as one of the most difficult maneuvers for airline pilots in normal operation. A characteristic maneuver at landing is the flare maneuver, which involves lifting of the nose to both land the aircraft on the main gear first and decrease the sink rate at the landing. A hard landing, which means landing with high sink rate, leads to damage of the airframe and passenger discomfort. During the final landing phase, the information changes very rapidly, and the out-the-window-view (i.e., visual information) is the main information source for the pilot. In the flare maneuver, both estimation skill and control skill are required for a good landing, so the relationship between visual cues and pilot control was modeled using a neural network (NN), which has been analyzed subsequently[1][2]. As the flare is merely a pitch-up, only the longitudinal motion was considered in the previous study.

1. INTRODUCTION
With the increase of air traffic, the number of aircraft accidents has been increasing steadily. A large number of aircraft accidents have occurred during final approach and landing, and some of them could have been avoided if the pilots control had been appropriate. Pilots skills depend on their experience, which makes it difficult to make a guideline for control. Even if the pilots skills seem sufficient under normal flight conditions, nobody knows whether the pilot can execute appropriate control in case of an emergency. Therefore, it is important to examine the pilots potential control skill quantitatively, and the authors 1
1 2

978-1-4244-2622-5/09/$25.00 2009 IEEE. IEEEAC paper#1044, Version 4, Updated 2008:11:01

The main topic of this paper is the analysis of pilot control for lateral movement. Many flight simulator experiments were conducted with different pilots and under different wind conditions, and NN models were constructed. Sensitivity analysis was applied to analyze the NN models obtained by the proposed modeling method. The analysis results imply that the control strategies differ depending on pilots and wind conditions, and they can be the key to clarifying the pilots control strategy under crosswind conditions.

the timing of this flare maneuver is not constant and proper control should be applied depending on the situation. The decrab control is observed under crosswind conditions. Under crosswind, without proper control, the aircraft is swept away down wind. In order to prevent this, there are two main control strategies. The first one is the so-called crab landing, in which the aircraft turns the nose windward. While it is easy to stabilize the aircraft, the lateral force can cause damage to the landing gear at the time of landing if the aircraft nose is kept windward. The second strategy is a winglow landing, in which sideslip is applied. In order to compensate for the side wind, the aircraft is slightly banked windward. This sideslip approach results in high workload for the pilot and discomfort for the passengers, while the lateral load on the landing gear decreases with this approach. Therefore, in general under crosswind conditions, crab control is applied first, and it is finally changed to a winglow landing. This reduces both passenger discomfort and lateral load at the landing. The transition process is called decrab control, a maneuver just as difficult as the flare maneuver itself. Of course, in addition to these characteristic maneuvers, the pilot has to maintain the proper glide path and aircraft attitude. In this research, the relationship between visual information and pilot control is modeled with NN, and the pilots information processing flow is analyzed. Neural Network Theory Artificial NNs model the biological nervous system, and consist of smaller units called neurons[5][6]. NNs can make an appropriate nonlinear mapping of input/output data through a learning process. Neuron inter-connection weights are used to store the knowledge. The output of each neuron is calculated using the following expression.
n y = f ( wi xi + b / n ) i =1

2. PILOT MODELING USING NEURAL NETWORK

Landing Control at Visual Approach During a visual approach, the pilot can generally not afford to watch the instrument panel, so the visual information proves to be the main source for the pilot during the final landing phase. At that time, around the flare maneuver, the pilot has to obtain detailed information in real time in order to deal with varying flight situations. The closer the aircraft is to the airport, the more sensitive the visual information is, compared to the instrument panel. The optical flow of visual cues, such as the shape of runway or the position of the horizon, can be used to estimate the states of the landing airplane[4]. In the current study, the target of the analysis is twofold: flare maneuver and decrab control in the final landing phase. The image of the landing control is shown in Fig. 1. The previous study focused on the flare maneuver, and thus the case of longitudinal motion only has already been considered.

(1)

x (e x e x ) /(e x + e x ) (hidden layer) where f : , and xi , (output layer) x x y, and wi are the input, output, and weights, respectively, and b is a bias of the activation function f.

Figure 1 Landing control. The flare maneuver is a pitch-up control before touchdown. If the aircraft lands with the same high sink rate as that of approach, this will result in aircrafts damage and passenger discomfort. By pitch-up control, the sink rate is decreased and the aircraft can land on the main gear first. However, 2

In this study, a normal three-layer feedforward network is applied, and the tan-sigmoid function is selected as activation function for the hidden layer neurons. The teaching data sets, which are model inputs and outputs, are presented to the network, which then iteratively updates its internal parameters (weights and biases) according to an error-backpropagation scheme to minimize the mean square error between the teaching output data and the NN output values. After the training process, the NN can be used as a model of the pilots control behavior.

Neural Network Structure In order to construct NN pilot models, the NN structure should be determined. The control devices available to the pilot are elevator, throttle, aileron, and rudder, so four outputs are needed in the network. For each output, the corresponding inputs have to be chosen to imitate the pilot control, but the selection of NN inputs is especially important for the performance of the NN. In this research, the visual information is considered to be the main source for the pilot, and several visual cues are quantified to be used as NN inputs, as shown in Fig. 2. These visual cues are carefully chosen based on pilot comments, and they are sufficient for the pilot to execute a successful landing. Each visual cue is strongly related to specific aircraft states, as summarized in Table 1.

Figure 2 Visual cues. The NN structures are decided as shown in Fig. 3 These structures are carefully chosen to imitate the characteristics of pilot control[7], and it has been confirmed that such networks can perform the flare maneuver and decrab control when they are used as automatic controllers in landing simulations[3]. In this network structure, several characteristics are observed. Firstly, all inputs include a 0.2 s time delay, which accounts for the delay of human response. Secondly, the networks also include the time derivatives of visual cues, as the derivative brings important information for the pilot, and expresses further time delay. Thirdly, separate networks are constructed for each output, as the necessary input information is different depending on the output, and otherwise the network loses the generalization capability.

Figure 3 Neural network structures. The generalization capability is also important in the construction of the pilot model. Generalization means the ability to reproduce the corresponding outputs to inputs not included in the teaching data. Of course, the NN models cannot imitate the pilot control characteristic completely, because the teaching data does not include all characteristics of the pilot control, nor is it possible to obtain an infinite amount of data. However, it is considered that human-like-

Table 1. Relationship between visual cues and aircraft states. Visual cues (see also Fig. 2) Y (height of the horizon) (mean inclination of runway sidelines) W (width of runway markers) (inclination of the horizon) X (lateral position of the runway) (difference of right-and-left runway sideline inclination) 3 Related aircraft state Pitch angle Altitude Longitudinal position Roll angle Yaw angle Yaw angle and lateral position

pilot models can be obtained based on limited teaching data. One of the main reasons why the model loses its generalization capability is that the obtained data includes human noise, but the commonly used learning schemes assume that the noise is white noise, which is very different from human noise. In order to delete such noise, a new learning scheme was proposed[8]. It was confirmed that the new learning scheme effectively deletes the human noise while learning through simulations. Analysis Method In order to analyze the pilots control mathematically, a sensitivity analysis is applied to the NN models in this study. In the sensitivity analysis, the degree of change of the output is calculated for small changes of the inputs. The sensitivity can be calculated as follows:
o yk = xi xi

flight time is 8370, and 8340 hours, respectively. Pilots C and D are junior pilots, whose total flight time is 2700, and 1320 hours, respectively. This paper focuses on the analysis of lateral control mainly, so each pilot operated the simulated aircraft under two different wind conditions: calm wind condition and crosswind condition. In each case, small wind turbulence was also added. The simulated landing airport was Shimojishima airport in Japan, which is often used for pilot training by Japanese airlines. The details are summarized in Table 2.

Table 2. Relationship between visual cues and aircraft states. Case A-calm B-calm C-calm D-calm A-cross B-cross C-cross D-cross Pilot A B C D A B C D Number of flight 2 3 2 1 3 3 3 3 Wind condition Calm Calm Calm Calm 20 kt crosswind 20 kt crosswind 20 kt crosswind 20 kt crosswind

S k ,i =

(w
j =1

o k, j

o o yh j + bk / m ) = wk , j j =1

y h j xi

o h = wk , j w j ,i f j =1

(2)

o where, xi is the ith input, and yk is the kth output. bko is the bias of the kth output, and f is an activation function in the hidden neurons.

When constructing the network, the inputs and outputs are scaled to the interval [1, 1], but the sensitivity is scaled back to the absolute values (which have a unit) so they can be compared between different experiments. Moreover, if the sensitivity is assumed to correspond to the gain in a linear feedback control model, the stability of the direction of the sensitivity can be determined. For example, if the pitch angle gets lower, the stable direction of the column is to pull up. In this paper, the direction of the sensitivity is also discussed. In the graph of sensitivity in the next section, the horizontal axis indicates the time before the landing and 0 s means touchdown. The vertical axis indicates the sensitivity, where a positive sensitivity indicates the stable direction.

Figure 4 B767-300 full-flight simulator. In each case, the 1 3 landing data sets were used to construct four NNs (elevator, throttle, aileron, and rudder network) as previously explained. Obtained flight data The time sequences of aircraft attitudes are shown in Figs. 5 (calm wind condition) and 6 (crosswind condition). The horizontal axis indicates the distance from the runway and 0 ft indicates the front edge of the runway, which is defined as threshold. The runway marker, i.e. the target landing position, is located at around 1200 ft. The vertical axis indicates the angle of each aircraft attitude (roll, pitch, and yaw). The yaw angle is defined so that an aircraft at 0 deg yaw is parallel to the runway.

3. ANALYSIS RESULTS OF SIMULATOR EXPERIMENTS

Using the proposed analysis method, pilot controls are analyzed for different pilots and under various flight conditions. First, the experiment condition is explained, and next the pilot controls are discussed using the obtained flight data and NN analysis results. Experimental conditions An experiment was conducted with a B767 full flight simulator (Fig. 4) owned by All Nippon Airways (ANA). The examinees were four pilots: two veteran pilots and two junior pilots. Pilots A and B are veteran pilots, whose total 4

First of all, the data under calm wind condition (Fig. 5) is discussed. As for roll angle, the junior pilots cause a roll oscillation before 1000 ft. This oscillation is considered unwanted because it might lead to passenger discomfort. The position at 1000 ft corresponds to almost 100 ft altitude and that is where junior pilots seem to change their control strategy. In all cases, the roll angle at touchdown is within 1 degree, which means a good landing. The yaw angle at touchdown is within 1 degree for veteran pilots, while that is not the case for the junior pilots. A small yaw angle at the landing is better because the lateral force on the undercarriage is decreased. Under calm wind condition, theoretically, it is possible to decrease the yaw angle to almost 0 degrees. Moreover, when the junior pilots were in charge, small oscillations of the yaw angle (not as big as that of the roll angle) with high frequency are also observed. Therefore, we could conclude that significant differences between the junior pilots and the veterans are noticed.

needed, which in turn may result in the wing tip touching the runway. Therefore, in decrab control under strong crosswind, the yaw angle is decreased to half the angle at approach. However, there is a significant difference in the yaw angle at touchdown between veteran and junior pilots. The veteran pilots can decrease the yaw angle at touchdown to within 5 degrees, whereas the junior pilots cannot. Moreover, in some cases the junior pilots keep the yaw angle equal to that used at the approach, which means that no decrab control is conducted. In particular, two out of the three landings of pilot D failed. Such a junior pilots control is not appropriate under the conditions simulated and might end up damaging the aircraft and its passengers.

Figure 6 Time histories of aircraft attitudes (roll, pitch, and yaw) under crosswind. The roll angle at touchdown is about 2 degrees for veteran pilots because of proper winglow control, while it is almost 0 degrees for junior pilots. In addition, interestingly enough, the roll oscillation for veteran pilots is bigger than that for junior pilots, while it is vice versa in calm wind condition. From the obtained data, many differences between the pilots can be found. The reason for these differences is investigated with the NN analysis in the next section. Neural Network and More Detailed Analysis Some points of interest were found in the control data in the previous section, so the pilot controls are analyzed in more detail using the NN analysis method. As previously noted, using the obtained flight and control data, the NN models are constructed. The inputs (visual cues) are calculated from the obtained flight data and the airport data. The NN models 5

Figure 5 Time histories of aircraft attitudes (roll, pitch, and yaw) under calm wind. Next, the data under crosswind condition is discussed. Under this condition, decrab control is also observed. The yaw angle is around 6 8 degrees first, and it decreases after passing the threshold. When the yaw angle is around 6 8 degrees, the crab control is applied. This crab yaw angle is fixed depending on the wind speed of the crosswind. As previously noted, before touchdown, winglow control is applied, and the yaw angle has to be decreased. It is better to decrease the yaw angle at touchdown as much as possible by decrab control. However, in the winglow control, in order to cancel the sideslip, the aircraft has to be banked windward. Under strong crosswind conditions, if the yaw angle is decreased too much, a high bank angle will be

are constructed separately for pilots and flight conditions and outputs (four control devices), i.e., a total of 32 NN models were made: eight cases (see Table 2) x four outputs (elevator, throttle, aileron, and rudder). Fig. 7 shows an example of the time histories of obtained control data and NN corresponding output data, which are aileron and rudder data for pilot B under crosswind. This figure shows that both sets of data share the same trend and the NN models can imitate the pilot control. Using the obtained NN models, the pilot controls are considered with sensitivity analysis.

decreased before touch down. It seems that such big sensitivities for junior pilots cause the roll and yaw oscillation. Further work is needed to clarify and explain these differences.

Figure 8 Time histories of aileron and rudder deflection under calm wind. Figure 7 Comparison between obtained control data and NN outputs of pilot B under crosswind. Based on the flight data analysis conducted, two significant points have to be noted: 1) roll oscillation and high yaw angle at touchdown for junior pilots under calm wind, 2) failed decrab control of junior pilots under crosswind. These points are discussed in more detail in this section. First of all, the roll oscillation for junior pilots under calm wind condition is considered. This is mainly happening before passing the threshold, which comes down to 11 12 s before touchdown. Fig. 8 shows the time histories of aileron and rudder control. This figure clearly shows that the junior pilots control aileron and rudder more aggressively. This can be attributed to the different sensitivity of aileron and rudder controls. Looking at the data acquired, one can easily see that the aileron and rudder controls vary greatly between the veteran and the junior pilots. Firstly, Fig. 9 shows the sensitivity of X (the runway lateral position) and (a characteristic of the projection; defined in Fig. 2) to aileron, in which the junior pilots show a bigger sensitivity. X and correspond to the yaw angle and yaw/sway, respectively. This means that the junior pilot control is sensitive to the yaw angle and sway. Another difference is also found in the sensitivity of dX/dt (the derivative of runway lateral position) and d/dt to rudder control, as shown in Fig. 10. Prior to 10 s, a big derivative gain of the yaw angle and sway is shown. Generally speaking, derivative control works to prevent the overshoot, but excessive gain should be avoided. After 10 s, the sensitivities gradually decrease and have almost the same values as those for the veteran pilots. This trend also agrees with the time histories of roll and yaw angle, i.e., firstly a big oscillation and sensitivity are observed, which gradually 6

Figure 9 The sensitivity of X and to aileron under calm wind.

that pilots A, B, and C control the lateral position with aileron mainly, while pilot D uses the rudder. This is considered to be the reason that it is difficult for pilot D to control the lateral position. According to the results obtained, pilot Ds lateral control strategy is very different from that of the others, as the pilot D shows the worst landing result. Here, it is worth noticing that pilot C also had a bad landing and another difference of the control strategy of pilot C can be found in more detailed analysis.

Figure 10 The sensitivity of dX/dt and d/dt to rudder under calm wind.

Figure 11 The lateral trajectories of the aircraft under crosswind. Next, some mistakes in the flare maneuver by junior pilots are considered. The trajectory of the aircraft in each case is shown in Fig. 11. The horizontal axis indicates the longitudinal position, and the vertical axis indicates the lateral position. The crosswind blows from minus to plus of y position. According to this figure, pilot Ds aircraft flies in the most downwind area. Generally speaking, it is easy to go leeward, but difficult to go windward. When pilot Ds aircraft flies in the downwind area, if the winglow control is applied, the aircraft will be likely to sweep away downwind. However, the aircraft cannot go further leeward anymore, so the decrab control cannot be applied. From this result, it is considered that pilot Ds sway control is a problem. Fig. 12 shows the sensitivity of (difference of right-and-left runway sideline angle) to aileron and rudder control. corresponds to the lateral position mainly. This figure clearly shows that a positive sensitivity to aileron and negative sensitivity to rudder are found for pilots A, B, and C, while it is the other way around for pilot D. This implies 7

Figure 12 The sensitivity of to aileron and rudder under crosswind.

lateral trajectory and distinguishing between important and minor information. Actually, junior pilot do try to conduct decrab control, but their attempt end up being unsuccessful. These results are shown to a captain pilot (neither A nor B), and interesting comments were obtained: The rudder is normally not used much during approach, because the lateral position cannot be maintained by rudder control. It is strange that junior pilots use rudder control so much. Moreover, especially under crosswind, it is more important to keep the lateral path than to stabilize the roll angle. It is considered that veteran pilots made roll oscillations under crosswind in order to maintain the lateral position, and that the junior pilots did not try to maintain the lateral position. Concluding, the proposed NN analysis agrees well with the pilot comments, and can pinpoint the problems in control strategy. It is expected that the analysis results can provide pilots with hints about how to control the aircraft more safely and effectively without relying merely on their flight experience. In the future, we would like to discuss the NN analysis results with the pilots who actually operated the simulator and confirm whether these comments help them to improve their control strategy.

Figure 13 The sensitivity of and d/dt to aileron under crosswind. Another interesting point is that the roll oscillation is bigger for veteran pilots under crosswind conditions. Fig. 13 shows the sensitivity of (inclination of the horizon) and d/dt (the derivative of inclination of the horizon) to aileron under crosswind condition. and d/dt correspond to the roll angle and roll rate, respectively. The sensitivity of is higher for junior pilots, while the roll oscillation is smaller for junior pilots. This sensitivity corresponds to the gain for roll angle, and low sensitivity causes a big oscillation of the roll angle. The sensitivity of d/dt shows a similar trend among all pilots. In all cases, a negative sensitivity is observed. This result implies that the roll rate information is not used effectively. According to the result, the roll control under crosswind condition is performed better by junior pilots. It is considered that an aircraft can be controlled better when all gains are positive. However, depending on the flight conditions such values might lead to excessive workload, which means that having all gains positive is not necessarily the optimal configuration. The main point is how to distinguish which parameter is relatively important. However, this characteristic applies to both veteran pilots, and it has some advantages under crosswind conditions. This may be a workload problem, that is high workload is inevitable under crosswind condition, and less attention to roll angle can decrease the workload. Another explanation would be that such a control makes it easy to adapt to an unexpected situation, such as a significant change of the wind. Summary of the results and pilot comments Based on the flight data and NN analysis results, several differences of control strategy depending on the flight experience are observed. Under calm wind condition, high sensitivity to aileron and rudder in the case of junior pilots are observed, which causes roll and yaw oscillation. In addition, the junior pilots tend to fail the decrab control. There are several possible explanations of this failure, e.g., the problem of controlling 8

4. CONCLUSIONS
The proposed neural network analysis can clarify a pilots information processing flow during a visual approach. In this paper, mainly the lateral control has been analyzed, while only the longitudinal control was a topic in the previous study. The most difficult lateral control is decrab control, so experiments were conducted under calm wind and crosswind conditions, and the results were compared and discussed. The examinees were two veteran pilots and two junior pilots, and the difference in control strategies between these groups with different levels of flight experience was also considered. It was revealed that the time histories of flight data varied widely between veterans and freshmen, and the neural network analysis could explain the difference in control strategy. In a future study, the basis for the veterans skilled control will be investigated, and it is hoped that it will improve the performance of an automatic landing system.

REFERENCES
[1] Suzuki, S., Sakamoto, Y., Sanematsu, Y., and Takahara, H., Analysis of Human Pilot Control Inputs using Neural Network, Journal of Aircraft, Vol. 43, No. 3, 793796, 2006. [2] Mori, R., Suzuki, S., Sakamoto, Y., and Takahara, H., Analysis of Visual Cues during Landing Phase by using Neural Network Modeling, Journal of Aircraft, Vol. 44, No. 4, 20062011, 2007. [3] Mori, R., Suzuki, S., Analysis of Pilot Maneuver under Crosswind Condition using Neural Network, Mathematical Problems in Engineering, Aerospace and Sciences, June 25-27, 2008. [4] Gibson, J. J., The Perception of the Visual World, Houghton Mifflin, Boston, MA, 1950. [5] Arbib, M. A., Brains, Machines and Mathematics, 2nd ed., Springer Verlag, New York, 1987. [6] Holland, J., Adaptation and Natural and Artificial System, Univ. of Michigan Press, Ann Arbor, MI, 1975. [7] Mori, R., Suzuki, S., and Takahara, H., Optimization of Neural Network Modeling for Human Landing Control Analysis, AIAA Infotech@Aerospace 2007 Conference and Exhibit, AIAA-2007-2840, May 7-10, 2007. [8] Mori, R., Neural Network Analysis of Pilot Maneuver During Landing Phase, 26th ICAS Congress, ICAS20085.4.4, September 14-19, 2008. [9] Garson, G. D., Interpreting neural network connection weights, Artificial Intelligence Expert 6, 4751, 1991. Shinji Suzuki was given the Ph.D. degree in Engineering from the University of Tokyo, in March 1986. He was a researcher at Toyota Central Research and Development Laboratories from 1976 to 1986 and became an Associate Professor of the Department of Aeronautics in the University of Tokyo in 1986. He has been a Professor of Department of Aeronautics and Astronautics since 1996.

BIOGRAPHY
Ryota Mori is currently a PhD student at the University of Tokyo, majoring in Aeronautics and Astronautics. He entered the above university straight after graduating from high school and since then has been involved in various projects at Suzuki and Tsuchiya laboratory to which he belongs. At present, he is the leader of the project focusing on analysis of pilot control meant to develop a tool to aid the training of new pilots.

10