Optimal Artificial Neural Network Topology for Foreign Exchange Forecasting

Computer Science Department Western Kentucky University - Bowling Green, KY, 42101- (270)745-5011

Dr. Ahmed Emam

Ahmed.emam@wku.ed, ABSTRACT
Foreign exchange market is one of the highest investments markets the average daily trade volume is 1.8 trillion USD. Foreign exchange rate forecasting has been always one of the most challenging subject and area of researches. Trader around the world is relying on the technical indicators which just following the price and has emerged a lag results. When the currency market has a random move (when the market is not trending) most of the indicators gets confused because of the fact that classical linear methods are unable to react with the non linearity in the data and hence with the market behavior. This research reports empirical results that tend to confirm the applicability of a neural network model to the prediction of the foreign exchange rates market. Artificial neural networks have proven to be efficient and profitable in forecasting financial time series in particular, feed forwarded back propagation. It is important to use an optimal ANN topology that emerged great results in short term prediction and the daily predication results showed that ANN model learns well and most likely to generalize well. Weekly predication results demonstrate good results in the low prediction while failed to have a good results on the high and the close prediction while the monthly prediction did not give a satisfactory results due to a very few data samples. dataset. ANN have two great futures first they have a universal mathematically function approximation capability which can solve many complex relationships problem which is very important feature in find relationship between the variable to be predicted and the other input variables. In addition, the non linear structure of neural networks is capable of dealing with the most complex problems that is why ANN become a very promising tool for market predictions. Second feature of ANN is a data driven and it does not need restrictive assumptions on the process from which the data are generated and this feature makes it is easy to learn from the available data and will be highly recommended tool in a situations where a large data is available. The financial time series specially the FOReign EXchange market (FOREX) is characterized by a nonlinear and non stationary behavior and as it has been suggested in may researches before that most of the financial markets are not predictable and it follows a random walk hypothesis. In the past, some researches states that time series follows a Markov model which is a stochastic process i.e. the probability of a transition from one state to another state depends only on the current state which means in this case that the prediction of tomorrow rates requires today rates and not the past rates. This research aims to present and evaluate the Artificial Neural Network (ANN) performance as a forecasting tool on predicting financial time series specially FOREX. Therefore, the main objective of the study is to implement and develop an ANN model capable of doing a real market simulation.

Categories and Subject Descriptors

I.6.5 Model Development.

Keywords
Foreign Exchange, Artificial Neural Networks, Optimal topology.

2. Commercial ANN Forecasting Packages

There are many ANN commercial packages exist in the market to predict the FOREX market similar to the objective task of this study and they proven a very high success in terms of profit. The famous and most usage ANN commercial packages for predicting the FOREX are Trading Solutions (www.tradingsolutions.com), NeuroShell Trader (www.neuroshell.com/geewhiz.asp), Neural FX forecasts (www.neuralinvestmentforecast.com), and Neuro dimension (www.nd.com). The entire pervious package used financial dataset to forecast the FOREX currency markets with a higher return but non of them used function approximation feature of ANN. Since forecast the relationship between multiple factors in financial data is a type of function approximation problem, ANN a several solution for function approximation problem, stating from using Multi-Layer Perceptron (MLP), Radial Basis Function (RBF) and CANFIS (Co-Active Neuro-Fuzzy Inference System) type networks.

1. Introduction
Forecasting the financial market have been a challenge for many decades. For many years researcher and analyst has used linear techniques since it was very simple and easy to be applied by users however these linear indicators has always worked well for linear moves but it stood helpless when dealing with nonlinear market behavior. Artificial neural networks (ANN) provide very promising results for predictions, especially for time series
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ACM-SE 08, March 2829, 2008, Auburn, AL, USA. Copyright 2008 ACM ISBN 978-1-60558-105-7/08/03$5.00.

63

3. Foreign Exchange Market Background

Prediction of financial markets has become one of the most important topics not only among economists but among traders in all over the world this because of the evolutionary of the internet. Foreign exchange market is an international foreign exchange market, where money is sold and bought freely and it was launched in the 1970s. The word FOREX is derived from Foreign Exchange and it is the largest financial market in the world and has an estimated $1.8 Trillion in turnover every day. There is no single centralized place for the trade of FOREX. Traders buy and sell FOREX via telephones and computers linked to brokers, bank and other traders around the world. Currency is traded in pairs and each currency has assigned a symbol. For example the Japanese Yen it is JPY, for the Pound Sterling it is GBP, for Euro it is EUR and for the Swiss Frank it is CHF. There are four terminology related currency marker, which are high and low to indicate the highest traded price and the lowest traded price and open and close to indicate the price recorded during the period designated by the exchange as the official opening or closing. A rise in the rate between a currency pair (EUR/USD) means that the first named currency has strengthened against the second named currency and that is why when someone is selling a currency rate he is selling the firstnamed currency and buying the second-named currency. FOREX exchange market associated with some terms such as Spread (currency quote received from a dealer), Lot size (the size of the currency unit to be traded and each lot is worth 100,000 units of the base currency), and Pips (the smallest unit in which a currency price is normally quoted or the last decimal place of a currency). Two types of analysis are used for the market movements forecasting Fundamental and technical dataset. The technical analysis is the chart study of past behavior of the prices while the fundamental one focuses on the theoretical models of exchange rate determination and on the major economic factors and their likelihood of affecting the foreign exchange rates [5]. Technical analysis is appointed to analyze market movement and using technical indicators. Indicators are one of the most important tools to analyze the past behavior of the market in order to try to predict the futures such as Moving Averages( the most commonly used time frames for moving averages are 10, 20, 50,100 and 200 periods on a daily chart), Slow Stochastic (A 14-day %K (14-period Stochastic Oscillator) would use the most common close), Momentum (oscillator that measures the rate at which prices are changing over the Observation Period[38]), Moving Average Convergence Divergence (MACD), Rate of Change (ROC) is another version of the momentum oscillator.

intermediate trend lasts for three to four months maybe less, the minor trend is the short term may last for hours and not more than 1 or 2 days. Most of analyst tends to use averages to express the long, intermediate and short terms. According to Yao study [6], this works well for some currencies and certain periods. Moving averages is a well-known method used to identify the trend and smoothing its noise. An input model refers to the input data used as input for the ANN, as well as the target used for comparing the prediction against the target For example, using purely delayed daily averages and predicting the value one day ahead constitutes an input model. A neural network model was applied by Yao [7] also to forecast the composite index of stock exchange. He build the ANN forecasting model the input data of 7 years. Yao preformed many experiments and tests and he found that 5-4-1, 5-3-2-1, and 6-3-1 have proven success of 85% comparing the predicted to the targets and the smallest NSME with 0.033. Combining several models to improve forecasting accuracy has been widely studied in the traditional forecasting. The idea of model combination lies in the basic assumption that the true underlying structure of data is difficult or impossible to model by one model exactly and different models may play a complementary role in capturing different patterns in the data. In the first M-competition [8] which was neural network models for time series forecast, most conventional forecasting models are tested using more than 1000 real time series and the combination of forecasts from more than one model often leads to improved forecasting performance. Much research has been done in forecasting combination in the last two decades. Most of the research, however, focused on the linear model combination. Recently, several studies have proposed to combine multiple neural networks or combine neural networks with statistical methods. Ginzburg [9] showed good results with multiple neural network models. Wedding and CIO) propose a hybrid model of radial basis function (RBF) networks and ARMA models. Luxhoj et al. present a Methodology that combines ANN models with econometric models to better forecast sales. Donaldson and Kamstra and Shi et al. suggest different approaches for forecasting by combining ANNs. Khotanzad et al. propose a two-stage approach to combine several neural networks for natural gas consumption prediction. Medeiros and Veiga use a hybrid linear-neural model for time series forecasting. Zhang and Berardi develop two ANN-based combining methods for time series forecasting.

3.1 Time Series Forecasting Pervious Work

Many attempts has been made to forecast the stock index ,FOREX rates and indices in general this section demonstrates some of the pervious work done to predict financial time series which will guide this research. On the FOREX market the trend which identify the direction of the market can be up or down trend for hours, years, weeks, and maybe years that is why the trend can be classified to be a long term trend (major trend), the intermediate trend and the short term trend. The long term trend expected to lasts for several months maybe a year, the

series is a sequence of data points, measured typically at successive times, spaced at (often uniform) time intervals. Time series prediction is the use of a model to predict future events based on known past events, to predict future data points before they are measured. The standard example is the opening price of a share of stock based on its past performance. Models for time series data can have many forms. Three broad classes of practical importance are the autoregressive (AR) models, the integrated (I) models, and the moving average (MA) models. These three classes depend linearly on previous data points [10].

4. Time series Prediction using ANN In statistics and signal processing, a time

64

Time-series forecasting is about the forecasting method that uses a set of historical values to predict an outcome. These historic values, often referred to as a "time series", are spaced equally over time. Time-series forecasting assumes that a time series is a combination of a pattern and some random error and the main goal is to separate the pattern from the error by understanding the pattern's trend, its long-term increase or decrease, and its seasonality, the change caused by seasonal factors such as fluctuations in supply and demand. There are many tools for investigating time-series data include: consideration of the autocorrelation function and the spectral density function, performing a Fourier transform to investigate the series in the frequency domain, use of a filter to remove unwanted noise, principal components analysis (or empirical orthogonal function analysis), and artificial neural networks. Neural network learning methods provide a robust approach to approximating real-valued, discrete-valued and vector-valued target functions. For certain types of problems, such as learning to interpret complex real-world sensor data, artificial neural networks are among the most learning methods currently known [11]. Artificial neural networks have emerged as an important quantitative modeling tool for business forecasting. Artificial Neural network has the ability to predict currency exchange market up and down moves. The potential advantages of ANN over other statistical techniques is that there are universal function approximates and, being nonlinear, making them excellent choice for detecting nonlinear ties, but suffer from long training time and a very high number of alternatives such as architectures and parameters. Many attempts has been made by the satiations to crack the codes of the financial markets especially the FOREX market unfortunately none of them has made a good percentage of success The linear models assumes that the market will move in a linear direction whether trending up or down because of the fact that the market is trending most of the time up or down and the trend may continues for several hours, days, weeks and maybe years in some cases. It is not easy to implement a successful neural network model for predicting the FOREX market because of the various factors influence such as political events generally take place over a period of time, but political crises strike suddenly and prices dry out quickly, and sometimes the spreads between bid and offer jump from 5 pips to 100 pips [5]. Other factors such as financial factors are vital to fundamental analysis and using the interest rates independently from the real economic environment translated into a very expensive strategy [12].

model can learn by example. That is why the linearity of the trend should be removed. The de trending process can be achieved by calculating the difference between 2 sequence vales, for example this day moving average and the day before the formula is give by [14]

y = x t x t 1

Where

Is the current day and

t 1 Is the pervious day. The last step in preprocessing stage is transforming the dataset to keep the consistency and distribution. The logarithmic transformation is useful for data which can take on both small and large values. Another known transformation is using a ratio of the input variables.

4.2 Performance of ANN Forecasting System

Evaluation the performance of the forecasting ANN is very important for future modifications or updates. Testing performance involves measuring the difference between the output predicted values from the NN forecasting system and the target value. In this work , Mean square Error (MSE) as defined by [6] was used as error measurement. Performance measurements can be considered the inverse of the corresponding error measurements, one of the most common performances metric is the correlation coefficient (CORR) it has a range from 0 to 1 and the higher correlation coefficient value the better performance as mentioned by [6]. Profit measurements can express the profits in case of up move the strategy will be buy and hold until having the profit and sell and hold until having the profit in case of down move this can be expressed by the

Re turn =

following formulas Where K is either the selling or buying time steps in case of up or down move and profit can be measured in relation to the number of trading days [15]

X t +k X t * 100 Xt

profit = (moneyobtai / seedmnoey) 365 / D 1 ned

Where D is the number of the trading day of the test period. Another metric used to measure the performance of forecasting is average pipe deviation, which is shows the predicted value from the target values in pipes and can be expressed by Average (Absolute (actual-predicted)*10000) (Absolute (actual-predicted)*100) Or Average

4.1 Data Collection and Preprocessing

Historical data are available on the web for download and in the purpose of the experiment a historical data were downloaded from [7] in the form of high ,low ,close ,open and volume in a comma separated files for the daily ,weekly ,monthly data. Before data are fed to the ANN model the data has to be smoothed to eliminate the noise and to be scaled in the boundaries of the activation function. To avoid this problems, all values can be normalized to the mean using the formula several
Where Is the standard deviation, and x is our input data. Another preprocessing activity take place was de trending to remove the trending from data since the trend may lasts from several days to several months so the neural network
y = x mean

Multiplying by 10000 if the currency rate has 4 digit decimal places or multiplying by 100 if the currency rate has 2 digit decimal places.

5. Experimental setup and Results

In this paper the four major currencies will be used ; the historical data for the USD/JPY, GBP/USD, EUR/USD and USD/CHF are downloaded from the FOREX site [5]. The daily /weekly/monthly data are downloaded in comma separator delimited files the data are the open, high, low, volume and close. In addition , the close, high, low of the next day/week/month, where the close prices will be used to build up the indicators to prepare inputs to the ANN model which will forecast the next day close. A special excel program has developed to import the sample data called excel currency calculator which will prepare

65

the data and clean it then the technical indicators are to be calculated and normalized performing the inputs to be suitable for the ANN model. All normalization done to scale the data from 0 to1 because the activation function used is the sigmoid. The whole process is described on the following figure (1): Several experiments have been conducted to achieve the best topology, where the dataset is split by 2/3 for the training and 1/3 for testing. Table 1 demonstrates the influence of the number of neurons on the MSE and the pipes deviation in the training and testing.

No Hidden layers 2 3 4 5

of

Training MSE 30E-5 29E-5 31E-5 27 E-5 Pipes 40 42 44 40

Testing MSE 16E-5 16E-5 15E-5 15 E-5 Pipes 42 42 41 42

Table 2 : Influence of the number of neurons in each layers

Download Dataset in CSV f Preprocessing and cleaning the D Calculate the technical indictors as input for ANN and Divide the dataset into training(0.66) and testing Run the matlab using training d Evaluate of the training and testing accuracy using MSE, The process is repeated for d il hi h d d il l
No of neurons 2 4 6 12 Training MSE 3E-4 3.1E-5 2.9E-5 1.7E-5 Pipes 46 41 42 39

It was important to choose what indicators to use and how many of them can be used because using a few indicators will not give as much information about the currency under investigation, also using many indicators can generate much noise on the ANN forecaster. For that purpose many experiments has been done until the best results achieved in term of minimizing errors and the pipes differences between the predicted and the forecasted. Table 3 shows the influence of using 2, 4, 5,6and 7 indicators as an input and it was clear that using 6 inputs produces better results in the training and testing. Inputs Training MSE 2 4 5 6 7 3.1E-5 3.1E-5 3E-4 3E-4 2E-5 Pipes 43 43 44 39 45 Testing MSE 1.5E-5 1.5E-5 1.5E-5 1.3E-5 1.1E-5 Pipes 39 39 39 34 36

Table 3: Influence of the number of the inputs * 2 inputs are (current high, MA50), 4 inputs are (current high, MA20, Ma50, Lag high), 5 inputs are (current high, MA10, MA20, Ma50, Lag high), 6 inputs are 5 inputs are (current high, MA5, MA10, MA20, Ma50, Lag high), 7 inputs are (5 inputs are (current high, MA10, MA20, Ma50, Lag high, RSI)

Figure 1: Forecasting processing diagram Testing MSE 3E-4 1.6E-5 1.8E-5 1.7E-5 Pipes 47 39 43 85

Table 1: Influence of the number of neurons in each layer Increasing the number of hidden layers did not improve the results but on other hand it may lead to over fitting, 2 hidden layers are fair enough for the ANN prediction model. Table 2, shows that neither the training nor the testing have been improved after increasing the number of the hidden layers. Figure 2: ANN topology design

66

The best suggested topology is supervised Feed forward Neural Networks (FFNN), with error correction learning rule, based on the Multilayer Perceptron (MLP) architecture as shown in figure.
USD/JPY daily prediction from 20/8/2006 to 20/9/2006
118.50 118.00 117.50 C lo s e vlau e s 117.00 116.50 116.00 115.50 115.00 114.50 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 Days Actual Predicted

Feed forward neural network with back propagation learning algorithm are efficient method for predicting financial time series and since the currency market is very noisy market with a very high risk, the suggested ANN models will play a very important role in currency predication market. In the future, several experiments will be conducted for different period of times such as weekly and month and other currencies not only Japanese currency.

REFERENCES
[1] Zahedi, F., Intelligence Systems for Business, Expert Systems with Neural Networks, wodsworth Publishing Inc., 1993. [2] Jacek M. Zurada, Introduction to Artificial Neural Systems, PWS Publishing Company, 1992, ISBN:534-95460-X [3] S. Kang, An investigation of the use of feed forward neural network for forecasting, PhD Thesis, Kent State University, Kent, USA, (1991). [4] Phua, P.K.H. Ming, D., Lin, , Neural Network with Genetic Algorithms For Stocks Prediction, Fifth Conference of the Association of Asian-Pacific Operations Research Societies, July, Singapore, 2000, [5] "Forex information www.webtrading.com/downloads/forexmanual.pdf [6] "Trading forex site". forex.com/en/Forex.FinancialCalendar.aspx site".

Figure 3: USD/JPY daily high predictions Figures 3 shows the results of the tests on the daily close, low and high for a month period. The results of the testing (the predicted values against the actual values) shows that the MSE error achieved during the test is very near to the MSE achieved during the simulation which means that the suggested ANN model learns well and most likely to generalize well.

http://www.easy-

6. Conclusions and Future Work

Developing an ANN model for any forecasting problem is not an easy task. There are many ways to build an ANN model and a large number of choices have to be made during the model building and selection process. This research showed that ANN is a data driven techniques, developing a very good model without a good data preparation, input selections, variable selection, training algorithms and a predefined model for validation this model will not evolves a satisfactory results. The experiment started using 2 inputs with 2 hidden layers to test and based on evaluation of the results in terms of errors and pipes deviations, the network topology changed with 4, 5 and 6 inputs until best results is achieved. The hidden layers in the experiments was increased from 2 to 3 and then to 4 hidden layers until best results achieved and the optimum numbers of hidden layers was three hidden layers. Also it is important to use an optimal number of input units, because the more inputs to be used the higher the noise. All input does not contain transfer functions. And concerning the output it will be one output which will be the predicted or the result forecasted value whether our study going to forecast the close, high or low. This work showed that the ANN forecasting model performance is depends mainly on the quality of the input data and while the forecasting system has evolved a very good results on some currencies it failed to give these good results on other currencies because of the input data quality. This work has emerged great results in short term prediction, the daily prediction, while the weekly prediction has given good results for some currencies.

[7] Yao, J. & Tan, C. L, A case study on using neural networks to perform technical forecasting of forex. Neurocomputing, 31, 8199, 2000 [8] Makridakis, S.,Anderson, A.,Carbone, R.,Fildes, R.,Hibdon, M.,Lewandowski, R.,Newton, J.,Parzen, E., & Winkler, R., The accuracy of extrapolation (time series) methods: Results of a forecasting competition, Journal of Forecasting, 1, 111153. (1982). [9] Ginzburg, I. & Horn, D., Combined neural networks for time series analysis. Advances in Neural Information Processing Systems, pp 224231, 1994. [10] "Wikipedia, free http://en.wikipedia.org/wiki/Time_series . encyclopedia".

[11]Esfandiar Maasoumi, Entropy and predictability of stock market returns, Journal of Econometrics , pp. 280-330, 2002 [12] "Forex information http://www.webtrading.com/downloads/forexmanual.pdf site".

[13] "Math Lab help". http://ww.mathworks.com/access/helpdesk/help/toolbox/nnet/nnet .html [14] Adya, M. & Collopy, F., How effective are neural networks at forecasting and prediction? A review and evaluation, Journal of Forecasting, 17, 481495. (1998). [15] "Trading forex site". forex.com/en/Forex.FinancialCalendar.aspx http://www.easy-

[16] Wedding, D. K. & Cios, K. J., Time series forecasting by combining RBF networks, certainty factors, and the Box-Jenkins model. Neuro computing, 10, 149168, 1996.

67

[17] James D Thomas, News and Trading Rules ,PhD Thesis, Graduate School of Industrial Carnegie Mellon University, January 2003. [18] Kaastra and M. Boyd, Designing a neural network for forecasting financial and economic time series. Neuro computing , April 26, 2002.

68