Backpropagation neural networks with adaptive

differential evolution for time series forecasting

Lin Wang Yi Zeng Tao Chen
Paper Link: https://www.sciencedirect.com/science/article/pii/S0957417414004941

Presented by:
Charu Jain 2015B1A30825P
Shubham Jain 2016A7PS0043P
Abhiroop Bhattacharjee 2016A3PS0702P
Group ID: 44
Introduction
● One of the most widely employed time series analysis models is the autoregressive
integrated moving average (ARIMA), but it assumes linear time series.
● We propose BPNN with adaptive differential evolution for time series forecasting
which works for non linear data also.
● ADE is used to search for global initial connection weights and thresholds
(biases) of BPNN.
● The proposed ADE–BPNN is effective for improving forecasting accuracy
compared to basic BPNN,ARIMA and other hybrid models.
Contribution of each member in the project
● Charu Jain - Studying and searching of different papers for the assignment,
Handling the dataset, integrating ADE with BPNN, preparing the presentation

● Shubham Jain - Referred to various sources to understand DE algorithm, coding

the DE algorithm, making DE adaptive, readme file, preparing the presentation.

● Abhiroop Bhattacharjee - Studying and searching different papers for the

assignment, coding, training and testing the BPNN, preparing the presentation
Hypothesis and Methodology
● According to the BPNN theory, final training result depend on initial connection
and weights which were randomly initialized.

● Hence, the training result easily falls into the local minimum point rather than
into the global optimum; thus, the network cannot forecast precisely.

● To overcome this shortcoming, many researchers have proposed different

methods to optimize the initial connection weights and thresholds of traditional
BPNN.
Hypothesis and Methodology
● This study uses adaptive DE (ADE) to select appropriate initial connection weights and
thresholds for BPNN to improve its forecasting accuracy.

● DE has less complex genetic operations because of its simple mutation operation and
one-on-one competition survival strategy.

● DE can also use individual local information and population global information to search for
the optimal solution

● One real-life time series data set with nonlinear features is employed to compare the
forecasting performance of the proposed model with those of other forecasting models.
Implementation details: Function prototypes
def differential_evolution(fobj, bounds, mut, crossprob, popsize, gens, mode='rand/1')

def rmse(pred, real)

def eval_objective(params)

def my_init(shape , dtype=None)

def my_bias(shape , dtype=None)

def errorval(arr)
Flow Chart
 Equations For ADE
● Initialization: Xij = Umin + rand * (Umax-Umin)
● Mutation: Vij = Xr1 + F * (Xr2 - Xr3)
● F = Fmin + (Fmax-Fmin) * e.1-genM/genM-G+1

● Crossover: Uji = vji if Rj ≤ Cr or j = rn(i) ,

Xji if Rj > Cr and j != rn(i) (j denotes jth gene)

● Selection: Xi g+1 = UiG if f (UiG ) ≤ f(XiG), i = 1,....., Np

XiG otherwise
Xij =Population , Vij =Mutated Population, Uji=After crossover Population
Rj=random number between [0,1] , Cr= Crossover rate, rn(i)= gene index
Np=total population , F= scaling factor
Dataset
Table 1 : Monthly electric load data in NE China [1]

Training data: Dec 2004 to Oct 2008

Test Data: Nov 2008 to April 2009
Results
● No. of training epochs = 3000

Time point (Month) Actual ADE-BPNN DE-BPNN

Nov 2008 180.56 175.87 169.05

Dec 2008 189.03 182.48 184.07

Jan 2009 182.07 174.55 164.44

Feb 2009 167.35 159.49 167.08

March 2009 189.30 178.49 172.62

RMSE 7.75 12.21

● RMSE for ADE-BPNN is less than that of DE-BPNN at same no. of epochs
Conclusion
● The computational results show that the proposed ADE can effectively improve
the forecasting accuracy of BPNN compared with the basic BPNN model. It is due
to the appropriate determination of the initial parameters of the BPNN model.

● The proposed ADE–BPNN outperforms the DE-BPNN model in terms of RMSE

(as shown in the Results)

● From earlier studies, it has also been found that DE performs better than other
popular intelligent algorithms, such as GA and PSO, based on 34 widely used
benchmark functions (Vesterstrom & Thomsen, 2004) and so, we conclude that
ADE is the best algorithm in terms of improving prediction accuracy [2].
References
[1] Wang, J., Zhu, W., Zhang, W., & Sun, D. H. (2009). A trend fixed on firstly and
seasonal adjustment model combined with the e-SVR for short-term forecasting
of electricity demand. Energy Policy, 37(11), 4901–4909.

[2]Vesterstrom, J., & Thomsen, R. (2004). A comparative study of differential

evolution, particle swarm optimization, and evolutionary algorithms on numerical
benchmark problems. Proceedings of IEEE Congress on Evolutionary
Computation, 2, 1980–1987.

[3]https://www.youtube.com/watch?v=UZGiapWcoA4
THANK YOU