2004 IEEE International Conference on Electric Utility Deregulation, Restructuring and Power Technologies (DRPT2004) April 2004 Hong

Kong

Input Dimension Reduction for Load

Forecasting Based on Support Vector Machines
Xu Tao, Student Member, IEEE, He Renmu, Wang Peng, Member, IEEE and Xu Dongjie

Abslract--The traditional methods for load forecasting
can not supply the required accuracy for the engineering
application because we only get limited history data sets and
the factors that aflect the load forecasting are complex. This
paper presents a new framework for the power system short-
term load forecasting: firstly, this paper establishes the
feature selection model and uses floating search method to
find the feature subset: then this paper makes use of the
support vector machines to forecast the load and takes full
advantage of the SVM to solve the problem with smafl
sample and nonlinear. Hence the accuracy of the estimation
result is improved and a better generalization ability is
guaranteed. The EUNITE network is employed to
demonstrate the validity of the proposed approach.
Limiting the number of features can sometimes he helpful
because it cuts down the model capacity and thus reduces
the risk of over-fitting; second, this paper uses the feature
subset to build a load forecasting model based on support
vector machine.
11. LOAD FORECASTING FFL4MEWORK
This paper proposes a novel method for load
forecasting, shown in Figure 1. It is composed of data
preparation, feature subset selection and regression model
construction. We will describe it thoroughly in the
following section.

Index Terms-feature selection; load forecasting; floating

search; support vector machine.

I. INTRODUCTION Fe& rekdhn

I N ,a deregulated environment, the load forecasting & ,M
O &&&set
which provides mformation of future electric demand is
Loadfiorecastingmdel
an integrant part for transmission and distribution or
market planning. In recent years intelligent system
technologies have provided promising applications in load
R e d vdidrhon
forecasting. Examples include neural network, expert
systems and integrated techniques: These methods still Acovacy 4 assesmen.,

have many difficulties in achieving the perfect accuracy.
The load forecasting is a problem of multi-dimension
and nonlinearity. In a typical scenario, each data unit is
represented by many individual components. However, it
is not known a priori which of these components are
relevant for a given load forecasting problem. At the same
time, the observational data sets are obtained by finite
sampling, which only form a sparse distribution in the
load forecasting input space. In this instance, the
traditional neural network approaches will suffer from the
difficulty of generalization, and the resulted models often
overfit the data. This is a consequence of the optimization
algorithms used for parameter selection and the statistical
measures used to select the 'best' model ['I.
In order to solve the problems mentioned above, this
paper proposes a novel method of load forecasting. First,
we propose a strategy to rand individual components
according to their influence on the load forecasting.
Xu Tao is with the Depamnent of Electric Power Enginccnng, North
China Electric Power University. Bcijing 102206 China (e-mail:
xuwo@nccpubj.edu.cn).
Wang Pcng is with thc Depanment of Eleclric Power Engineenng,
North China Electric Power University, Bcijing 102206 China (e-mail:
wangpeng~nc~p:pubj.cdu.cn).
Fig. 1 Model design for load forecasting
111. INPUT DIMENSION REDUCTION
For SVM-based load forecasting, the task can be
regarded as a regression problem. Given an original data
set S of q points with input data x; ER" and
corresponding load y ; € R :
.., .1 (.,#,I
S = ( ~ f s l ) > .(xi#;)>.. (1)
In equation ( I ) , n is the dimension of load affecting
features x. By training the data set S, we can acquire the
SVM load model. In most practical situations, the original
input features x tend to over-represent the training data.
Therefore, we should remove the irrelevant or redundant
feature from x before training the SVM model.
This paper presents a feature selection method for load
forecasting model, shown in Figure 2. We split the data
set S into training set 1 and training set 2. Firstly, the
feature subset of d features is selected from the load
affecting features x with n dimensions. Secondly, we use
the given evaluation algorithm and the training set 1 with
d dimensions to obtain the evaluation model. Then the
evaluation model is assessed by the training set 2 with the
same d dimensions. The result can show the ability of

0-7803-8237-4/04/$ I7.00020041EEE
510
2004 IEEE lntemational Conference on Electric Utility Deregulation, Restructuring and Power Technologies (DRPT2004)April 2004 Hong Kong

feature subset to forecast the load. The search is going on theory (support vector ma~hine)'~].
until the stopping criterion is met. At last, an optimal Let's assume that the SVM data set S' is S of q points
feature subset is selected for use as i n m t variables for with d features subset that has been selected in section 111,
SVM-based load forecasting model. S ' = ( x; 3 , ) ,..., (XI 3;),..., (Xb +$
T m s e t1 Training set 2
where xz!ERd is the input data point. According to the
SVM theory, the importance of load forecasting is evident
that SVM is supposed to learn mapping function(f:
Feature Performance Rd+R)between input and output variables on the basis of
subset" estimation provided data set S. At the same time, fshould minimize
the risk,

Feature Estimation
(4)
where L is the loss function. As the distributionp(x,y) of
data set S is unknown, this paper adopts the e- Insensitive
loss function that enables a sparse set of support vectors
I to be obtained.
0 E- Insensitive loss function

/or If(.)bYI(& (5)

l / ( x ) - yl- E otherwise
Input feature subset for SVM-basedload In SVM-based electrical load forecasting, the
forecasting model regression model maps the input into a higher dimensional
Fig.2 Feature subset selection far load forccasdng feature space in which a linear regression model is
If we select a subset of d features from the given set constructed. Hence the regression ftmion of load
of n features directly, all the subset is C,d (if n=41, d = forecasting is as the following:
q
9 , then C.d =350343565). So an exhaustive search is
unpractical (as afforded by the Branch and Bound f ( x , )= (ai - a. k(xI, x, )+ b (6)
i=l
method). In order to enhance the search efficiency and
prevent the so-called "nesting effect" of feature subsets, where 4 Points 1 x: P J ( ~ = L . .are
. ~ the given training
this paper makes use of the floating search methods to data set S. The R~=JFkernel K(., .) meets the Mercer's
select feature subset [21' At the Same lime' the SVM condition. The Lagrange multipliersai, a; are given by,
algorithm is employed to build the evaluation model of
feature subset. The maximal feature number is regarded as q
the stopping criterion. According to the characteristic of W ( a , a' )= max a. (Yi - 6)- sib; + &)
CI.d n.a. i=l
load forecasting, this paper defines the estimation index
J(xd)of the evaluation model:
J(xd)=l - M P E
(2)
-iAA(a.
2
-ajIar
,=Ij = ,
-aj)K(xj,xi.) (7)
where the mean absolute percentage error(UAPE) is with constraints,
defined by the following equation
OSai,al.< C , i = l ; . . , q
MPE =-c1 r l ~ c t u a(/i )- Forecast (ill

T ,=I Actua/(i)
.loo% (3)
T ( a i-a.)= 0
(8)
where Tis the size of the training set 2, Actual ( i ) is the i=l
actual load and Forecast ( i ) is the forecasting- load. Solving
- Equation
. (7)
. . with constraints Equation (8)
.,
The feature subset replaces the original feature set determines the L~~~~~~~multipliers a I,,a.! ,
without significant loss in forecasting accuracy. The
If we put the data point x, into the Equation (6), we
generalization ability of the model even increases due to
the implicit regularization achieved by feature pruning. can obtain the forecasting load y=f(xm)conveniently. The
By use of the feature subset selection, the representative SVM-based load forecasting model is founded on the
features are served as the input variables for SVM structural risk minimization principle that equips the
model with a greater ability to generalize.
regression model.

IV. LOAD FORECASTING SVM MODEL v. SIMULATION RESULTS AND DlSCUSSlON

In order to avoid the shortcoming of the traditional A . Data description

neural network algorithm, this paper presents a new ~ i ~ data~ is ~collected
~ i by ~the World.Wide
~ l
model for the load forecasting based on statistical learning Competition the EJNITE network a period of

51 1
2004 IEEE Intemational Conference on Electric Utility Deregulation, Restructuring and Power Technologies (DRPT2004) April 2004 Hong Kong

2 years. The target is to forecast the daily maximum load day), Averagely maximal loads and minimal loads of the
in the Eastem Slovakia for the 31 days in January 1999. past week (2 attributes). avcrage tcmpcrahlrc af the

the past wcek (21 attributes)

Relationship curve
8 attributes indicate which day in a week (the past seven days
and forecasting day)
avcrage tempcraturc of the past week (1 attributc)
Averagcly maximal loads, minimal loads of the past week (2
57
attributes)
8 attributes indicate whether this is a holiday or not (the past
seven days and forecasting day)
average tcmpcrature of the forecasting day ( I attributc)
Information such as temperature and minimal loads in
January 1999 is unknown. So we train three SVMs, two
SVMs for predicting maximal loads and minimal loads
and the other for average temperature. The SVM for
predicting maximal loads is the same as the SVM for 0 10 20 30 40 50
predicting minimal loads. This paper train the temperature Feature number
SVM where each training data point has seven
temperature attributes of the past week as the input and Fig.3 Relationship between the feature number by floating scarch
the predicting temperature as the output141. mcthods and MAPE index
0 Output: the maximal load of the predicting day As is shown in the Table 3 and Figure 3, the feature
According to section Ill, this paper splits the subsets (3 and 4) express preferably forecasting ability.
historical data into two parts during the period of 1997- Comparing the difference between the feature subsets 3
1998. One part has 31 samples in number (in January and 4, we can see that there is one more attribute, average
1998, training set 2), another part has 692 samples in temperature of the forecasting day, in feature subset 4.
number (training set 1). This attribute value is known in training data but
unknown during the period of January 1999. All the
B. Training the SVM-based load forecasting model temperature value in January 1999 is predicted by this
We used MATLAB toolbox for experiments on paper. So it may bring on some forecasting error. Taking
training the SVM-based load forecasting model. the factor into account, we define a rule for feature
I ) Feature Evaluation selection: if several feature subsets are selected and have
According to section Ill, we firstly selected an optimal similar training error, we should choose the feature subset
feature subset for use as input variables for SVM-based which has more known attributes value. In term of this
load forecasting model. The result is as follows, shown in rule, we regard the feature subset 3 as the optimal result.
Table 2 . 2) Training SVM-based loudforecasting model
In order to validate the method proposed by this
paper further, we train the SVM load forecasting model
based different feature subsets. Predicting the maximal
Daily maximal loads of the past weck (7 attributes) , I
load in January 1999, we can draw the following result,
Fcaturc shown in Table 4. We only list 7 days result in Table 4
attribute indicatcs which day in a wcck (Ihc forecasting
subsct 2 1 day) because the page is limited.
Daily maximal loads of thc past week (7 attributcs) , I Tablc 4 Forecasting result compare using different fearure
attribute indicatcs which day in a wcck (thc forecasting
day), Averagely maximal loads of the past weck (i
attribute)
Daily maximal loads of the past wcek (7 attributes) , I
anributc indicatcs which day in a wcck (thc forecasting
day), Averagely maximal loads of the past week ( I
attribute)- avcragc IcmpcraNre of the forecasting day (1
attribute)
Daily maximal loads of thc p u t weck (7 attributes) , I
FeaNrc attribute indicates which day in a week (the forecasting

512
2004 IEEE International Conference on Electric Utility Deregulation, Restructuring and Power Technologies(DWT2004) April 2004 Hong Kong

1.61 755 I 749 I 750 I 746 I 758 I 674 VI. CONCLUSION

1.71 753 I 754 I 756 I 758 I 774 I 674
This paper introduces a new method for SVM-based
Calculating the W E index, shown in Table 5 ,
Table5 Comparison for MAPE indcx of different load forecasting. We propose a strategy to rand individual
components according to their influence on the load
forecasting, limiting the number of features can
sometimes be helpful because it cuts down the model
capacity. Hence this paper uses the feature subset to build
a load forecasting model based on support vector machine
The EUNITE network is employed to demonstrate the
validity of the proposed approach. Additionally, each
we can see that the feature subset 3 realizes a better system has its characteristic and representative features
generalization. So the rule proposed by this paper is differ in many ways. It requires us to provide abundant
feasible. data for power system load forecasting.
C. Result comparison and discussion
The result of this paper is compared with EUNITE VII. REFERENCES
competition winner's paper, shown in Table 6, Table 7 S . R. G u m '"Support Vector Machincs for Classification and
Regression," Tcchnical Report, Image Speech and lntelligcnl
and Figure 4. Systems Research Group, University of Southampton, 1997
Fig.4 forecasting result camparc between this paper and the best paper
P. Pudil, J. Novavicova, and J. Kinler. "Floating Search Methods
in EUNITE compctitian
in Feature Selection, " Ponem Recognilion Lerrers, vol. 15, pp.
1,119- 1,125,Nav. 1994.
N.Vapnik. " The nature of slatistical learning theory". New York
Springcr,l995
World-wide competition within the EUNITE nelwark, EUNITE
competition Report, Company behind East-Slovakia Power
Distribution Company Available: http://neuron. tuke. Sk/
c o m p e t i t i o d i n d e x . php
David Esp. "Adaptive Logic Nctwarks for East Slavakian
Electrical Load Forecasting," EUNITE compctition Rcpon,
Company behind: East-Slovakia Pawcr Distribution Company.

VIII. BIOGRAPHIES

Xu Tao was bom in China, in 1976. He

._.___.-.____,.___I .dm . -~ . -L graduate from ShanDong University of
Table 6 forccasting result comparison between this paper and the best Technology in 1998, and reccivcd his master
degree from North China Elcctrie University in
2000. Hc is now working for his Ph.D. degree
in North China Electnc Power University in
Beijing. His intcresls are in the area of pawcr
system analysis, data mining application etc.

He Renmu was bom in China, in 1944. Shc

graduated from power system & Automation
Department of TsingHua University in 1967.
She obtained Ph. D. degree from Lausannc
InstiNte of Science and Technology (EPFL) in
1984. Now she is the professor of North China
Elcctric Power University in Bcijing. Her
interests are power system dynamics, simulation
veracity, dynamic load modeling, deregulation,
soAwarc enginccring, wide area meaurcmcnt
CtC.

Tablc 7 MAPE indcx comparison belween this paper and the best paper .. .- -.
in EUNITE competition Wsng Peng was bom in Henan Pravincc,
I Method I MAPE indcx I P.R.China on November 24, 1973. He received
M a ~ t e rdegrcc and Ph.D degree from Nonh
China Elcctric Power Univcrsity (NCEPU) in
1997 and 2002. Now he is assistant chief of
It can be seen from the table that the method Power System Control lnStiNtC of NCEPU. His
special fields of i n t c ~ included
~~t elccuic powcr
proposed by this paper improves the estimation accuracy system analysis and control, electricity market,
and generalization ability of load forecasting. The risk of dynamic stability and etc.
over-fitting is reduced.

513
2004 IEEE Intemational Conference on Electric Utility Deregulation, Resttucturing and Power Technologies (DRPT2004) April 2004 Hong Kong

Xu Dongjie was bom in China, in 1974. He

graduated from Electrical Enginccnng
Lkpartment of North China Elcctrie Power
University in 1996. He obtained M.Sc
degree from (NCEPU) in 2001. Now He is
working towards his Ph. D.degrcc. His
research interests me power system
dynamics, small signal stability analysis and
control.

514