304 Journal of International Council on Electrical Engineering Vol. 1, No. 3, pp.

304~308, 2011

Proposal of Reliable and Real-Time Communication Method for SCADA on IP-Based Networks for Next Generation Electric Power Systems
Hiroyuki Yusa , Mai Kiuchi * and Eiji Ohba *
Abstract In this paper, we propose a communication method with IP-based wide area networks for SCADA systems. This method controls communications between substations and a control center and can be implemented as a communication middleware. The method supports functions for UDP-based transmission, route control of double route transmission, retransmission control and priority control. The functions work cooperatively to assure reliable and real-time communications. We also examined the effectiveness of the method under circumstances where there are not only transmission delay and route breakdown in network but also information congestion in computer. The results show that the proposed method is more effective for assuring real-time and resilient feature than TCP-based communication method. Keywords: SCADA, Middleware, Reliability, Quality of service, Real-time, Internet protocol (IP)

1. Introduction
Japanese electric companies utilize information and communication technologies to reduce blackout time and support high quality electric power supply. Supervisory control and data acquisition (SCADA) systems play important roles for these by utilizing Internet Protocol (IP) technology and software technology. It is expected that even higher quality electricity will be required, and distributed generation systems such as photovoltaic (PV) generation systems will be introduced massively. Substations and control centers should become more sophisticated to accommodate these situations. A communication method is required to support the sophistication. The communication method should be suitable for applications utilizing object oriented technology [1] and transmit various types of information from many substations to a control center assuring reliability and real-time features via IP-based networks. In this paper, we propose a communication method with IPbased wide area networks for SCADA systems. The results of evaluation of the method are also shown.

2.1 SCADA Application A SCADA system is composed of applications for monitoring, control, measuring, recording and engineering support. Communication requirements for reliability and real-time features differ according to applications. In this paper, we define reliability and real-time features as described below. A) Real-time feature completes information transmission within the specified time. B) Reliability feature ensures resilience to network faults. The monitoring application is basic and important to utilize information for status changes as event or alarm of electric apparatus. The communications for the information of status changes require higher real-time and reliable features than other communications. Therefore a communication method for SCADA systems is required to enable transmitting a large amount of information from substations to a control center. 2.2 Off-the-shelf Communication Middleware SCADA systems can be classified as a distributed processing system. The fundamental behavior of general distributed systems is to pass messages among application programs. A message passing method with TCP/IP is required to support 1) converting messages from and to communication data and 2) managing sockets for transport

2. Applications and Communication Middleware

for SCADA System

Corresponding Author: System Engineering Research Laboratory, Central Research Institute of Electric Power Industry, J a p a n (yusa@criepi.denken.or.jp) * System Engineering Research Laboratory, Central Research Institute of Electric Power Industry, Japan (mai@criepi.denken.or.jp, ohba@criepi.denken.or.jp) Received: January 27, 2011; Accepted: June 14, 2011

Hiroyuki Yusa , Mai Kiuchi and Eiji Ohba

305

protocol. These make the design and implementation of application programs complicated. There are communication middleware, which cover message-passing method as shown in Fig. 1 and include Java remote method invocation (RMI) [2] and common object request broker architecture (CORBA)[3].
Control Center Computer Control Center Application Program Communication Middleware IP-based Network Substation Computer Communication Middleware
Message message

Possibility of Information Congestion

Wide Area Network

Influence of Transmission Delay Possibility of Route Breakdown

Substation

e.g. Status Change Events or Alarms

Application Program Control Bay Unit Power System Fault Causes Status Change of Electric Apparatus

Fig. 1. SCADA systems with communication middleware. 2.3 Communication Requirements for SCADA systems The middleware for CORBA and Java RMI often utilize TCP as transport protocol. Delay and fault in IP network, including breakdown and congestion of transmission route, affect real-time feature for communications with TCP. But communications for SCADA systems are required to assure real-time feature. SCADA systems should handle information on status change event or alarms of electric apparatus, and the amounts of the information vary according to the situation of power system fault. Therefore serious faults of power system cause transmission of a lot of messages from many substations to a control center. The systems are required to handle messages adequately considering the priority of the information. The systems are also required to adapt the breakdown of transmission route and the loss of communication data for assuring reliability.

communication middleware. The method involves the specification of combination and its ordering of functions for UDP-based transmission, route control of redundant or double route transmission, retransmission control and priority control, as described in Fig. 2. The specification is determined by the result of analysis and experiment of these functions. This method features the following characteristics. A sender application creates messages with data objects and requests the middleware to transmit the messages. In a receiver computer, the middleware invokes the destined application and passes the messages. The UDP-based transmission utilizes the bandwidth of IP networks and enables real-time data transfer. The route control function of redundant or double route transmission adapts to any breakdowns of the transmission route. The retransmission function recovers the loss of data transmitted with UDP in IP networks. The priority control function handles messages adequately considering the priority of the information on the situation where a lot of messages burst from many substations into a control center. The middleware converting messages from and to communication data and manages sockets to cooperate with transport protocol and transmit messages via an IP-based network.
Invocation of Program Priority Control Control Center Computer Control Center Application Program RetransmissionReceiver Route ControlDiscard Communication Middleware
Cooperation w ith Transport Protocol Request of Retransmission

TCP

Suitable Function Order

UDP

UDP

Router Wide Area Network Router

Router Router

Applying UDP Faster than TCP Prioritized Transmission of Messages

Data

Data

Substation Computer Communication Middleware Substation Application Program Bay Control Unit
Cooperation w ith Transport Protocol

Route ControlDuplicate RetransmissionSender Priority Control Reception for Sending

Fig. 2. Proposed communication method. The proposed method utilizes UDP-based transmission and pacing message forwarding interval. The UDP-based transmission assures the real-time communications. The pacing is executed in priority control functions and reduces influences of bottlenecks through cooperation with transport protocols and applications to improve the realtime feature of the communications. This method also enables double-redundant routes and retransmission for UDP-based transmission to assure reliability. The method

3. Communication method for SCADA systems

In this chapter, we propose a communication method with IP-based wide area networks for SCADA systems. This method controls communications between substations and a control center and can be implemented as a

306

Proposal of Reliable and Real-Time Communication Method for SCADA on IP-Based Networks for Next Generation ~

enables each function to work cooperatively with other function and provide real-time and resilient communications superior to TCP-based communications.
a) Adding Transmission Delay b) Breakdown of Route and Link c) Information Congestion in Computer
Communication Middleware

4.1.2 Adaptation function middleware The adaptation function middleware simplifies assuring real-time and reliable feature to application programs. As shown in Fig.4, the middleware supports UDP and TCP as transport protocol and functions shown below. a) b) c) d) Communication interfaces as API Quality of Service (QoS) ensuring functions Reliability ensuring functions Cooperation function with lower protocol

A-side Network

Communication Middleware

Substation Computer Switch

Communication Middleware

Control Center Computer

Substation Computer

Switch

Communication Middleware

Substation Computer

The middleware supports customizing the combination of functions and its ordering to adapt to the requirements of application programs.
Application Program

B-side Network

Fig. 3. Configuration of experiment system.

Communication Interface Reception for Sending Invocation of Program

4. Evaluation and results

4.1 Experiment system 4.1.1 System configuration We built an experiment system to evaluate the performance of the proposed method as shown in Fig. 3. The system is composed of a small IP network and computers. The network consists of two-gigabit Ethernet switches (Cisco Catalyst 3550) and provides doubleredundant routes between computers. Six-substation computers and a control center computer are connected to the IP network. The computers have the specification described below. CPUIntel Xeon processor E3110 for four computers (3.0GHz, 6MB L2, 1333MHz FSB) and E3065 for three computers (2.33GHz, 4MB L2, 1333MHz FSB) Memory2GB Network interface10/100/1000Base-TX* 3 cards We developed the adaptation function middleware [4][5], which support various information collection and delivery for next generation power system communications. The middleware is installed into all computers. The middleware and its application program for the evaluation are built with Sun Java. Linux is used as OS. The details about the software are shown below. Java: SUN Java Standard Edition 6.0 OS: openSUSE Linux 11.1 (Kernel 2.6)

Adaptation Function Middleware (Upper Protocol)

Ensuring QoS Priority Control

Flow Control Information Sharing

Ensuring Reliability Route Control

Multiple Transmission

Retransmission

Cooperation with Lower Protocol

Lower Protocol

Transport Protocol Network Protocol

UDP IP

TCP

Fig. 4. Adaptation function middleware. 4.2 Influence of transmission delay We evaluated communications with the proposed method and a TCP-based method for messages burst from a substation computer into the control center computer supposed that there are faults of power system. Table 1 shows the configurations of the communications. The pacing interval of priority control functions in sender and receiver are both 5ms per-message. The substation computer transmitted 25, 50, 75, 100 messages respectively. The result is shown in Fig.5, where 10ms transmission delay is added. The communications with proposed method transmit messages in constant time, and faster than the communication with the TCP-based method when there are less than 50 messages. This means the proposed method enables real-time transmission and utilizes the transmission bandwidth effectively.

Hiroyuki Yusa , Mai Kiuchi and Eiji Ohba

307

4.3 Influence of transmission route breakdown Table 1. Configuration for experiments

Case Size of application data in a message [bytes] Number of transmitted messages per-computer Number of substation computer Number of control center computer Transmission delay, 10ms 5000 100 1 1 Breakdown of route 5000 10 1 1 Information congestion 5000 25, 50, 75, 100 6 (High 1, Low 5) 1

computer to compare the communication with the proposed method and the TCP-based method. Table 1 shows the configurations of the communications. The communication with proposed method worked stably. But the communication method with the TCP-based method missed transmission. 4.4 Influence of congestion in computer We examined the effectiveness of the proposed method on assuring real-time feature for assuming event or alarm messages bursting from many substations. Table 1 shows the configurations of the communications. The result is shown in Fig. 7. The end-to-end total transmission time for a high-prioritized communication is faster than lowprioritized communications from four substation computers. So it is confirmed that communication with the proposed method is effective for assuring real-time feature.
3500

Average transmission time per-Message [ms]

40 Proposed Method TCP-based Method

30

Total Transmission Time [ms]

20

3000 2500 2000 1500 1000 500 0 0 20 40 60 80 100 120 Number of Transmitted Messages per-Computer
Breakdown of Both Link

10

High Priority Low Priority

0 0 20 40 60 80 100 Number of Transmitted Messages per-Computer 120

Fig. 5. Influence of transmission delay.

Primary Route
Up Link Control Center Computer Down Link

Fig. 7. Influence of congestion in computer.

Substation Computer

Message

5. Conclusion
In this paper, we proposed a communication method with IP-based wide area networks for SCADA systems. This method features characteristic described below. 1) Real-time data transmission 2) Resilience to IP network faults 3) Effective handling of burst of messages for event or alarm information We also evaluated the influences from delay and breakdown of transmission route in the IP-based network and information congestion in the control center computer. The result shows that the proposed method is more effective for assuring real-time and resilient feature than the TCP-based communication method.

Secondary Route
Up Link Down Link Breakdown of Down Link
Message

Fig. 6. Breakdown of transmission route and link. The double-redundant transmission eliminates the influence of single route breakdown and enables stable communication. Worse network fault would be a single route breakdown and down link breakdown of another route described in Fig. 6. We caused the fault and then a substation computer sent 10 messages to the control center

308

Proposal of Reliable and Real-Time Communication Method for SCADA on IP-Based Networks for Next Generation ~

References
[1] Communication networks and systems in substations Part 7-4: Basic communication structure for substation and feeder equipment - Compatible logical node classes and data classes, IEC 61850-7-4 ed1.0, May 2003. [2] Object Management Forum Homepage, http://www. omg.org, Mar. 2010. [3] Sun Java Homepage, http://java.sun.com, Mar. 2010. [4] H. Yusa, M. Kiuchi, Y. Serizawa, Middleware architecture for ensuring QoS and reliability in IPbased power system communications, IEEE Power Eng Soc. Transm. Distrib. Conf. Expo, May 2006. [5] Y. Serizawa, T. Tanaka, H. Yusa, Y. Koda, G. Yamashita, M. Miyabe, S. Katayama, T. Tsuchiya, K. Omata, Verification of distributed real-time computer network architecture associated With off-the-shelf and dedicated technologies, IEEE Transactions on Power Delivery, Vol. 24, No. 3, pp. 1206-1217, Jul. 2009.

Eiji Ohba Eiji Ohba received B.S. and M.S. degrees in electrical engineering from Tohoku University, Sendai, Japan in 1984 and 1987, respectively. In 1987 he joined the Central Research Institute of Electric Power Industry (CRIEPI). His work has been in the research of communication network reliability, IP network technology and cyber security. He is a member of the IEEJ and IEICE.

Hiroyuki Yusa received the B.S. and M.S. degrees in electrical engineering from Tokyo University of Science, Japan, in 1993 and 1995, respectively. In 1995, he joined the Central Research Institute of the Electric Power Industry (CRIEPI). His work has been in the research of distributed object, mobile agent, and active network technology and management systems for telecommunication networks. He is a member of IEEJ and IEICE.

Mai Kiuchi received the B.E. and M.E. degrees in electrical and electronical engineering from the University of Tokyo, Japan in 1997 and 1999, respectively. In 1999 she joined the Central Research Institute of Electric Power Industry, (CRIEPI), Japan, where she has engaged in research on electric utility communication systems and cyber security. She is a member of IEEJ and JSAI.