21, rue d'Artois, F-75008 Paris

http://www.cigre.org

35-101

Session 2002
CIGR

New tools to observe and control large networks

Dr.-Ing. G. Danils*,
PSI AG, Germany
Dr.-Ing. Gnter Beiler,
Siemens AG, Germany
Dr.-Ing. Bardolf Engel,
ABB Utility Automation GmbH, Germany

Keywords: control network architecture engineering acquisition workforce

Due to economic aspects, the area under control of one
control system has been steadily growing for the last
few years. This augments the workload on the operators, who additionally loose familiarity with their
network. For these reasons, new tools to improve the
communication of the operator with the network and
other staff in the field have been developed.
These tools are presented in this paper. First, a distributed architecture of control systems is presented, which
enables the operation of such large areas. Second, new
tools to lower the workload of the operators in such
large and complex structures are discussed.

Introduction

Main control centre

Secondary control centre
Data acquisition
Data engineering

Figure 1 Control structure in a large control area

The deregulation of the electricity market has increased

the economic pressure on utilities. This leads to mergers
and acquisitions for small utilities. Large utilities reorganize their structures. Especially in network operation,
this process leads to larger areas under operation of one
organizational unit.

The system architecture changes in such way, that the

network is being controlled from few large control centres, which can be switched together in periods with low
workload. This is referred to as a distributed control
system.

Consequently, the areas under control of one control

centre are being enlarged. This augments the stress on
the control staff: the amount of information rises, the
familiarity with the control area diminishes.

Data acquisition will no longer be a central task of the

main control centre, but will also be decentralized. This
does imply the preparation of the captured data for the
control system not on-site of the main or secondary
control centres.

In order to enable utilities to operate their networks

properly, new tools for communication have to be built.
This implies communication on-site, i.e. in the control
room or between different control rooms in the same
network area, and communication in the whole network
area under control. In respect to this, both organizational
and technical aspects have to be dealt with.
An overview of a possible control structure, which can
emerge from this process, is given in figure 1.

The data engineering nowadays is organised centrally.

In future it will be performed more decentralized, either
by the staff in the different control centres, or even by
staff in the field, on mobile workstations.
Finally, restoration techniques have to be implemented,
which guide the staff in the field and coordinate them
among themselves. This is the task of modern workforce management systems.

* PSI Energie EE, Boschweg 6, D - 63741 Aschaffenburg, Germany, Tel.: +49 6021 366 896, e-mail: gdaniels@psi.de

2 Distributed Control Systems

2.1 System architecture
The system architecture of distributed control systems
differs from that of a conventional system, which generally includes on one location RTU-connection-frontends, data-acquisition servers (often named SCADA),
application servers and HMI-workplaces as well as
related data engineering subsystems:
On each location, only parts of the conventional
system may be installed. For example, at a location
for data acquisition, only data acquisition computers
are installed.
All locations are connected by WAN (TCP/IP). This
implies, that the normal internal communication
between the components of the control system can
take place between different locations. Even if by
reasons of different lifecycles of the systems a standardised protocol between the different locations
(different blocks) is often recommended.
The next figures contain the process information flow.
For better overview the application servers, which are
attached to a data acquisition system are left out, as well
as the different redundancies. The information flow
between the single (sub-) systems in information and
command direction explains the mechanisms of the
system architecture.
The example shows a control system, consisting of a
main control centre and a secondary control centre,
which are both in charge of different parts of the control
area (figure 2).
Each computer has its own parts of the dynamic and
static data models. The contents of these data models
are identical from the view of the related network within
each subsystem, but may differ between them. The
version number of the static data model indicates this.
All computers with the same version of the static data
model belong to one operational group.
The same data point may occur in the data model of
different groups of the control system. For this reason,
the static data model contains the information, to which
group each data point normally belongs, i.e. which
group is responsible for which data point. This responsibility can be changed on line and is also referred to as
authority.
In figure 2 the communication in information direction
is shown. In this example, the secondary control centre
receives an information, which is relevant for the main
control centre.
As soon as an information enters the system it is
checked, to which group the data point belongs.
In case, the data point is represented in another group as
well, the information is sent to a data acquisition computer in the other group. The data acquisition computer
distributes the information in its own group.

main control centre

secondary control centre

ow 1

ow 2

ow 3

da 1
1

de 1

da 2

de 2

ow operator workstation
da data acquisition
de data engineering

dynamic data model

static data model

Figure 2 Information direction

In analogy to this, in command direction, the software
has to determine, which data acquisition computer is the
master for the effected data point and thus has to send
the command into the process.
In case, for example, an operator in the main control
centre operates a circuit breaker, which is controlled
from a secondary control centre, the command has to be
released from the secondary control centre, as shown in
figure 3.
In this figure, the distribution of the switching command
to all other computers is left out for reasons of clarity.
Also the communication needed between the workstations (internal tests, interlocking) within one control
centre is left out for this reason.
The check-back signal of the circuit breaker is processed the same way as other signals in information direction.
Using this technique, the main control centre can take
over functions of the secondary control centre, for example during off- times. Of course, this mechanism only
works if the data models of both groups contain this
circuit breaker. Special mechanisms have to secure the
transfer of authorities in these cases.
main control centre

secondary control centre

ow 1

ow 2

ow 3

da 1

de 1

da 2

de 2

ow operator workstation
da data acquisition
de data engineering

Figure 3 Command direction

dynamic data model

static data model

2.2 Data acquisition

The process of data acquisition, as shown in figure 4,
consists of three steps:
First, the telegram, coming from the process, has to
be collected. This is the function of the RTU (c).
The telegram must be sent to the appropriate data
acquisition computer. For this purpose, an interface
to the TCP-basing computer network of the control
system is needed (d).
The data acquisition computer has to convert the
content of the telegram to the appropriate data point
and event in terms of the function of the control system (e).
These three steps can be combined or carried out on
different locations, as shown in figure 4 too.
In the conventional control system, the telegram is send
by the RTU directly to the data acquisition computers.
These computers have direct serial interfaces (V.24,
X.25) with the RTU.
A device, developed in the last few years, the telecontrol gateway, is replacing the interface of the data acquisition computer. It consists of the before mentioned
interfaces and a TCP/IP connection to the data acquisition computer.
The telecontrol gateway has no other function than to
embed the information content of the complete telegrams in a TCP/IP block. The advantage of this technique is, that the telecontrol gateway needs a reduced up
to nearly no data model. For each interface, in minimum
only the modulation of the signal is parameterised.
For reasons of performance, more intelligent gateways
can be used. These gateways have increased capabilities
on preparation of the information for the control system
and packaging. Of course, they need a more sophisticated data model.
The result of this is that data acquisition computers
became standard workstations.
The next step is to locate the telecontrol gateways directly in the substations, and connect them to the main
control centre via WAN.
Conventional

Distributed

Operator
workstation

Operator
workstation

Data
acquisition
computer
Interface

Data
acquisition
computer

Operator
workstation

e
Data
acquisition
computer

e
Data
acquisition
computer

d
Telecontrol
gateway

Operator
workstation

d
d

Telecontrol
gateway

c
RTU

RTU

RTU

Telecontrol
gateway

c
RTU

Figure 4 Overview of data acquisition techniques

It should be mentioned here, that RTU, using TCP/IP

based protocols like IEC870-5-104 can be connected
directly to the data acquisition computers. The advantage of a better preparation of the information for the
control centre is being lost in this case.
In a fully distributed system, the data acquisition computers can be located anywhere in the control area, as
long as a WAN-interface to the main or secondary control system is available.
2.3 Applications
Applications in the control centre have to be provided
with all necessary data. Furthermore, applications often
must run on one location. This is quite evident for the
example of automatic generation control (AGC).
For these reasons, also in distributed systems, applications run on the location they are needed.
It has to be taken care in data engineering, that the application is always served with the necessary amount of
data. I.e. these data have to be included in the data
model on the location of the control centre running the
applications.
2.4 Data engineering
The main aspect of data engineering in a distributed
control system is keeping the database consistent for the
entire system.
In case, each control centre has its own database, it will
be difficult to merge the databases. This merging becomes necessary, as soon as for example the main control centre takes over functions of the secondary control
centre. In this case, the only possibility is a locking
mechanism over all databases in all systems.
For this reason, it may be better, when all data engineering staff works on the same database.
However, in case the same database is being accessed
from different systems and locations, two aspects must
be considered:
The amount of data exchange between the different
locations has to be reduced to its minimum, for reasons of network capability.
The portions of data necessary to lock have to be
kept small, for reasons of interference of the work of
different data engineers.
The example of data engineering by the staff in the field
on a mobile computer, having access to the central system via modem may clarify these two aspects.
For data engineering, two application types are of interest:
Single data points are entered directly in the central
database. In this case, the amount of data to be exchanged is low and communication capacity will not
be a problem.
Complex data, like large network diagrams, cannot
be shared with other data engineering staff. Once a

staff member starts editing a diagram, it is loaded to

his computer and remains locked in the central system, until it is released again.
However, in the case of diagrams, also parts can be
locked, so multiple staff members can work simultaneously in the same diagram. This is especially important
in case large one-line diagrams, which may have a size
of over 3000 monitor screens, have to be edited.
This mechanism is shown in figure 5. In a first step, the
diagram to be edited is selected by the staff in the field.
The database sends a reduced version of the diagram,
which is displayed on his workstation. In this selection
mode, a part of the diagram can be chosen to be edited
and will then be loaded to the distributed system. The
database locks all elements, which are selected this way,
centrally. Other data engineering staff can select these
locked data in read only mode. They are informed on all
changes, which occur in it, if they want.
As it can be derived from this example, also complex
changes of large one-line diagrams, involving various
stations can be handled this way. It should however not
be forgotten, that the largest part of the work in the field
is to change telegraphy parameters of RTU, which are
entered directly in the database.
1. Select a diagram

2. Select part of the diagram

Diagrams:
Area North
Area South
Area West
Area East

3. Add a new feeder

Click!
Click!

4. Others cannot edit this part

Most of the above-mentioned advanced applications are

based on the Distribution Power Flow which needs to
be tailored to the specialities of distribution networks
(large networks, mostly radial operated, often unbalanced loads).
In case the required power flow data, such as network
equipment impedances or parameters and load models,
cannot be provided for a distribution system, then simplified power flow algorithms may be considered for
being used in such cases.

4 Outage and workforce management

The interaction between the detection of a fault, the staff
in the control room and in the field has to be coordinated. This implies the integration of ERP-packages like
SAP, and new tools to organize restoration in the control room.
Faults are reported in the transportation network by the
scada system, in the distribution network additionally
directly by customers, who communicate to the utility
by means of phone.
These phone calls are received by call centres, which
serve as a first filter and sort the phone calls, a special
application localizes the outages related to the topology
information of the phoning customers.
In the control room, new visualisation approaches can
be used, basing on geographical information of the area
under control. In this way, not only the real position of
the affected transformer substations, cables or lines, but
also the momentary position of crews for restoration is
available.
Additional tools select the right crews and coordinate
the workflow of these crews, report the actual status of
the work to the operator and help the call centre to inform the customer more precisely.
ERP-systems are integrated in this process. This way, it
can be kept track of the material stock in parallel.

Figure 5 Distributed editing of large diagrams

Distribution Network Applications

The operator of a large distribution system needs to be

supported by advanced applications in his control system for fault location, fault isolation, service restoration
and intelligent network reconfiguration [1].
Although these tools are reported in many technical
publications since a couple of years, it can be observed,
that they are more and more required for new distribution control centres and that they are really used in dayto-day distribution network operation to enhance efficiency and reliability.

Figure 6 illustrates the workforce management (WFM)

business process and the necessary interfaces to the
outage management system (often integral part of distribution management system) and the distribution management system [2,3]. In addition interfaces to customer
care systems and geographic information systems are
possible.
The WFM system controls and optimises all crucial
components of the dispatch management operation. It
automatically recommends to dispatchers the most appropriate crew for each specific task based on selection
criteria such as crew skills, proximity and workload.
The integration of a workflow in the process of restoration, which directly controls the different communication channels and organizes the communication between
the different organisational entities, is needed, in order
to coordinate the different phases of restoration.

Outage Calls or
Service Request
and Status
Exchange

Customer
CustomerInformationInformationSystems
Systems

Outage
Outageand
andNetwork
Network
Management
ManagementSystems
Systems

Workforce
WorkforceManagementManagementSystem
System(WFM)
(WFM)
automatic
automaticdispatching
dispatchingbased
based
on
onappointments,
appointments,priorities,
priorities,
skills
skillsand
anddistances
distances

Dataexchange
and Datasharing

Crews
Crewswith
withmobil
mobil
computer
computer
Other
OtherSystems
Systems(Geographic
(Geographic
Information
InformationSystem,
System,SAP,)
SAP,)

Job Tickets
and Status

Data
DataRadio-System
Radio-System

Figure 6 Work Force Management Business Process

The WFM closes the information loop between control
room and field. Figure 7 sketches some details of WFM
technology. In most cases the communication between
the mobile PCs of the maintenance crew and the control
room is based on GPS and GPRS, but also other radio
networks may be used.
The benefits of workforce management are:
Better planning of work schedules
More flexible management of emergency situations
in distribution networks
Reduction of outage times and lost revenues
Paperwork kept to a minimum
Better control of activities in the field
Improvement of customer service, e. g. reduced response time, provision of accurate information to
customers
Increased efficiency and higher productivity of the
maintenance stuff
Reduction of operational costs
These tools make use of external applications on different platforms and means of communication. The integration needs to be task adjusted in such cases. Sometimes a simple straightforward interface can be built, in
other cases, these interfaces need more intelligence
related to the distribution of tasks to the applications
and the level of integration.

computer
computeraided
aideddispatch
dispatchfor
for
minimizing
minimizingmanual
manualinteraction
interaction

CDPD

GPS-Tracking
GPS-Tracking

GPRS

GSM
Automatical
Automaticalswitching
switching
between
betweendifferent
differentradio
radio
infrastructures
infrastructures

Figure 7 Work Force Management - Technology

For example an interface to the call centre should reduce the number of messages, in case similar messages
from one area occur. For such more integrated solutions
the problem of the related data models and engineering
is a main topic.

Conclusion

The change in the size of the controlled networks need

new approaches to the architecture as well as additional
applications. The change in the information infrastructure implies the integration of applications, which are
separated up to now.
As these applications are of growing complexity, the
aspect of training should not be neglected.

6 References
[1] Beiler G., Mangler R.: Emergency management
as integral part of modern network control systems (in
German). ETG Congress 23.-24.Oct.2001, Nuremberg.
http://www.vde.de/vde/img/fach/etg/kongress/1_04.pps
[2] Spanheimer R.: Workforce Management with
GPRS and GPS to more efficiency in the field service
(in German). EW Jg. 100 (2001), Heft 12
[3] Schneider, H.-J.: Staff dispatching systems in
network operation (in German). ETG Congress 23.24.Oct.2001, Nuremberg.
http://www.vde.de/vde/img/fach/etg/kongress/1_01v.pd
f