1

Practical Experience from Design and Implementation of IEC 61850 based Communications Network in Large Offshore Wind Installation
Maciej Goraj1, Yannick Epassa2, Jim McGhee3
1, Utility Market Manager, RuggedCom Inc., Estartetxe 5, OF 227, 48940 Leioa, Vizcaya, Spain 2, Department of Professional Services, 300, Applewood Cres. Unit 1, Concord, Ontario, L4K 5C7, Canada 3, Utility Market Manager, RuggedCom Inc., 269 Deerview South East, Calgary, Alberta, T2J 6W7Canada

Abstract: This paper discusses special considerations that

shall be taken into account when designing and deploying Ethernet networks for wind power applications. It also contains description of practical experience from a real project of a big offshore wind farm recently built in United Kingdom. The paper is organized into two parts. First part describes the characteristics of an environmentally robust Ethernet network for wind farm automation applications and identifies key issues and considerations when designing an Ethernet network. The second part provides the description of a real project of a big offshore wind farm composed of in excess of 140 of wind generators where an Ethernet network of more than 200 switches had been deployed.

acquisition, monitoring and control system is running. New state of the art Intelligent Electronic Devices (IEDs) can be accessed remotely for the retrieval of process information. Operations and Protection & Control Engineers are interested in having access to such information in a secure and reliable manner via the enterprise communications network. By enabling reliable access to information, ensuring network availability, and enabling automation and control, provides a method of secure and reliable integration of wind farms into the transmission or distribution network. The possibilities introduced by using an Ethernet network are endless with immediate benefits including fully controllable systems, with either local or remote access to equipment and SCADA providing status information, network health and emerging VoIP and camera solutions.

Keywords: Offshore Wind Farm; Ethernet Network;

Redundancy; Remote Access; IEC 61850

1. Introduction
The spectacular worldwide growth of renewable energy generation plants and distributed energy resources created the need for specialized communications solutions for this market. Connection of distributed energy resources to the electrical grid as well as its integration to the data communication systems is characterized by technical challenges not present in conventional electricity generation plants. Wind farms can be onshore or offshore, frequently built in remote sites that are exposed to extreme weather conditions and lack established telecommunications infrastructure making it difficult to connect to communications network in the central location where data

2. Considerations when Designing Network for a Wind Farm

Offshore wind power applications are different from typical onshore wind turbine projects. Most onshore installations run a low maintenance cost program while offshore maintenance is typically ten times more expensive. Wind turbines in onshore sites can be located on large geographical areas in remote places where terrain forms such as hills and slopes make access difficult. In case of offshore sites the situation is even worse as the only way of accessing the wind farm is by boat or by helicopter. This is not only the cost issue but also issue with weather conditions which in many situations may cause the work onsite to be postponed for unspecified period of time. For offshore installations the cost

related to personnel dispatch in case on-site maintenance or when repairs need to be done is extremely high. To minimize the onsite works on offshore sites the utility requires maximization of function automation and remote diagnostics, maintenance and remote control as well as monitoring capabilities. The topology of a network interconnecting IEDs and other Ethernet enabled device is conditioned by the physical topology of wind farm which depends on a number of factors such as wind profiles, available generator tower placement, costs, etc. In order to keep the installation cost at a reasonable level, most wind turbine sites are made up of many turbines. Wind farm topologies and the most commonly used feeder arrangements have been described in details in [1]. One of the typical wind farm feeder arrangements in radial topology. The physical locations of offshore wind turbines form strings or rows typically of a few up to several dozens of towers. Strings of wind turbines can extend in radial form from a central point where offshore substation is erected. Taking the advantage of such physical topology the network topology that spans the wind farm is of multiple rings connected to the backbone formed by the substation backbone network. Each ring of switches is formed by a loop that results from connecting two strings of wind turbines.

networking devices shall comply with EMC specification as per IEC 61850-3 and IEEE 1613.

2.3 Form Factor Requirements and Physical Media Requirements

All networking devices shall have dual power supply. Ethernet switches that are to be installed in wind generators shall have small form factor as there is little room available on onshore wind tower. The communication interface for connecting IEDs shall be 100BaseTX or fiber optic. For the purpose of creating ring of wind generators the networking devices shall be equipped with fiber optic single mode ports as there can distances up to several dozens of kilometers between wind towers. If the network is planned to handle video surveillance traffic or other bandwidth consuming applications the devices shall be equipped with 1Gbit or 10Gbit interfaces.

2.4 Redundancy
Wind farm networks shall always guarantee proper function against single point of failure. For that reason ring architecture is commonly used for connecting wind turbines where single level of redundancy is required. All critical networking devices such as switches connecting sub-rings or routers connecting to WAN shall be duplicated. For achieving higher degree of redundancy additional elements can be introduced in those points of the network. For example in case of connecting two ring networks of two offshore substation instead of using pair of switches in each network and pair of links, an additional switch can be added in each substation making the interconnection via three switches at each side. Routers and gateways shall implement redundancy application protocols such as VRRP for fast take-over of duties in case of device failure.

2.1 Environmental issues

Offshore wind farms are exposed to high levels of pollution, salt, humidity and abrupt temperature changes. Devices that are going to operate in maritime environment shall have special robust design preventing corrosion caused by salt, high humidity and bird guano, if mounted outdoor.

2.2 EMI Immunity Requirements

A key requirement of network devices in electrical substations and wind turbines is that they must operate properly under the influence of a variety of EMI phenomena [2]. To name just a few of these phenomena we can mention inductive load switching, lightening strikes, electrostatic discharges from human contact, radio frequency interference due to personnel using portable radio handsets, ground potential rise resulting from high current fault conditions within the substation, etc. In particular

2.5 Fault Tolerant, Optimal and Self-Healing Network

The network shall be resilient to failures and shall be characterized by fast recovery time, not exceeding 10-20 ms per hop. The network shall implement mechanisms for avoiding loops and for ensuring Efficient and optimal data transfer through shortest paths.

2.6 Remote Secure Connectivity

The network shall permit remote access so that all devices and systems can communicate to/from a central onshore location. The remote access shall support cryptographic services such as IPSec and it is desirable that user authentication services are also available for access control. The remote access shall be limited only to those devices or segments of the network where devices being accessed are located. It means that for example a SCADA application accessing data from protection and control IEDs may not be allowed to access IP cameras if such are connected to the same network. This can be accomplished with the use of VLANs (Virtual LANs).

engineers are working on-site and need to connect to companys resources or to the internet for downloading specific software, upgrade, configuration file etc. There is not always cellular coverage on offshore sites or the staff may not be equipped with handheld satellite communication devices.

3 Overview of the Offshore Wind Project

The UK Government has outlined ambitious plans to reduce CO2 emissions by as much as 80% by the year 2050. Within this plan it is expected that the wind energy will play an important role and total wind generation power shall reach 28 GW by year 2020. The objective of the UK government is to split the wind generation power between onshore and offshore farms in the proportion of 1/3 for onshore and 2/3 for offshore. The installation described in this paper is going to be world's largest offshore wind farm with total installed generation capacity of 500MW. It consists of 140 wind turbines each of 3.6MW generation capacity. The wind farm is located around two sand banks located 25 kilometers off the coast and the turbines are installed in water depths of between 24 and 34 meters. The 33/132kV transformer station with its three 180MVA transformers is mounted on two offshore platforms located at the wind farm area. The wind turbines are being connected to the off-shore transformer stations via sub-sea 33kV cables. The power is then exported via three three-phase 132kV submarine cables to the onshore substation. The onshore substation is equipped with large capacitor banks and SVC (Static VAR Compensation) reactive power compensation devices in order to meet the requirements for dynamic stability and power quality. The substation is connected to the transmission grid via underground cables. The installation is expected to have a load factor of over 40% and produce around 1,900GWh of electricity in a typical year. The wind farm is scheduled for commissioning in two phases, with the construction work scheduled to be totally completed in 2012, and first generation of electricity planned for 2010.

2.7 Network Management

It shall be possible to manage all network devices within the wind farm using standard SNMP based network management system. The network devices and the management system shall permit discovering network devices and creating graphical network maps. It is also required that the system shall be continuously monitoring all devices status allowing proactive maintenance. Furthermore the system shall facilitate performing bulk upgrades and shall permit managing of configuration files including functions such as configuration file backup or restore.

2.8 Multiple Applications and Multiple Protocols

The network design shall permit seamless functioning of multiple applications and also the use of multiple protocols at the same time. It is common situation that in todays installations various communication protocols have to co-exists. Necessary provisions shall be made for efficient segregation of different types of traffic in the network as well as of filtering of traffic, e.g. multicast traffic generated by applications such as IEC 61850 GOOSE.

2.9 Access to Internet

It is common that all offshore wind facilities such as wind generators or offshore substations shall permit secure access to the internet. This is a provision for situations where maintenance

4 Practical Experience from an Offshore Wind Farm Installation

The wind farm consists of 140 wind turbines, two offshore substations and one onshore substation. Considering wind turbines there are three communications networks with three independent sets of switches that interconnect equipment at all wind generators and connect them to the onshore substation permitting remote connection the control systems. The decision to deploy three communications networks had been dictated by the different characteristics of the applications each network will be supporting and the fact that the scope of work during the project for each of the three applications is split between different providers. In this paper we will be referring to the three communications networks as: Balance of Plant network Wind Turbine Generator network Telephone and Security network

wind towers and substations, between different elements within the same substation, between the different substations and up to central control system via the utility backbone network. The part of the network located in the onshore substation will have connection to the Wind Turbine Generator network and Telephony and security network. The main requirement for the communication network in this project was to efficiently allow the different endpoints to co-exist and share the same data backbone while maintaining the highest degree of reliability and fault tolerance with optimal traffic forwarding. The following equipment will share the network backbone: RTUs (Remote terminal Units) SCADA Servers Terminal Server Web Server Substation Automation Central Units Remote HMIs Protection and Control IEDs The communication protocol used for communication with IEDs in Balance of Plant network was IEC 61850 including MMS protocol and GOOSE messages.

All three networks are based on Ethernet communications and consist of an independent set of switches and communications links located in every wind turbine. In fact these networks are connected together via a common gigabit Ethernet backbone and also at the onshore substation via redundant routers. Another set of routers located in the onshore substation provides WAN links to each of the three networks in order that corresponding SCADA applications from central control centers can access them. In this paper the Balance of Plant network will be referred as the substation protection and control network and its main function is to connect all protection and control IEDs at each wind turbine as well as at the offshore and on-shore substation. The Wind Turbine Generator network has been realized another company and it connects wind turbine controllers via Ethernet to central control SCADA. The Telephone and Security network connects video surveillance cameras as well as provides IP telephony service. This paper focuses only on the Balance of Plant SCADA network, the details of the other two networks are out of scope of this work. The devices that form Balance of Plant network are located in each wind generator and in two offshore substations and the onshore substation. Communication is required between IEDs in

Figure 1. Robust Ethernet Switch used to interconnect devices to the network inside each wind tower

4.1 Network Topology

The communication infrastructure is a switched network, meaning that all the network elements will be part of the same LAN (Local Area Network), and no routers are needed for the communication within and between the substations. The conceptual architecture of the installation is depicted on Figure 2.

with multimode fiber whereas the cables interconnecting switches between substations and between wind towers are single mode fibers. All the links between wind towers are 100Mbps and the links between switches inside substations and between substations are 1Gbps.

4.3 Multiple Applications and Traffic Segregation

One of the requirements of the design was that the

Internet/VPN Link (ADSL)

SCADA/Control/Supervision/HMI

multipurpose below:

communications

architecture

shall

be

used

Routers

simultaneously for various applications that have been listed SCADA P&C IEDs for the purpose of substation Metering - power quality meters for continuous power factor and frequency

Onshore Substation Network

Protection & Control Panels

automation and remote control and monitoring monitoring of 3 phase system variables such a current, voltage,
Network Offshore Substation #1 Network Offshore Substation #2

real and reactive power,

Internet access - secure access point to Internet in

every wind tower and in substation for generic purposes, e.g. maintenance staff needing to access Internet in order get to companys network resources, etc.
Figure 2. Simplified network architecture of the offshore wind farm

Cathodic protection - specialized cathodic protection

devices in every wind tower VLANs (Virtual Local Area Networks) had been used in order to provide traffic isolation in the switched network and help prevent unauthorized users accessing restricted part of the network

For simplification the depicted architecture does not contain all the switches in each ring neither all rings of wind generators. The backbone network consists of pairs of core Ethernet switches. The core switches are duplicated to provide for redundancy in case of a switch failure. They play the role of aggregation switches as well, connecting the three substations together via Gigabit Ethernet export cables. The Gigabit backbone is extended within each substation through a ring of switches terminating at the core switches.

4.4 Fault Tolerant and Resilient Network

To fulfill the redundancy and fault tolerance requirements, multiple provisions have been made at the network design stage. All the switches at the backbone network are duplicated. The substations backbone is connected in a ring topology which provides a redundant path for communication throughout the network. For calculating alternative paths in redundant network eRSTP protocol is used throughout the network. This is a proprietary version of RSTP which allows switches to detect loops and internally block messages from circulating in the loop and provides 5ms recovery time per hop, compared to 50ms for RSTP. For ensuring gateway redundancy for all the switches in the network the Internet Gateway Routers are configured with VRRP

4.2 Network Links Requirement

Standard 100/1000TX electrical cables with RJ45 connector have been used to connect all IEDs to network switches in wind towers and also inside the protection and control panels in the substations. All communication links between switches are fiber optic. The inter-switch links inside the substations are realized

(Virtual Router Redundancy Protocol).

between physical and logical separation of the network traffic that can be translated into degree of function integration, redundancy and total cost of the installation. Finally the design decisions are dictated by customer requirements. Last but not least in future projects other communications media such as wireless links based on 2G/3G/4G cellular networks or broadband microwave technology such as IEEE 802.16e may be considered as an alternative for backup connections.

4.5 Secure Remote Connectivity and Generic Access to Internet

The secure remote connectivity to network is realized with the use of two Internet Routers configured as L2TP (Layer 2 Tunneling Protocol) VPN Access Concentrators. L2TP is a tunneling solution that enables the routers to accept remote connections securely on their ADSL connection. To enhance the

6 References
[1] M. Reichard, D. Finney, J. Garrity, Sr., Windfarm System Protection Using Peer-to-Peer Communications, 60th Annual Conference for Protective Relay Engineers, College Station, Texas, March 2007 [2] M. P. Pozzuoli, Ethernet in Substation Automation Applications Issues and Requirements, White paper, www.ruggedcom.com

security of this process, IPSec had been implemented along with L2TP. This method provides encryption and authentication. The routers are configured with a pool of addresses to assign to each remote user. Additionally a Radius Server is provided for the routers to enable verification of users credentials for authentication. For the purpose of generic access to internet there are DHCP servers in the network which allows any station to automatically get an IP address when connected on any port specifically destined for Internet usage. The DHCP server functionality has been enabled in the routers that are located in the onshore substation. The stations connecting to ports destined for Internet usage will be assigned IP addresses in a different subnet than the rest of the network. Firewalls Rules are implemented to prevent any station connected on an internet port from being able to communicate with a device in the BoP SCADA. All the above provisions provide for an additional level of security and isolation of internet access traffic from the wind farm traffic. Maciej Goraj received his B.Sc. and M.Sc. degrees from the Warsaw University of Technology in 2000, 2001 respectively. After graduation Maciej moved to Spain and joined General Electric Company in 2001 were he worked in different positions. Maciej has been involved in the design and implementation of multiple communication protocols and has field experience as he commissioned network systems in high voltage substations. In 2009 Maciej joined RuggedCom where is currently working as Global Utility Market Manager. He is a member of CIGRE WG

Biographies

Conclusions
The use of industrially hardened networking equipment

D2.28 and member of IEC TC57 WG10. Email: maciejgoraj@ruggedcom.com Yannick Epassa holds a BSc. In Electrical Engineering (telecommunication) from the University of Montreal, CCDP (Cisco Certified Design Professional) and CCNP (Cisco Certified Network Professional) certifications. Yannick is also a double CCIE (Cisco Certified Internetwork Expert) certified in Routing and Switching, and Security. Yannick joined RuggedCom in August 2008 as a Network Consultant in the Professional Services Department. He is responsible for designing, configuring, jointly with carefully designed network and open standards such as IEC 61850 enables reliable communications and ensures maximum availability of data and automation of functions. The described project is characterized by a high level of integration of applications in the communications network and high level of redundancy. However there still exists possibility for further optimization in future installations by means of using one common set of Ethernet switches per wind generator instead of three independent set of switches. There shall always be a trade-off

installing and optimizing customers networks. Prior to joining RuggedCom. Mr. Epassa has held several positions within the networking field including a network support analyst position at Videotron (Canadian ISP) , and a Senior Service Assurance Engineer position with AVAYA. Email: yannickepassa@ruggedcom.com Jim McGhee received his BSc in Computer Science from University of Calgary in 1986. He joined Westronic Inc. in 1987 as a Software Developer. Afterwards Jim became part of General Electric when it acquired Westronic. Jim was promoted several times through the Research and Development department and in 1996 held the position of R&D Manager. In 1999 Jim moved into Product Management where he was responsible for developing new products for the market. Jim left General Electric in 2003, returning in 2005 as Segment Manager and his most recent position/title was Marketing Program Manager of Substation Solutions. In 2010 Jim joined RuggedCom where he holds the position of Utility Market Manager for North America and APAC region. Email: jimmcghee@ruggedcom.com