Redundant and Synchronized EtherCAT Network

Gunnar Prytz, Jonny Skaalvik

ABB Corporate Research
Bergerveien 12
NO-1396 Billingstad, Norway

Abstract - Communication redundancy and time synchroni-
zation are two important requirements commonly found in
industrial systems. However, achieving high redundancy
performance in combination with highly accurate time syn-
chronization at all times is a challenging task. EtherCAT is a
high performance, functionality rich communication tech-
nology offering both advanced redundancy features and
accurate time synchronization. This paper will describe a
robust solution where an EtherCAT system offers high
redundancy performance in combination with accurate
time synchronization. The solutions is based on using Ether-
CAT slaves integrated into the EtherCAT master nodes. In
the redundant case there is a need for two of these inte-
grated slaves and they need to be configured appropriately
to offer a robust solution.
Keywords - redundancy; synchronization; networking
I. INTRODUCTION
The requirements of industrial systems may be quite
different from the ones found in typical office environ-
ments when it comes to some specific features. Industrial
systems may have requirements related to factors like
safety, time synchronization, determinism and availabil-
ity that are not common in office systems. In other situa-
tions the requirements are much harder than those in the
office environment. Typical examples are determinism
and time synchronization requirements. Therefore special
communication solutions directly targeting industrial
requirements have been developed for these challenging
situations. EtherCAT [2] is one such technology which is
extensively used today and it is the focus of this paper.
EtherCAT is a communication technology offering
very high real-time performance accompanied by a rich
functionality set making it a leading technology from a
technical perspective. This combined with openness of
the technology and the wide availability of software and
hardware components has made EtherCAT a market lead-
ing technology as well [1], extensively used in many dif-
ferent industrial systems requiring high-end deterministic
communication performance. The technology is also
increasingly used in laboratory and instrumentation envi-
ronments. A detailed presentation of the EtherCAT tech-
nology is however outside the scope of this paper.
Like many of the automation protocol suites Ether-
CAT has developed adapted solutions for redundancy
(e.g. network redundancy in the context of this paper).
Also when it comes to time synchronization the various
automation protocol suites sometimes specify their own
time synchronization solutions, either by specifying how
to use the IEEE 1588 [4] protocol or by making adapted
and optimized solutions. EtherCAT therefore offers very
accurate time synchronization of the devices reaching
down well below the microseconds level by the aid of
hardware assisted functionality in the devices.
Since EtherCAT has become a main technology in this
domain it is of key interest to investigate how the tech-
nology could be used in order to achieve both high levels
of redundancy in combination with accurate time syn-
chronization in a robust way. This is not a trivial topic
and this paper will outline one possible method for how
to handle this in EtherCAT.
II. TIME SYNCHRONIZATION IN ETHERCAT
By using EtherCAT out-of-the-box the achievable
level of time synchronization accuracy is typically speci-
fied to be better than 1 μs. This means that the clocks of
all devices are synchronized to within 1 μs at all times.
This is similar to most so-called real-time Ethernet tech-
nologies and sufficient in many cases.
However, EtherCAT also offers a significantly higher
performing time synchronization mechanism referred to
as the distributed clocks (DC) functionality. The DC
functionality is an attractive feature for high performance
control systems and instrumentation systems and depends
on special hardware support in the slave devices (FPGA,
ASIC etc.) to achieve a synchronization accuracy of less
than 100 ns, in many cases down to a few tens of nano-
seconds (implementation and situation dependent). The
DC functionality can be enabled in most EtherCAT hard-
ware implementations and will make it possible for all
EtherCAT slaves with DC support to be very accurately
synchronized (any devices without DC support will still
be synchronized to within 1 μs or so).
It is normally the slave closest to the master which is
the clock master or reference clock in an EtherCAT net-
work. In the case with a mix of devices with DC support

978-1-4244-5841-7/10/$26.00 ©2010 IEEE 201 SIES 2010

 and devices without it the device with DC support sitting Later in the paper it will be shown that some special
nearest to the master will normally be the clock master. measures needs to be taken to guarantee uninterrupted
In EtherCAT the highest time synchronization accu- time synchronization accuracy when a (media) redundant
racy can only be guaranteed between the slaves. This is EtherCAT system experience a failure (e.g. when a cable
the same for both the ordinary time synchronization as is accidentally cut or damaged).
well as the DC feature. Since the EtherCAT master nor-
mally runs in a standard CPU the jitter in the transmission
and reception times of telegrams (i.e. the variability of
the cycle times) may be of the order of microseconds
(assuming a real-time operating system is used). The Eth-
erCAT slaves on the other hand have the complete net-
work interfacing and telegram processing in hardware
and thus the timing is virtually constant. Therefore, the
synchronization accuracy between the master and the
slaves may be in the order of microseconds whereas the
synchronization accuracy between the slaves is either
better than a microsecond (without DC) or down to some
tens of nanoseconds (with DC).

III. INTEGRATING THE ETHERCAT MASTER

WITH AN ETHERCAT SLAVE

In many situations it is only required to synchronize

the slave devices, e.g. in some motor control applications.
When both the EtherCAT slaves and the master needs
high time synchronization accuracy some additions have
to be made. Rehnman and Gentzell [9] described a solu-
tion where the EtherCAT master and one EtherCAT slave
was integrated into one physical node or device. The ben-
efit of this is that the accurate clock of the slave device is
available to the EtherCAT master via some direct inter-
face (e.g. a digital line). This enhanced master device will
then be accurately synchronized with the other slave
devices. In this setup all EtherCAT nodes on the network
could be synchronized to the same accuracy. Further-
more, by enabling the DC functionality of the inbuilt
slave the enhanced master device could be very accu- Figure 1. A combined EtherCAT master and slave device
rately synchronized to the other devices with DC support where an external clock source is used to set the internal
clock of the EtherCAT slave via the process data interface
since it has direct access to this slave via a dedicated (PDI). The EtherCAT slave then synchronizes the EtherCAT
hardware interface. master via the SYNC (S) line.
The coupling between the EtherCAT master and the
integrated slave needs to be tight in order for this to work. V. MEDIA REDUNDANCY IN ETHERCAT
The work of [9] described how the EtherCAT master can
be synchronized with the collection of slaves by the use Redundancy is a common way of increasing the avail-
of an integrated slave in the master. In this setup the Eth- ability of a system. Redundancy means that some parts of
erCAT master therefore is a clock slave. the system has been duplicated. There are various levels
of redundancy in a system, ranging from media redun-
IV. ACCURATE SYNCHRONIZATION WITH AN dancy where certain communication paths have been
EXTERNAL CLOCK duplicated via devices with multiple network interfaces
to systems where the devices are themselves duplicated.
In other situations it is a requirement that all Ether-
In extreme cases duplication may not be enough so paths
CAT nodes (master and slaves) are clock slaves of some
or interfaces or devices can be present in triplets or more.
other master clock outside the EtherCAT system.
Such systems will not be discussed in this paper.
A setup for doing this is depicted in Fig. 1. The solu-
Redundancy performance or redundancy requirements
tion shown in Fig. 1 connects the external reference clock
are often described in terms of recovery time. The recov-
to the EtherCAT slave via the PDI (Process Data Inter-
ery time of a system is the sum of two terms: 1) the time
face) of the EtherCAT slave. The master is synchronized
it takes to detect a failure and 2) the time it takes to rees-
with this slave via a dedicated digital line.
tablish communication. The most common network

978-1-4244-5841-7/10/$26.00 ©2010 IEEE 202 SIES 2010

 redundancy solution is the well-known RSTP technology
which is commonly used in office type of networks.
RSTP typically offers recovery times of seconds or
slightly below this for reasonably complex topologies
(the recipe for achieving faster recovery times with RSTP
have been specified in IEC 62439 [3]).
When it comes to network or media redundancy Eth-
erCAT offers a ring redundancy approach which can han-
dle many common failure modes. Using a physical ring
topology an EtherCAT system can handle the failure of a
link or a device while still being able to maintain the
functionality for the remaining system. The recovery
time will be in the milliseconds range. Such a solution
cannot handle the failure of the master, in this situation
EtherCAT master redundancy using two masters in paral-
lel needs to be used but this is outside the scope of this
paper.
In a redundant EtherCAT setup the master is con-
nected to an EtherCAT ring via two network interfaces.
In every cycle the EtherCAT master sends its telegram(s)
on both of these ports. These telegrams will be forwarded
throughout the ring and eventually return at the master.
The master will thus receive the telegram(s) it sent on
both ports.
When a failure happens the EtherCAT devices next to
the failure (be it a link failure or a broken device) will
enable the loopback functionality and all telegrams will
be reflected back in the direction they came from. This in
practice means that the EtherCAT ring network has
turned into two EtherCAT line networks.
This transition from a ring topology to a dual-line
topology has a significant influence on the time synchro-
nization accuracy. Previously the benefits of having an
EtherCAT slave device integrated into the EtherCAT
master physical node were discussed. However, this is
not enough in the redundant setup with an EtherCAT
ring. When the ring turns into a dual-line topology after a
failure the result is that one of the two lines lack both an
EtherCAT reference clock and an integrated slave. The
absence of an EtherCAT reference clock means that no
DC functionality is available for these slaves. Thus there
is a significant asymmetry between the two lines where
only one of the two EtherCAT line networks have accu-
rately synchronized devices.
VI. ENHANCED ETHERCAT MASTER
In order to guarantee the time synchronization accu-
racy of a redundant EtherCAT network using a ring topol-
ogy there is therefore a need to integrate two slaves into
the physical EtherCAT master device. Upon a failure the
ring will transition into a dual-line topology which both
need an integrated slave to be able to synchronize the
master and all devices together accurately. This is true
regardless of whether the DC functionality is used or not.
Furthermore, it is important to realize that the process-
ing directions are structurally opposite in the two lines
978-1-4244-5841-7/10/$26.00 ©2010 IEEE
(EtherCAT devices only process traffic in one direction).
This has significant consequences.
Since the EtherCAT telegrams are processed in one
direction only the result is that in one of the lines the pro-
cessing will be in the direction away from the master
whereas in the other line the processing will be in the
opposite direction.
Since the telegram processing only takes places in one
direction one of the internal slaves need to be connected
in reverse, i.e. to have the opposite processing direction,
in order to ensure that the first processed slave is the inte-
grated one in both of the two line networks. This situation
is shown in Fig. 2. For completeness it should be said that
it would not be necessary to configure the integrated
slave in reverse if the DC functionality was not used.
In the ring topology state (Fig. 2) the EtherCAT mas-
ter will transmit the telegram(s) on both ports. In one
direction (A to H) all slaves except slave H will process
the telegram(s). In the other direction (H to A) only slave
H will do processing. This is transparent to the applica-
tion but needs to be handled properly by the master.
In the dual-line state (Fig. 2) with a failure somewhere
in the ring the system now consists of an EtherCAT mas-
ter with two lines connected. The telegram sent out on
one of the lines (A-D) will first the processed by slave A
and then subsequently by slaves B, C and D before
returning to the master. The telegram sent out on the
other line (H-E) will be processed by slave H before slave
E, F and G and then return to the master. Since the second
internal slave (H) has been connected in reverse this is
the first slave that processes the telegrams from the mas-
ter. Since it is the first slave it can act as the DC master
for all the EtherCAT devices of this line. The result is
then that both EtherCAT lines are accurately synchro-
nized to the first slave of the respective line. Since both
of these first slaves are integrated into the EtherCAT mas-
ter node they can both be accurately synchronized with
the EtherCAT master. The result is a redundant EtherCAT
system where both the master and all the devices are
accurately synchronized in both the ring state and in the
dual-line state.
As a final remark it should be noted that the EtherCAT
telegrams ideally should be reconfigured upon a transi-
tion from the ring topology to the dual line topology. This
would be more efficient since the line with slaves A to D
(Fig. 2) would not have any use for data destined for
slaves E to H and vice versa. However, such a reconfigu-
ration may be too demanding to do for a system in run-
time.
VII. ADDITIONAL REMARKS
After the transition from the ring topology to the dual
line topology the concept outlined here will enable accu-
rate synchronization of all devices present. However, the
master and the slaves E to H in Fig. 2 would need some
transition time before the desired time synchronization
203 SIES 2010
 Figure 2. Enhanced EtherCAT master node with two integrated slaves. The integrated slaves are coupled with opposite processing
directions. The position of the failure (a link failure in this example) is indicated by the cross in the ring to the left. The processing
direction of the EtherCAT slaves are indicated by the arrows and the solid box inside the slaves.
accuracy can be obtained after a failure and transition
from the ring to the dual line state. This is due to the fact
that the EtherCAT master needs to measure internal
delays in the system, i.e. delays between the devices.
These delays are used to calculate the clock offset
between the devices (and the master). Special care has to
be taken to avoid this initial transition but this is outside
the scope of this paper. There may also be other failure
modes that needs special handling, e.g. for systems sup-
porting hot swapping or hot connect.
VIII. CONCLUSION
This paper has outlined a robust approach for the com-
bination of accurate time synchronization of all nodes in
an EtherCAT system combined with high performance
network or media redundancy. The approach uses only
standard EtherCAT functionality. The solution depends
on tight integration of the EtherCAT master with two
internal EtherCAT slaves, resulting in an enhanced Ether-
CAT master with some additional functionality. In addi-
tion the approach requires reversing the processing
direction of one of the integrated slaves.
Using this synchronized and redundant solution the
EtherCAT system will have a very interesting feature set
978-1-4244-5841-7/10/$26.00 ©2010 IEEE
relevant for high-end industrial communication systems.
Accurate synchronization of all devices is guaranteed
immediately after failure recovery, i.e. after transition
from a ring topology to the dual line topology.
REFERENCES
[1] Ethernet-Based Device Networks Worldwide Outlook.
Market analysis and forecast through 2012. ARC Advisory
Group 2008.
[2] IEC 61158. Industrial Communication Networks - Field-
bus Specifications. IEC 2007.
[3] IEC 62439 ed. 2. High Availability Automation Networks.
IEC 2010.
[4] IEEE 1588-2008 IEEE Standard for a Precision Clock
Synchronization Protocol for Networked Measurement
and Control Systems.
[5] IEEE 802.1D-2004. MAC Bridges. IEEE 2004.
[6] http://www.ethercat.org.
[7] Prytz, G: A performance analysis of EtherCAT and
PROFINET IRT. ETFA 2008. Proceedings, pp. 408-415,
2008.
[8] Prytz, G: Redundancy in industrial Ethernet networks.
WFCS 2006. Proceedings, pp. 380-385, 2006.
[9] Rehnman, M. and Gentzell, T: Synchronization in a force
measurement system using EtherCAT. ETFA 2008. Pro-
ceedings, pp. 1023-1030, 2008.
204 SIES 2010