Mesh Control

B0700AZ
REV L
I/A Series
System
The MESH Control Network Architecture Guide
October 15, 2009

Invensys, Foxboro, FoxView, I/A Series, and the IPS logo are trademarks of Invensys plc, its subsidiaries and
affiliates.
All other brand names may be trademarks of their respective owners.
Copyright 2004-2009 Invensys Systems, Inc.
All rights reserved
SOFTWARE LICENSE AND COPYRIGHT INFORMATION
Before using the Invensys Systems, Inc. supplied software supported by this documentation, you
should read and understand the following information concerning copyrighted software.
1. The license provisions in the software license for your system govern your obligations
and usage rights to the software described in this documentation. If any portion of
those license provisions is violated, Invensys Systems, Inc. will no longer provide you
with support services and assumes no further responsibilities for your system or its
operation.
2. All software issued by Invensys Systems, Inc. and copies of the software that you are
specifically permitted to make, are protected in accordance with Federal copyright
laws. It is illegal to make copies of any software media provided to you by
Invensys Systems, Inc. for any purpose other than those purposes mentioned in the
software license.
iii

Contents
Figures................................................................................................................................... vii
Tables..................................................................................................................................... ix
Preface.................................................................................................................................... xi
Purpose .................................................................................................................................... xi
Audience .................................................................................................................................. xi
Revision Information ............................................................................................................... xi
Reference Documents ............................................................................................................. xii
Terms and Definitions ........................................................................................................... xiii
1. Introduction...................................................................................................................... 1
Overview of The MESH Control Network Architecture ........................................................... 1
Switched Ethernet Characteristics ........................................................................................ 1
The MESH Control Network Features ................................................................................ 2
Standard Configuration Features ..................................................................................... 2
Security Enhanced Configuration Features ...................................................................... 2
Loop Detection Policy (LDP) Deployed on The MESH Control Network ..................... 3
Virtual Local Area Networks (VLANs) on The MESH Control Network ........................ 3
The MESH Control Network .............................................................................................. 4
The MESH Control Network Topologies ....................................................................... 4
The MESH Control Network ................................................................................................. 12
Network Example .............................................................................................................. 12
The MESH Control Network Specifications ...................................................................... 15
The MESH Control Network Workstations ...................................................................... 15
The MESH Control Network Ethernet Switches .................................................................... 16
Advantages of Invensys-Supplied Switches .......................................................................... 16
Control Network Cabling ....................................................................................................... 17
Category 5 Cabling ............................................................................................................ 18
Fiber Optic Cabling ........................................................................................................... 18
Single Mode Cable ........................................................................................................ 19
Multimode Cable .......................................................................................................... 19
The MESH Control Network Management Software Tool, NetSight
Console .................... 20
Netsight Policy Manager .................................................................................................... 20
Obtaining Network Management Software ........................................................................ 22
2. Site Planning................................................................................................................... 23
Site Planning Overview ........................................................................................................... 23
Network Considerations ..................................................................................................... 23
B0700AZ Rev L Contents
iv

Traffic Considerations ........................................................................................................ 26
Network Loading ............................................................................................................... 26
Equipment Considerations ................................................................................................. 27
Hardware Requirements ..................................................................................................... 28
Switch Utilization in the Standard Configurations ........................................................ 28
Switch Utilization in the Security Enhanced Configurations ......................................... 28
Firmware Considerations .................................................................................................... 29
The MESH Control Network Design Rules ............................................................................ 29
Standard Configuration Design Rules ................................................................................ 29
Non I/A Series Equipment ............................................................................................ 30
Security Enhanced Configuration Design Rules ................................................................. 30
Additional Guidelines for Planning Security Enhanced Configuration ............................... 34
Security Enhanced Configuration Design Considerations .............................................. 34
The MESH Control Network Design ..................................................................................... 36
I/O Network Design Rules ................................................................................................. 36
The MESH Control Network Topologies .......................................................................... 36
Standard Configurations ................................................................................................ 37
Security Enhanced Configurations ................................................................................. 52
I/O Network Topology Configurations ......................................................................... 58
Switch and Fiber Cable Budget and Loss ............................................................................ 59
Fiber Cable Budget Cable and Loss ............................................................................... 59
Fiber Optic Budgets ...................................................................................................... 59
The MESH Control Network Cabling ............................................................................... 59
Switch and Converter Fiber Optic Cabling .................................................................... 59
FCP270, ZCP270, FCM100Et and FCM100E Fiber Signal Cabling ............................ 63
Twisted-Pair Cabling ......................................................................................................... 66
Null Hub ...................................................................................................................... 66
3. Installation (Cabling) ...................................................................................................... 67
Connecting The MESH Control Network Components ......................................................... 67
Fiber Optic Cabling Guidelines .......................................................................................... 67
Interconnecting Ethernet Switches ..................................................................................... 67
Uplink Port to Uplink Port ........................................................................................... 68
RJ-45 Port to RJ-45 Port ............................................................................................... 70
Switch Configuration .............................................................................................................. 70
4. Maintenance.................................................................................................................... 71
The MESH Control Network Addresses ................................................................................. 71
General Troubleshooting Guidelines ....................................................................................... 72
Characterize the Problem ................................................................................................... 72
Determine Which Devices are Affected .............................................................................. 73
Troubleshoot the Affected Devices ..................................................................................... 73
System Management Displays ................................................................................................. 74
Accessing SMDH Switch Network Displays ....................................................................... 74
Switched Network Display ................................................................................................. 76
Switch Equipment Change Display ............................................................................... 77
Contents B0700AZ Rev L
v

Switch Equipment Information Display ........................................................................ 78
Switch Configuration Information Display .................................................................... 80
Switch Domain Display ..................................................................................................... 81
Switch Ports Display .......................................................................................................... 82
Switch Port Equipment Change Display ....................................................................... 83
Switch Port Equipment Information Display ................................................................ 84
Indicators ................................................................................................................................ 85
Ethernet Switches ............................................................................................................... 85
Media Converter ................................................................................................................ 85
Fiber Optic Cable Handling and Cleaning .............................................................................. 85
Handling Fiber Optic Cable ............................................................................................... 85
Cleaning Fiber Optic Cable ................................................................................................ 87
Contamination of Fiber Optic Connectors and Sockets ................................................. 87
Contamination Prevention ............................................................................................ 88
Contamination Removal ............................................................................................... 88
Examples of Fiber Optic Connector Cleaning Products ................................................. 89
5. Combining Two or More MESH Control Networks ...................................................... 91
Overview ................................................................................................................................. 91
Planning Stage .................................................................................................................... 91
Station Addressing .............................................................................................................. 92
Bridge Switch Functionality ............................................................................................... 92
Topology Constants ........................................................................................................... 92
Combining Star Network Topologies ..................................................................................... 93
Combining Star with Tree Network Topology ........................................................................ 95
Combining Tree Network Topologies .................................................................................... 96
Combining Ring Network Topologies .................................................................................... 97
Appendix A. COMEX Fault Handling on The MESH Control Network............................ 99
COMEX Layers ...................................................................................................................... 99
COMEX Applications Layer .............................................................................................. 99
Application Layer Protocol and Timers ......................................................................... 99
Transport Layer ................................................................................................................ 100
Transport Layer Protocol and Timers .......................................................................... 102
Network Layer ................................................................................................................. 103
Operation .................................................................................................................... 103
Logical Link Control Layer .............................................................................................. 103
Transmit Operation .................................................................................................... 103
Receive Operation ....................................................................................................... 103
MAC Layer ...................................................................................................................... 104
Appendix B. The MESH Network Fault Handling ........................................................... 105
LINK .................................................................................................................................... 105
PORT TEST Packets ......................................................................................................... 105
B0700AZ Rev L Contents
vi

LLC_PING Test ................................................................................................................ 106
ZCP-FCM Communications ................................................................................................ 107
DIAGNOSTIC Information ................................................................................................. 108
Index .................................................................................................................................. 111
vii

Figures
1-1. Linear Topology ............................................................................................................ 5
1-2. Ring Topology (Standard Configuration Only) ............................................................ 6
1-3. Star Topology ............................................................................................................... 8
1-4. Inverted Tree Topology .............................................................................................. 10
1-5. Modified Inverted Tree Topology ............................................................................... 11
1-6. The MESH Control Network (Star Topology Shown) ................................................ 13
1-7. The MESH Control Network with an I/O Network ................................................... 14
2-1. Large Network (Security Enhanced Configuration) ..................................................... 33
2-2. Small Network (Standard Configuration) ................................................................... 38
2-3. Medium Network (Ring) (Standard Configuration) .................................................... 39
2-4. Medium or Large Network (Star) (Standard Configuration) ....................................... 40
2-5. Medium or Large Network (Double Star) (Standard Configuration) .......................... 41
2-6. Large Network - All Blades (Standard Configuration) ................................................. 42
2-7. Large Network - Inverted Stepped Tier Tree - All Blades (Standard Configuration) .... 44
2-8. Large Network - Blades and Low-Cost Switches (Standard Configuration) ................. 45
2-9. The MESH Control Network Tiers ............................................................................ 47
2-10. Root Switches Connected ........................................................................................... 48
2-11. Second Tier Connections ............................................................................................ 49
2-12. Third Tier Connections .............................................................................................. 50
2-13. Fourth Tier Connections ............................................................................................ 50
2-14. Labeling the Switches .................................................................................................. 51
2-15. Small Network (Security Enhanced Configuration) .................................................... 52
2-16. Star Topology (Security Enhanced Configuration) ...................................................... 53
2-17. Double Star Topology (Security Enhanced Configuration) ......................................... 54
2-18. Inverted Tree Topology (Security Enhanced Configuration) ....................................... 56
2-19. Modified Inverted Tree Topology (Security Enhanced Configuration) ....................... 58
2-20. Switch to Switch and Switch to Patch Panel Connections ........................................... 63
2-21. FCP270, ZCP270, FCM100Et and FCM100E Signal Cabling .................................. 65
3-1. Switch-to-Switch Fiber via Uplink Port ....................................................................... 69
3-2. Port-to-Port Connection via Fiber Optic Ports ........................................................... 69
3-3. Connecting Switches via RJ-45 Ports .......................................................................... 70
4-1. Accessing SMDH Switched Network Displays ............................................................ 75
4-2. SMDH Switched Network Display ............................................................................. 76
4-3. Switch Equipment Change Display ............................................................................. 77
4-4. Switch Equipment Information Display ...................................................................... 78
4-5. Switch Configuration Information Display ................................................................. 80
4-6. Switch Domain Display .............................................................................................. 81
4-7. Switch Ports Display - Typical .................................................................................... 82
4-8. Switch Equipment Change Display ............................................................................. 83
4-9. Switch Port Equipment Information Display - Typical ............................................... 84
4-10. SC Connector, Typical ............................................................................................... 86
4-11. Multimode MT-RJ Connector .................................................................................... 87
4-12. Multimode Duplex LC Connector .............................................................................. 87
4-13. Lint in Fiber Optic LC Socket .................................................................................... 88
B0700AZ Rev L Figures
viii

5-1. Combining Two Star Network Topologies Into One Tree Network Topology
(Before) ....................................................................................................................... 93
5-2. Combining Two Star Network Topologies Into One Tree Network Topology (After) 93
5-3. Combining a Star Network Topology with a Tree Network Topology Into One Tree
Network Topology (Before) ........................................................................................ 95
5-4. Combining a Star Network Topology with a Tree Network Topology Into One Tree
Network Topology (After) .......................................................................................... 95
5-5. Combining Two Tree Network Topologies Into One Tree Network Topology
(Before) ....................................................................................................................... 96
5-6. Combining Two Tree Network Topologies Into One Tree Network Topology
(After) ......................................................................................................................... 96
5-7. Combining Two (4) Tier Tree Network Topologies Into One (4) Tier Tree Network
Topology (After) ......................................................................................................... 97
5-8. Combining Ring Network Topologies ........................................................................ 98
ix

Tables
1-1. The MESH Control Network Specifications ............................................................... 15
1-2. NetSight Console Policies ........................................................................................... 20
1-3. NetSight Client Policy ................................................................................................ 21
1-4. NetSight Policy Bundle ............................................................................................... 21
1-5. NetSight Advanced Bundle ......................................................................................... 21
1-6. NetSight Single User Policy ........................................................................................ 22
1-7. NetSight A-La-Carte Plug-Ins ..................................................................................... 22
1-8. NetSight Appliance Policy ........................................................................................... 22
2-1. Fiber Cable Power Losses ............................................................................................ 59
2-2. Multimode Fiber Cables with LC Connectors ............................................................. 60
2-3. Multimode Fiber Cables with MT-RJ to ST Connectors ............................................ 60
2-4. Multimode Fiber Cable with LC and SC Connectors ................................................. 61
2-5. Copper Cables with RJ-45 Connectors ....................................................................... 61
2-6. Single Mode Fiber Optic Cable - Maximum Transmission ......................................... 61
2-7. Single Mode Fiber Optic Jumper Cables ..................................................................... 62
2-8. Fiber Optic Cables ...................................................................................................... 64
2-9. CAT5 Cable - Maximum Transmission Distance ........................................................ 66
2-10. Prefabricated CAT5 STP Cables with RJ-45 Connectors ............................................ 66
2-11. Null Hub .................................................................................................................... 66
3-1. Methods of Connecting Ethernet Switches ................................................................. 68
4-1. IP Address Assignments ............................................................................................... 71
4-2. Switch Equipment Change Display Actions ................................................................ 77
4-3. Switch Equipment Information Display Fields ............................................................ 78
4-4. Switch Configuration Information Display Fields ....................................................... 80
4-5. Switch Equipment Change Actions ............................................................................. 83
4-6. Switch Port Equipment Information Display Fields .................................................... 84
A-1. Out of Sequence DT and Ack Packets ...................................................................... 101
B0700AZ Rev L Tables
x

xi

Preface
Purpose
This document provides overview guidelines and requirements for designing, installing, and
maintaining The MESH control network. Topics include:
Site Planning
Installation
Maintenance.
Additional documentation provides the information specific to the hardware for The MESH
control network. These books are listed in Reference Documents below.
For detailed and specific information on the Ethernet equipment, refer to the documentation sup-
plied by the switch vendor. These documents may also be available on the IPS Global Client Sup-
port web site at http://support.ips.invensys.com. The MESH control network documents for
I/A Series systems are available on The MESH Network Configuration Tool CD-ROM
(K0173ZU).
Audience
This book is intended for use by process engineering, operations, installation, and maintenance
personnel. They are expected to have a working knowledge of Ethernet LANs and I/A Series con-
figurations.
Revision Information
For this revision of the document (B0700AZ-L), the following changes were made:
Chapter 1 Introduction
Significantly expanded Network Example on page 12.
Updated references to Invensys in Advantages of Invensys-Supplied Switches on
page 16.
Updated Figure 1-6 The MESH Control Network (Star Topology Shown) on
page 13.
Updated The MESH Control Network Management Software Tool, NetSight

Console on page 20 to reflect the NetSight products currently offered.
Chapter 2 Site Planning
Minor edits to Switch Utilization in the Standard Configurations on page 28.
Added I/O Network Design Rules on page 36.
Added I/O Network Topology Configurations on page 58.
B0700AZ Rev L Preface
xii

Reference Documents
The following documents provide additional or related information to the hardware used in The
MESH control network:
A-Series (P0973BH/P0973BJ/P0973BK) Switches, Hardware and Software Configura-
tion Instructions (B0700CH)
The MESH Control Network Hardware Instructions for C-Series Switches
(P0973BL/HA) (B0700CJ)
The MESH Control Network Hardware Instructions for N-Series Switches
(P0973AR/P0973AS/P0972YE) (B0700CK)
V-Series (P0972WP/P0972YC) Switches, Hardware and Software Configuration Instruc-
tions (B0700CL)
E7 Chassis and 16-port Fiber (P0972MK/P0972MJ) Switches, Hardware and Software
Configuration Instructions (B0700CM)
I-Series (P0973GB) Industrial Switch, Hardware and Software Configuration Instruc-
tions (B0700CN)
Media Converter Installation and Configuration Guide for Control Networks (B0700CP)
The following documents provide additional or related information to The MESH control
network concepts:
The MESH Control Network Operation, and Switch Installation and Configuration
Guide (B0700CA)
The MESH Control Network Architecture (PSS 21H-7C2 B3)
The MESH Control Network Ethernet Equipment (PSS 21H-7C3 B4)
I/A Series System Definition: A Step-by-Step Procedure (B0193WQ)
I/A Series Configuration Component (IACC) Users Guide (B0400BP).
Preface B0700AZ Rev L
xiii

Terms and Definitions
10Base-T 10 Mb twisted-pair Ethernet
100Base-TX 100 Mb twisted-pair Fast Ethernet
100Base-FX 100 Mb fiber optic Fast Ethernet
1000Base-LX IEEE 802.3z specification for Gigabit Ethernet over two strands of
50/125 or 62.5/125 micron core MMF or 9/125 micron core SMF fiber
cable using long wavelength optical transmission.
1000Base-SX IEEE 802.3z specification for Gigabit Ethernet over two strands of
50/125 or 62.5/125 micron core MMF fiber cable using short wavelength
optical transmission.
1000Base-ZX IEEE 802.3z specification for Gigabit Ethernet over two strands of 9/125
micron core SMF fiber cable using 1550 nm wavelength optical transmis-
sion.
1000Base-T IEEE 802.3ab specification for Gigabit Ethernet using CAT5 copper
Ethernet cable.
ANSI American National Standards Institute
Auto-Negotiation Signalling method allowing each node to select its optimum operational
mode (e.g., speed and duplex mode) based on the capabilities of the node
to which it is connected.
Backbone Another term for bus - refers to the main link that connects network
nodes. The term is often used to describe the main network connections
composing the network.
Beacon The packet type and packet on the network upon which the port disabling
is enacted.
BPP Beacon Priority Policy - A role/service that allows for the Beacon packet to
have the highest priority when propagating though the network. This
ensures the Beacon packet will be transmitted back to the PBQ in a
flooded switch.
BootP Bootstrap Protocol
Bridge Priority
Value
The range of priority values used to determine which device is selected as
the Spanning Tree root. This value can range from 0- 65535 for bridge
priority mode 802.1d (decrement by 1) or from 0-61440 for bridge prior-
ity mode 802.1t (decrement by 4096).
CAT5 Category 5 Twisted Pair Cable - such as 10Base-T, 100Base-TX and
1000Base-T.
xiv

CBP (Circuit Breaker Policy) a role/service that disables a port when a Beacon
packet is received from an edge switch or from the tier below.
Circuit Breaker A policy rule that will disable a port that receives an incoming packet of an
outgoing Beacon packet Loop.
Circuit Breaker
PBQ/SBQ
(CBPBQ/CBSBQ)
This is used to refer to policy rule that will disable an uplink port that
interfaces two root switches that receives an incoming packet of an outgo-
ing Beacon packet Loop. This function is a subset of the Circuit
Breaker.
CLI Command Line Interface
Core Switch Refers to the main body of switches that provide the network with its
backbone connections. A core switch can also be considered an edge
switch in reference to the root; however the outer most edge switches
within the network are normally not considered to be core switches.
CRC Cyclic Redundancy Check
CSMA/CD Carrier Sense Multiple Access/Collision Detection
Data Loop or Loop
Path
Refers to a condition where data traverses a redundant path with no
termination point.
DCE Data Communications Equipment (modem)
DSR Data Set Ready
DTE Data Terminal Equipment
DTR Data Terminal Ready
Edge Switch Refers to an outer switch in a network topology that is linked to the pri-
mary root or backup root bridge switch directly in one to two tier
configurations, and indirectly in three to four tier configurations.
ESD Electrostatic Discharge
FCS Frame Check Sequence
Fast Ethernet (FE) Set of Ethernet standards that carry traffic at the nominal rate of 100 Mbit
per second.
FTM Frame Transfer Matrix
Full Duplex Transmission method that allows two network devices to transmit and
receive concurrently, effectively doubling the bandwidth of that link.
GBIC Gigabit Interface Converter
HTTP Hypertext Transfer Protocol
Preface B0700AZ Rev L
xv

ICMP Internet Control Message Protocol
IEEE Institute of Electrical and Electronics Engineers
IEEE 802.3 Defines carrier sense multiple access with collision detection (CSMA/CD)
access method and physical layer specifications.
IEEE 802.3ab Defines a media access method and physical layer specifications for
1000Base-T Gigabit Ethernet.
IEEE 802.3u Defines a media access method and physical layer specifications for
100Base-TX Fast Ethernet over CAT5 cable.
IEEE 802.3x Defines Ethernet frame start/stop requests and timers used for paused flow
control on full-duplex links.
IEEE 802.3z Defines a media access method and physical layer specifications for
1000Base Gigabit Ethernet.
IGMP Internet Group Management Protocol, used to establish host member-
ships in particular multicast groups on a single network.
IOC Input/Output Controller (part of the Z-Module Control Processor
(ZCP270))
IOM Input/Output Module
IP Internet Protocol
LAN Local Area Network
LDP Loop Detection Policy (Refer to The MESH Control Network Operation,
and Switch Installation and Configuration Guide (B0700CA) for more
information.)
LED Light Emitting Diode
MAC Media Access Control
MDI Media Dependent Interface or Media Device Interface
MIB Management Information Base
MMF Multi-mode Fiber cable
NEM Network Expansion Module
PBQ Primary Beacon Queryer - The switch with the lowest IP address and with
the IGMP Beacon enabled.
Policy A group of rules which a network device uses to make forwarding, block-
ing or port-disable decisions.
xvi

RFC Request for Comment
RMON Remote Monitoring
Role A collection of services
RSTP Rapid Spanning-Tree Protocol (IEEE 802.1w standard)
Rule Hit An action when a packet classifier finds the packet.
Rules Packet classifiers that are used to identify packet types on the network.
RXD Receive Data
SBQ Secondary Beacon Query - The switch with the second lowest IP address
and with the IGMP Beacon enabled.
Service A collection of Rules
SFP Small Form Factor Pluggable (Type of Mini-GBIC)
SMF Single-mode Fiber cable
SNMP Simple Network Management Protocol
STP Spanning-Tree Protocol (IEEE 802.1d standard)
TCP/IP Transmission Control Protocol/Internet Protocol
TDR Transient Data Recorder
TFTP Trivial File Transfer Protocol
TXD Transmit Data
UTP Unshielded Twisted Pair
VLAN Virtual Local Area Network
1

1. Introduction
This chapter provides an introduction to the concepts and equipment used in The MESH control
network.
Overview of The MESH Control Network
Architecture
The MESH control network is a switched Fast Ethernet network based on IEEE 802.3u (Fast
Ethernet) and IEEE 802.3z (gigabit Ethernet) standards. The MESH control network consists of
a number of Ethernet switches connected in a MESH configuration.
The MESH control network configuration allows high availability by providing redundant data
paths and eliminating single points of failure caused by component link failures. The flexibility of
the architecture allows you to design a network configuration that fits the needs of the control sys-
tem. Configurations can be as simple as a workstation and controller connected with a single pair
of switches, or as complex as a multi-switch, fully meshed control network, communicating at
speeds up to 1 gigabit per second.
The MESH control network architecture integrates powerful control stations and workstations in
a 100 Mb/1 Gb Ethernet network. These control stations, workstations and networks comprise
scalable systems for process monitoring, process control and integration with industrial informa-
tion management systems.
High speed, coupled with redundancy and peer-to-peer characteristics, provides high perfor-
mance and superior security. Station interfaces to redundant Ethernet switches ensure secure
communications between the stations. Station interfaces can use single paths but this compro-
mises the security of The MESH control network.
NOTE
All graphics of switches and media converters in this document are intended as
generic illustrations of networking concepts and do not necessarily reflect the cur-
rently offered products.
Switched Ethernet Characteristics
Standard Fast Ethernet switches and fiber optic/copper cabling provide versatile solutions for
building MESH networks using industry standard protocols. The 16-port or larger managed
Ethernet switches used in The MESH control network allow connection of multiple control sta-
tions, workstations and other Ethernet switches. Unmanaged switches are not supported by The
MESH control network because they offer no redundancy and you cannot run diagnostics if the
switch should fail.
B0700AZ Rev L 1. Introduction
2

The MESH Control Network Features
Two types of network configuration methods are available in The MESH control network -
Standard Configuration and Security Enhanced Configuration.
Standard Configuration Features
The Standard Configuration of The MESH control network provides the following features.
System scalability by interconnecting Ethernet switches with 16-ports or more in a
linear, ring, star, inverted tree or modified inverted tree network topology (configura-
tion). The topology is dependent on the network site requirements.
Ethernet switches connected in a MESH configuration with up to 1920 I/A Series
stations
Support for Fast Ethernet (100 Mb) and Gigabit (uplink only) Ethernet (1000 Mb)
Modular uplinks to high-speed backbones using 1 Gb 1000Base-T, 1000Base-SX,
1000Base-LX and 1000Base-ZX standards
Full-duplex operation based on the IEEE 802.3 standards
Rapid Spanning Tree Protocol (RSTP - IEEE 802.1w) which manages redundant
paths, prevents loops, and provides high speed convergence time for a network
Network management and configuration via local port or Web access for various
switches
System Management software for monitoring the health of the control system and
managing equipment in the system
Software in every station that manages redundant Ethernet ports in response to net-
work faults
High speed response to network and station faults to provide a highly reliable redun-
dant network.
Security Enhanced Configuration Features
The Security Enhanced Configuration is now available to provide additional features. Due to the
recent advances in switch network technologies, the Security Enhanced Configuration of The
MESH control network offers improvements in network security, Loop Detection and many
additional features not offered in the Standard Configuration.
The Security Enhanced Configuration deploys specific network topologies and switch configura-
tions that allow for advanced network loop detection in the event of a RSTP failure. This
advanced network loop detection minimizes the potential for a single point of failure that will
degrade communications between devices in the network.
Deploying the advanced network loop detection is accomplished by carefully designing the net-
work and correctly deploying the Loop Detection Policy (LDP) algorithms required for the net-
work design, and by following the network configuration requirements provided in this
document.
1. Introduction B0700AZ Rev L
3

NOTE
When designing a Security Enhanced Configuration network, each device/switch
within the network is required to be connected to two different switches within the
network. If the network is constructed with less than two connections between
devices/switches, data traffic could be disrupted if any single device fails or there is a
fault that causes the loop detection algorithm Rule Hit.
Loop Detection Policy (LDP) Deployed on The MESH Control Network
Due to the design of the Standard or Security Enhanced Configurations of The MESH control
network, redundant links form physical loops in the network and are controlled (Blocked) by
Rapid Spanning Tree Protocol (RSTP), creating a logical loop-free network. In a Security
Enhanced Configuration in addition to RSTP, Loop Detection (LDP) is deployed to block
redundant loops that could occur in the event of an RSTP or Data Loop (storm) failure.
A loop is determined by establishing a well-known data path and its source. To establish a known
path, the concept known as the Beacon is developed. The Beacon routinely sends out an IGMP
data packet. When the data packet is seen at an unexpected source port, the assumption is that a
loop occurred and an action (Rule) needs to occur. A switchs port deployed with Circuit
Breaker will disable the first port on which the incorrectly sourced packet is received.
Recent chassis switches, such as the N-Series Platinum (DFE) switches (P0973BQ, P0973BR,
P0973BT, and P0973BS), offer advanced packet switching services that can scope data packets
beyond the source and destination MAC-address. By looking at other data points in the packets,
the switch can make decisions on which of these data points to mark a packet on. Once a particu-
lar packet is identified, the switch can take action on it. The action of interest is disabling a looped
port. Disabling this looping port maintains a loop-free network. The switches alert the network
administrator with SNMP traps and syslog messages. These should be acted upon to 'fix' the net-
work loop in a timely manner. When disabled by the LDP, a disabled port can be monitored by
SMDH via a link down trap. Other methods of monitoring and management of ports can be
accomplished by utilizing the switch's CLI port or NetSight Policy Manager.
Virtual Local Area Networks (VLANs) on The MESH Control Network
A VLAN acts like an ordinary Local Area Network (LAN), but in a VLAN, connected devices do
not have to be physically connected to the same segment. The VLAN allows devices located in
separate areas or connected to separate ports to belong to a single VLAN group. Devices that are
assigned to such a group will send and receive broadcast and multicast traffic as though they were
all connected to a common network. VLAN-aware switches isolate broadcast, multicast, and
unknown traffic received from VLAN groups, so that traffic from stations in a VLAN are con-
fined to that VLAN.
Additional details about VLANs are provided in The MESH Control Network Operation, and
Switch Installation and Configuration Guide (B0700CA revision F or later), in Chapter 10
VLANs Usage on The MESH Control Networks and Appendix D Understanding Virtual
Local Area Networks (VLANs).
4

The MESH Control Network
The MESH control network is designed to provide multiple communication paths between any
two devices or stations connected to the network. This network architecture provides very high
availability, while reducing network complexity, cost, and maintenance requirements.
The MESH Control Network Topologies
There are several basic MESH control network topologies supported by the I/A Series

system in
each of the Standard and Security Enhanced Configurations. These are:
Standard Configuration
Linear
Ring
Star
Inverted tree
Modified inverted tree
Security Enhanced Configuration
Linear
Star
Inverted tree
NOTE
When deploying the Security Enhanced Configuration, The MESH control net-
work should be constructed using one of the four enhanced topologies listed above.
The ring topology should never be used when deploying this configuration.
Each configuration/topology listed has unique features and the one chosen for a particular net-
work depends on the specific requirements of the site or installation. The following diagrams pro-
vide examples of the different topologies as well as recommendations on where they might be
used.
5

Linear Configuration
The linear configuration is appropriate for a small, two switch network, as shown in Figure 1-1.
In this example a failure of any one component in The MESH control network does not affect the
operation of the remaining components. The linear configuration does not require a root or back-
up switch configuration. Large chassis switches with hundreds of ports can be used in a linear
configuration to create a large control system. A multiple switch linear configuration (more than
two) is not supported due to its lack of network redundancy, which can result in loss of communi-
cation between two devices within the network.

Figure 1-1. Linear Topology
6

Ring Topology
This topology is suited to networks containing three to seven switches. As shown in Figure 1-2,
each switch has connections to the two adjacent switches. In the event of a failed switch, the ring
is broken and the network assumes the characteristics of the linear topology shown in Figure 1-1.
There is a limit of seven (7) switches between any two devices on the network. This limit is
imposed by the I/A Series system and cannot be exceeded.
Figure 1-2 illustrates a network composed of six managed switches configured in a ring. A net-
work in this configuration is able to handle a single component failure and still maintain its integ-
rity. The ring configuration does not require a root or backup root switch configuration.
The ring topology cannot be configured with the Security Enhanced Configuration. When
LDP is deployed in a ring topology, the loop detection algorithm will run before RSTP can
block the port resulting in an uplink port being disabled. The end result will produce a multi-
ple switch linear topology which is not a supported configuration. When designing Security
Enhanced Configurations, use one of the four supported topologies (on page 4).
Figure 1-2. Ring Topology (Standard Configuration Only)
7

Star Topology
The star topology is the preferred topology for all networks, provided that geographical con-
straints allow this topology. In a star topology, the switches at the outside edge of the network
have connections to each of the two root switches. The root switches are connected to each other
and the edge switches. Redundant data paths allow the network to continue to operate if any one
component fails.
Figure 1-3 illustrates a star network containing ten edge switches and two root switches. In a Stan-
dard Configuration star topology utilizing Gold Series blades, as many as 40 edge switches can be
connected to the Chassis switch using 1 Gb uplinks. When utilizing Platinum Series blades, as
many as 166 edge switches can be utilized.
For a Security Enhanced Configuration star topology (Policy enabled switches only) a maximum
of 166 edge switches can be connected to the Chassis switch using 1 Gb uplinks. An inverted tree
topology can be considered if there is a larger number of edge switches required (up to 250
switches).
Refer to the table Qualified Switch Standard/ Security Enhanced Configuration Compatibilities
Matrix in The MESH Control Network Operation, and Switch Installation and Configuration
Guide (B0700CA) for a list of switches capabilities.
8

Figure 1-3. Star Topology
9

Inverted Tree Topology
The inverted tree topology is suited to very large networks with specific physical constraints. In
this topology the switches are arranged in tiers, with the root switches in the top tier and up to
three tiers below them. The root switches (Tier 1) are the only switches in the network that have
connections between switches on the same tier; all other switches have two connections to
switches in the tiers above them. This topology is supported in both the Standard and Security
Enhanced Configurations.
When deployed in a Security Enhanced Configuration, all switches within the network must be
switch types that support Loop Detection (LDP). Refer to the table Qualified Switch Stan-
dard/Security Enhanced Configuration Compatibilities Matrix in The MESH Control Network
Operation, and Switch Installation and Configuration Guide (B0700CA) for a list of the switches
applicable to either of these configurations.
An inverted tree network topology is illustrated in Figure 1-4.There can be no more than four-
tiers of switches (including the root) in order to comply with the I/A Series system requirement
limiting the number of switches between devices to seven.
10

Figure 1-4. Inverted Tree Topology
Modified Inverted Tree Topology
The modified inverted tree topology is suited for very large enhanced networks with specific phys-
ical constraints and requirements. The modified inverted tree topology allows for standard quali-
fied (lower cost) switches to be utilized at the outer edge, which allows for larger networks to be
deployed at a lower cost. In this topology, the switches are arranged in tiers, with the root switches
in the top tier with up to three tiers below them. The root switches (Tier 1) are the only switches
in the network that have connections between switches on the same tier; all other switches have
two connections to switches in the tiers above them. All outer edge switches are interfaced to the
network on different tiers (e.g. tier 3 and tier 4). By doing this, all end devices with redundant
Note: Primary and Backup
Root Switches have two
connections.
11

connections interface to the network on different tiers. A modified inverted tree network topol-
ogy is illustrated in Figure 1-5.
There can be no more than four-tiers of switches (including the root) in order to comply with the
I/A Series system requirement limiting the number of switches between devices to seven.
Figure 1-5. Modified Inverted Tree Topology
12

The MESH Control Network
Network Example
The MESH control network utilizes qualified Fast Ethernet switches which are configured to
form a highly robust redundant network. Figure 1-6 on page 13 shows an example with several
I/A Series stations and Control Processors connected redundantly to The MESH control
network.
Workstations are also redundantly connected to the Fast Ethernet switches.
The Z-module Control Processor 270 (ZCP270) main processors Ethernet ports connect to The
MESH control network, where the ZCP270 I/O controller (IOC) ports have the option of con-
necting directly to The MESH control network or a dedicated I/O network.
The Fieldbus Communications Module (FCM100Et or FCM100E) provides the interface
between the ZCP270 and the FBMs and connects to The MESH control network or a dedicated
I/O network.
When designing The MESH control network architecture, the following concerns should be
addressed:
For critical I/O communications, which in this context can be defined as I/O applica-
tions that cannot allow for a disruption of fieldbus communications over a five second
span, it is recommended that the ZCP270 IOC and FCM I/O be placed on a separate
network.
For non-critical I/O communications, the FCM100s and ZCP IOC ports can con-
nect directly to The MESH. Whenever possible, it is recommended that the ZCP
IOC ports and FCMs be attached to the same switches within the network, keeping
the ZCP IOC to FCM communications local to the switch. (See Figure 1-6.)
If a dedicated I/O network is to be employed, a simple linear topology is used and switch selection
should be per the standard guidelines for The MESH, where the ZCP270 I/O controller (IOC)
ports A/B and FCM I/O ports A/B (Fieldbus A and Fieldbus B) A side and B side are separated
on independent networks. It is also possible to have multiple I/O networks, such as having an I/O
network for each ZCP IOC and its FCMs.
WARNING
!
Switches on a dedicated I/O network cannot be seen in SMDH or System Manager.
However, Fieldbus A and Fieldbus B errors are indicated in a normal manner via
the Sys Key and are visible in SMDH or System Manager.
Also, a dedicated I/O network cannot be used for FBMs using the GPS SOE/TDR
time sync package (see Time Synchronization User's Guide (B0700AQ)).
For more information on configuring a dedicated I/O network, refer to I/O Network Design
Rules on page 36 and I/O Network Topology Configurations on page 58.
13

Figure 1-6. The MESH Control Network (Star Topology Shown)
The MESH
ETHERNET
SWITCHES
ZCP270
FCP270
TO/FROM
PROCESS
CONTROL NETWORK
TO/FROM
PROCESS
DIN RAIL
FCM100Et
WORKSTATIONS
INFORMATION NETWORK
BASEPLATE
SPLITTER/COMBINER
FBM
FBM
(NOT USED WITH FCM100E)
SPLITTER/
COMBINERS
DIN RAIL
BASEPLATE
or FCM100E
14

Figure 1-7. The MESH Control Network with an I/O Network
ZCP270
SPLITTER/
COMBINERS
TO/FROM
PROCESS
DIN RAIL
FCM100Et
BASEPLATE
SPLITTER/COMBINER
FBM
(NOT USED WITH FCM100E)
or FCM100E
The MESH
CONTROL NETWORK
I/O NETWORK
A
B
15

The MESH Control Network Specifications
For more information, refer to the following Product Specifications Sheets:
The MESH Control Network Architecture (PSS 21H-7C2 B3)
The MESH Control Network Ethernet Equipment (PSS 21H-7C3 B4).
The MESH Control Network Workstations
I/A Series workstations can be connected to The MESH control network. These workstations
provide host services to fault-tolerant control processors. In addition, the workstation provides
the operator interface for the display of graphic and textual information.
Each workstation connects to the switches in The MESH control network by way of copper or
fiber interface cards in the workstation. Two Network Interface Cards (NIC) are offered:
100Base-FX
10/100Base-TX
The workstation can be directly connected to Ethernet switches or through the use of a media
converter, described in Media Converter Installation and Configuration Guide for Control Networks
(B0700CP). In addition, the built-in Ethernet interface on the workstations motherboard can be
used to interface to a plant information network.
Table 1-1. The MESH Control Network Specifications
Specifications
Number of I/A Series stations connected to
The MESH control network
1,920 I/A Series stations including switches, maximum (FCM100Ets
and FCM100Es are not included in count). Up to 250 switches.
Number of Ethernet switches between any
two stations
Seven maximum
Number of IP addresses 10,000 maximum includes switches, controllers, workstations and
FCM100Ets and FCM100Es.
VLANs on The MESH control network Six (6) configurable VLANs are supported on The MESH Control
Network
Standards Supported 100 Mb Full-duplex operation for fiber optic and copper cable.
Modular uplinks using 1 Gb 1000Base-T, 1000Base-SX and
1000Base-LX standards
Speeds Supported Fast Ethernet (100 Mb) and uplink Gigabit Ethernet (1000 Mb)
Protocols Used Rapid Spanning Tree Protocol (RSTP - IEEE 802.1w), 802.3,
802.3ad
Cable Lengths - Interconnecting stations or
Ethernet switches
CAT5:100Base-TX or 1000Base-T; 100 m (328 ft) maximum
Fiber optic:
100Base-FX; 2 km (6,560 ft) maximum (MMF)
1000Base-SX; 275 m (900 ft) maximum (MMF)
1000Base-LX/LH; 2 km (6,560 ft) maximum (MMF)
1000Base-LX; 10 km (6.2 mi) maximum (SMF)
1000Base-ZX; SMF, 80 km (49.7 miles) maximum
Cable Lengths - Total connection length
allowed between switches
Single mode fiber (SMF), 80 km (49.7 mi) maximum
Multimode fiber (MMF), 2 km (6,560 ft) maximum
MESH Control Network Distance It is possible to extend The MESH control network distance using
various third-party network equipment and extenders. It should be
noted that the total network delays between two end devices should
not exceed 100ms roundtrip.
16

The MESH Control Network Ethernet Switches
A switch is an active multiport network and bridge device that provides a separate collision
domain for each port, and uses Media Access Control (MAC) layer to direct network packets to
the appropriate station or switch. This allows multiple simultaneous communications among net-
work devices connected to the switch.
The MESH control network utilizes standard commercial off-the-shelf Ethernet switches to allow
you to configure your system to meet your functional, performance and plant requirements.
Switches listed in Reference Documents on page xii have been tested and qualified by Invensys
for use with I/A Series products. Other Fast Ethernet switches from other vendors may be allowed
to be on The MESH control network. Using non-qualified switches may cause unpredictable fail-
ures or responses. The list of vendors and their qualified switches can be obtained from the Global
Client Support Center (Global CSC) web site at http://support.ips.invensys.com.
Refer to the documentation included with your Invensys qualified Ethernet switches for details of
their capabilities.
NOTE
The MESH control network was designed and tested for operation with the Ether-
net switches listed in Reference Documents on page xii. The network may operate
with similar, off-the-shelf equipment, but Invensys Systems, Inc. is not responsible
for any system malfunctions that may occur if such equipment is used.
If you use your own network:
1. You must meet the bandwidth requirements for the I/A Series equipment you
have chosen (1 Gb for uplinks and 100 Mb for ports).
2. There can be no Layer 3 inter-network devices (for example, routers) between
any I/A Series equipment.
3. Typically a failover time of less than 1 second is achievable using Fast Ethernet
switches qualified and supplied by Invensys and configured in accordance with
I/A Series documentation.
The network management module for the Fast Ethernet switches, provides a menu-driven or
Command Line Interface (CLI) system configuration program with management capability. See
the documentation included with your Invensys qualified Ethernet switches and the vendor man-
uals listed in Reference Documents on page xii for further details.
Advantages of Invensys-Supplied Switches
The MESH control network requires switches purchased from Invensys. This provides a number
of advantages, described below.
Can customers include Ethernet switches from other suppliers for use in The MESH
control network?
Only Invensys-supplied Ethernet switches have been qualified for use in the
I/A Series system. Other switches may or may not work and may not meet the
performance specifications required for a secure, reliable high performance in The
MESH control network.
17

Why won't other switches provide the same performance?
The Invensys-supplied switch is off the shelf equipment and has been optimized
to meet the stringent performance specifications for The MESH network. For
example, the Rapid Spanning Tree Protocol algorithm that manages traffic paths
is optimized to meet the requirement for network recovery on failure of a root
switch in less than one second. This level of performance is much better than is
typically achievable with other vendor's hardware and RSTP implementation.
Can customers purchase switches from third parties with the correct certified firm-
ware to work with the I/A Series system?
Although switches qualified for use with I/A Series systems can also be purchased
from third parties, they will not necessarily have been furnished with the correct
firmware version that has been qualified. The I/A Series Switch Configurator
Application Software (SCAS), supplied with the Invensys-supplied switch, is also
designed to configure switches qualified by Invensys, making configuration easier,
quicker, more reliable and facilitates troubleshooting and configuration verifica-
tion.
How could a user get the correct firmware?
They would need to contact Invensys and either purchase replacement firmware
for each switch on a one time charge basis or purchase an Invensys support con-
tract which would cover the cost of replacement firmware.
Is the firmware provided by Invensys for The MESH network switches exclusive to
Invensys-supplied switches?
Changes to the firmware used in I/A Series systems are included in the standard
product.
Periodically, changes are made to the firmware; future versions of the firmware
which have not been qualified by Invensys may or may not be compatible with
I/A Series systems.
Control Network Cabling
Three different types of cabling of various lengths may be used:
Shielded twisted-pair 100Base-T CAT5 cabling - 100 m (328 ft.) maximum
Shielded twisted-pair 1000Base-T CAT5 cabling - 100 m (328 ft.) maximum
Multimode fiber optic cabling - 2 km (1.25 mi) maximum for 100Base-FX
Multimode fiber optic cabling - 275 m (900 ft) maximum for 1000Base-SX
Multimode fiber optic cabling - 550 m (1800 ft) maximum for 1000Base-LX
Multimode fiber optic cabling - 2 km (1.25 mi) maximum for 1000Base-LX/LH
Single mode fiber optic cabling - 10 km (6.21 mi) maximum for 1000Base-LX.
Single mode fiber optic cabling - 80 km (49.68 mi) maximum for
1000Base-ZX/ELX.
18

Category 5 Cabling
There are two basic configurations of Category 5 (CAT5) cables:
Straight-through cables: cables used to connect workstations to Ethernet switches, and
media converters to Ethernet switches.
Crossover cables (also called a null hub): cables used to interconnect Ethernet
switches.
Some switches have auto MDI/MDI-X ports (for example, 24-Port Copper) which provides the
crossover and do not require a crossover cable.
NOTE
All category 5 Cabling must be of shielded type for optimal interference mitigation.
Normally, when one switch is connected to another, the transmit and receive wires must be
crossed over, such that the transmit wires from switch #1 connects to the receive wires from
switch #2, and vice versa. Crossover cables are used much less frequently than straight-through
cables. A straight through cable can be used as a crossover cable if used in conjunction with a null
hub cable adapter (P0971PK).
A null hub is a very short cable that has a male RJ-45 connector on one end and a female RJ-45
connector on the other. The transmit and receive wires are reversed, so when it is connected to
the end of a straight-through cable, the resulting cable system can act as a crossover cable. The
null hub is used to interconnect switches using straight-through CAT5 when neither switch is
equipped with auto MDI/MDI-X ports (crossover port).
Fiber Optic Cabling
Fiber optic cable is used to connect workstations to Ethernet switches and to make connections
between Ethernet switches. The fiber optic cables electrical isolation characteristics provide pro-
tection from voltage differentials and ground loops and permit communication installations to
pass through areas where intrinsically safe operation is required. The fiber optic cable is unaf-
fected by electrical noise such as EMI and RFI and can be installed even in the following cases:
Areas containing rotating machinery, arc welders, and so forth
Cable trays containing high voltage power lines
Outdoor areas exposed to lightning hazards (with appropriately rated cable)
Areas containing strong magnetic fields
Longer distances than twisted pair cable.
Two different types of fiber optic cable may be used in The MESH control network:
Single mode cable
Multimode cable (switch-to-device connections and uplink connections).
19

NOTE
Single mode and multimode fiber optical devices are not compatible. Both devices
being connected (and the cable) must be of the same type in order to ensure proper
operation. In certain limited cases (connecting Ethernet switch uplink ports, for
example), a mode conditioning cable may be employed so that multimode fiber
cable can be used with a single mode device. Refer to the diagrams in the Ethernet
Switch Interconnection Diagrams in the documentation included with your
Invensys qualified Ethernet switches for specific information.
NOTE
The fiber optic cables mentioned in this document may require additional
mechanical protection, particularly when run between different enclosures.
Single Mode Cable
In the control network, single mode fiber optic cable can be used to connect Ethernet switches to
each other through each switchs uplink port. The uplink ports of each of the switches being con-
nected must be designed for single mode operation. Refer to the Ethernet Switch Interconnec-
tion Diagrams in the documentation included with your Invensys qualified Ethernet switches
for information on uplink modules for use with single mode fiber optic cable.
Multimode Cable
Multimode fiber optic cable is employed in different situations in The MESH control network:
Controller to Switch - A controller connects through splitter/combiner(s) to the
Ethernet switch(es) 100Base-FX port(s) using a multimode fiber optic cable termi-
nated with MT-RJ connectors. The connection is LC on one end and MT-RJ on the
other end between the switch and the splitter. Ethernet switch(es) without fiber optic
port(s), can use media converter(s) between the switch(es) and the splitter/com-
biner(s) to enable a connection.
Field Communications Module (FCM) to Switch - A redundant FCM connects
through splitter/combiners to the Ethernet switches 100Base-FX ports using a multi-
mode fiber optic cable terminated with MT-RJ connectors. The connection is LC on
one end and MT-RJ on the other end between the switch and the splitter. Ethernet
switch(es) without fiber optic port(s), can use media converter(s) between the
switch(es) and the splitter/combiner(s) to enable a connection. A single FCM module
does not use a splitter/combiner and connects directly to a switch via an LC/MT-RJ
fiber cable.
Workstation to Ethernet Switch - A workstation is capable of connecting directly to
Ethernet switchs 100Base-FX port using a multimode fiber optic cable terminated
with MT-RJ connectors. An Ethernet switch without a fiber optic port, can use two
media converters between the switches and the workstations. Connection is based on
NIC type and switch type. Media converters can be used when NICs do not match
switch ports.
20

Ethernet Switch to Ethernet Switch - Ethernet switches that are equipped with multi-
mode fiber uplink ports may be connected directly to each other. The fiber optic cable
should be terminated with MT-RJ type or LC type connectors, depending on the
switch interface modules.
The MESH Control Network Management Software
Tool, NetSight
Console
The NetSight Console enables enterprise-wide management of The MESH control network to
provide network-wide monitoring and troubleshooting, such as device discovery, topology map-
ping, and event management.
The enterprise-level command and control console provides:
Multi-element management approach to facilitate the abstraction of complex network
policy into everyday business language
System-level monitoring, and troubleshooting capabilities such as device discovery,
event management, and logging.
NetSight Plug-ins use the advanced features while reducing administrative burden and lowering
total cost of ownership. The Plug-in applications include the NetSight Policy Manager.
Netsight Policy Manager
An advanced plug-in application for NetSight, the NetSight Policy Manager enables simple
management of complex network security policies to greatly enhance reliable network connectiv-
ity. It provides the following.
Role-based enterprise management
Defines Roles, Rules and Services of the network
Matches the role of the device with available network services
Facilitates a distributed firewall to all edge points in the network
Automated capabilities - offers ease of implementation, administration and
troubleshooting
Complex policy management - provides an Audit Trail (event log)
The following tables list the various NetSight Policy Manager software available.
Table 1-2. NetSight Console Policies
Part Number Detailed Description
NS-CON-50 NetSight Console 50 Devices (50 device license for 1 server plus 3
concurrent user licenses)
NS-CON-U NetSight Console Unrestricted (Unrestricted device license for 1
server plus 25 concurrent user licenses)
NS-CON-U-UG NetSight Console 50 to Unrestricted Upgrade
21

Table 1-3. NetSight Client Policy
NS-USER NetSight User (Add 1 concurrent user license to existing NetSight
server)
Table 1-4. NetSight Policy Bundle
NS-PB-50 NetSight Policy Bundle 50-devices (50 device Console license for 1
server plus 3 concurrent users, Policy Manager and Policy Control
Console)
NS-PB-U NetSight Policy Bundle Unrestricted (Unrestricted device Console
license for 1 server plus 25 concurrent users, Policy Manager and Pol-
icy Control Console)
NS-PB-U-UG NetSight Policy Bundle 50 to Unrestricted Upgrade
Table 1-5. NetSight Advanced Bundle
NS-AB-50 NetSight Advanced Bundle 50-devices (50 device Console license for
1 server plus 3 concurrent users with Policy Manager, Policy Control
Console, Automated Security Manager, Inventory Manager, and
NAC Manager)
NS-AB-U NetSight Advanced Bundle Unrestricted (Unrestricted device Con-
sole license for 1 server plus 25 concurrent users with Policy Manager,
Policy Control Console, Automated Security Manager, Inventory
Manager, and NAC Manager)
NS-AB-U-UG NetSight Advanced Bundle 50 to Unrestricted Upgrade
NS-AB-50FT NetSight Advanced Bundle 50-devices FT (50 device Console license
for 1 server plus 3 concurrent users, Policy Manager, Policy Control
Console, Automated Security Manager, Inventory Manager, NAC
Manager, a redundant NetSight license for fault tolerance (manual
failover), Includes Lab License)
NS-AB-UFT NetSight Advanced Bundle Unrestricted FT (Unrestricted device
Console license for 1 server plus 25 concurrent users, Policy Manager,
Policy Control Console, Automated Security Manager, Inventory
Manager, and NAC Manager, a redundant NetSight license for fault
tolerance (manual failover), includes Lab License)
NS-AB-UFT-UG NetSight Advanced Bundle 50 to Unrestricted FT Upgrade
22

Obtaining Network Management Software
To obtain Invensys pricing discounts, use the following contact information:
A free evaluation copy of the software can be downloaded at:
http://www.enterasys.com/products/management/NSA-LIC/
Table 1-6. NetSight Single User Policy
NS-SU-10 NetSight Single User 10-devices (10-device, single client (on same
machine as server) license for Console, Policy Manager, and Inven-
tory Manager)
Table 1-7. NetSight A-La-Carte Plug-Ins
NS-ASM NetSight Automated Security Manager (NetSight Automated Secu-
rity Manager. Requires existing NS-CON-50 or NS-CON-U license)
NS-IM NetSight Inventory Manager (NetSight Inventory Manager. Requires
existing NS-CON-50 or NS-CON-U license)
NS-NAC NetSight NAC Manager (NetSight NAC Manager. Requires existing
NS-CON-50 or NS-CON-U license)
NS-PM NetSight Policy Manager (NetSight Policy Manager. Requires exist-
ing NS-CON-50 or NS-CON-U license)
NS-LAB NetSight Lab License (Non-production use, one-time fee inclusive of
maintenance. 50 device license for 1 server plus 2 concurrent users
with Policy Manager, Automated Security Manager, Inventory Man-
ager, NAC Manager, Policy Control Console. Requires existing NS-
CON-50 or NS-CON-U license)
Table 1-8. NetSight Appliance Policy
SNS-NSS-A NetSight Appliance (Hardware only, requires separate NetSight
license)
Holly O'Gara
978-878-4579 (U.S. Number)
hogara@enterasys.com
OR Christine Leblanc
978-684-1559 (U.S. Number)
cleblanc@enterasys.com
Enterasys Networks
Corporate Headquarters
50 Minuteman Road
Andover, MA 01810
U.S.A
23

2. Site Planning
This chapter describes the steps that should be taken and the options that should be considered
when planning The MESH control network.
The design of each instance of The MESH control network is different, and depends on the needs
and requirements of the individual site. The following paragraphs provide the information neces-
sary to help network designers to plan The MESH control network that meets the needs of their
specific site.
Site Planning Overview
The MESH control network allows communication between the control stations, workstations,
and Field Communications Modules (FCMs) connected to the network. This network is formed
by interconnecting the control stations and workstations through the use of fiber optic or copper
cable, Ethernet switches and, if necessary, media converters. Although copper cable can be used,
fiber cable is recommended for industrial networks. In a properly constructed network, all the sta-
tions on the network are able to communicate with each other. For a high degree of reliability,
The MESH control network should be constructed so that there are redundant signal paths
between each device on The MESH control network.
Although the design of The MESH control network is driven largely by the sites physical envi-
ronment, traffic and organizational requirements, there are rules that must be followed when con-
necting devices to form The MESH control network. The following two sections explain the
factors affecting the physical and the traffic considerations of The MESH control network.
Network Considerations
The physical location of equipment at the site influences The MESH control network design. To
choose equipment and cabling appropriate to the requirements for the specific site, use the
answers to the following questions, along with The MESH control network design rules and
device/cable specifications in this chapter and Appendix A COMEX Fault Handling on The
MESH Control Network.
What is the maximum end-to-end distance between devices in The MESH control
network?
The distance between the ends of The MESH control network may determine
what kind of Ethernet switches are used and what kind of cabling is used between
those switches. If individual cable runs are greater than a hundred meters, it is
necessary to use fiber optic cable, due to its capability for transmitting signals over
longer distances. All hardware must be considered when the network requires
multiple hops; no low end standalone switches should be used in the network
core.
It should be noted that the total network delays between two end devices should
not exceed 100 ms.
Will The MESH Control Network have VLANs?
B0700AZ Rev L 2. Site Planning
24

NOTE
If the answer to this question is No, it is recommended that when setting up a first
time installation of The MESH control network that VLAN 2 I/A Control Ports
is deployed on all ports used for communications between I/A Series devices. (See
the warnings below.)
If a system is up and functional without VLAN 2 deployed across the network, a
system shutdown is required to deploy VLANs. If VLAN 2 is deployed prior to
system startup, additional VLANs can be added at a later date without system
interference (shutdown).
If yes, how many VLANs will The MESH control network require?
The MESH control network will support up to six VLANs, one (1) of which must
be reserved for I/A Series system devices (VLAN 2 I/A Control Ports), and all
I/A Series control devices shall be attached to VLAN 2 FE ports.
Only one I/A Series control system can be supported on The MESH control
network.
There can be no duplicate MAC addresses across The MESH control network
VLANs.
If VLANs are deployed, all switches in the network must have VLAN 2 enabled
on the switchs uplink ports.
WARNING
!
When VLANs are added to an existing installation of The MESH control network,
if the I/A Series devices are on VLAN 1, they must be moved to VLAN 2, at which
time communications between the I/A Series devices will be broken.
WARNING
!
All I/A Series devices must be connected to Device ports which have been assigned
to VLAN 2 I/A Control Ports. If this is not done, the I/A Series devices on the
network will not communicate correctly with each other. Refer to the Flowchart
for Assignment of Uplinks and Ports to VLAN figure in the Switch Configura-
tion Parameters Dialog Box section in The MESH Control Network Operation, and
Switch Installation and Configuration Guide (B0700CA).
If VLANs are enabled, all switches in the network must have VLAN 2
I/A Control Ports set to ENABLED.
WARNING
!
If VLANs are to be utilized in The MESH control network, all switches within the
network are required to be configured for all utilized VLANs. If a VLAN is config-
ured on an outer edge switch and a core switch has not been configured for that
VLAN (in the case where no port assignment is required), data from the outer edge
switch VLAN will not propagate through the core switch.
2. Site Planning B0700AZ Rev L
25

What will the protocols be on the non-I/A Series VLANs?
VLANs other than VLAN 1 and 2 shall not utilize protocols other than TCP/IP
and/or UDP/IP.
What will be the loading on the FE device ports?
No VLAN Port shall exceed 50% loading.
What is the non-I/A Series VLAN end device speed usage?
No end devices with transfer (Tx) and receive (Rx) speeds greater than 100Mb are
supported.
Will the switch be installed in an adverse environment?
For the requirements for industrial switches in The MESH control network, refer
to I-Series (P0973GB) Industrial Switch, Hardware and Software Configuration
Instructions (B0700CN) for the I-series Industrial switch specifications.
How many control stations and workstations will be connected to The MESH con-
trol network?
The number of control stations and workstations influences how many Ethernet
switches and separate cables are included in The MESH control network. It may
be wise to plan for expansion and growth when arriving at this number.
How are the control stations and workstations distributed?
The physical distribution of control stations and workstations can have a bearing
on the type and number of Ethernet switches, as well as the type of cabling used.
If the control stations and workstations are clustered together, two larger capacity
switches may be able to accommodate them all. If they are dispersed, several
smaller switches may be a better solution. For large distances and noise protection,
you should use fiber optic cable between switches, or between switches and con-
trol stations, workstations, or FCMs. Cable routing between devices must also be
considered.
Will The MESH control network have redundant signal paths?
Yes, a control network with redundant paths is recommended for all control sys-
tems.
Through what kind of physical environment will the cabling pass?
Fiber optic cable is immune to magnetic fields and electrical noise, so it can be
used in places where copper cable would be unreliable. If cabling must be routed
through areas subject to high EMI or RFI, fiber optic cable is recommended. Due
to its immunity to lightning, fiber optic cable should also be used for outdoor
cable runs (rated for outdoor use).
Where will the equipment be located?
Will switches be mounted in racks, in cabinets, or placed on shelves or tabletops?
Take into consideration thermal requirements, especially if the equipment will be
mounted in an enclosed cabinet or area. High or moderate electromagnetic noise
sources, for example, machinery, switchgear, high-voltage lines, and so forth, in
close proximity must be avoided to ensure reliable operation.
Have plans for future expansion been considered?
26

Consider installing Ethernet switches with additional ports so that control stations
or workstations can be easily added in the future. Running extra copper or fiber
optic cable will allow for network expansion later.
Have maintenance and troubleshooting provisions been made?
Leaving at least one unused port on each Ethernet switch allows for maintenance
and troubleshooting of managed switches. A system network management pack-
age (SNMP) must be available that runs on a separate PC, not an I/A Series work-
station. Patch panels at the end of cable segments facilitate troubleshooting and
network re-configuration.
Newer switches use 1 Gb uplinks. Can a new MESH network be built including
legacy 100 Mb uplinks?
While a MESH network may be built using 100 Mb uplinks, this is not recom-
mended due to loading concerns. Note that Security Enhanced Configurations
require the use of 1 Gb uplink ports.
Are ample ac power outlets available at the proper location:
For the Chassis switch?
For switches?
For redundant power supplies?
For media converters and ac to dc power adapters for media converters?
Traffic Considerations
Specific traffic requirements should be considered when planning The MESH control network.
Should certain control stations or workstations be connected to the same switch?
To reduce traffic through the root switches, it may be advantageous to group con-
trol stations or workstations according to department, process, or other criteria
that is important to the site or organization.
Should certain control stations or workstations not be connected to the same switch?
It is recommended that each control station or workstation from a particular
group be connected to two separate switches. If one switch were to fail, these con-
trol stations or workstations can access the network through the redundant
switch. Without a redundant network, none of the workstations from that group
would have network access. Workstations can have two Ethernet ports to access
the network.
Network Loading
Understanding the details of the network traffic flow is an important part of planning and imple-
menting The MESH control network for an I/A Series system. This provides insight on how to
verify that there is sufficient network bandwidth available between network devices. This is diffi-
cult to measure, since available bandwidth is a dynamic quantity where the amount of traffic that
can be transmitted over a link can change given its current traffic conditions and what applica-
tions are running on those devices. The process of designing industrial networks is a very chal-
lenging task due to its inherent complexity. A load prediction can be achieved by approximating
the environment, modeling the network components and analyzing the interrelations. This pro-
27

cess works well for industrial networks since they tend to be more static and are typically slow to
evolve.
It is recommended that The MESH control network steady state load not exceed a maximum
load of 50% of any given link.
The MESH control network is a full-duplex, point-to-point system. This network does not have a
collision domain. Unlike half-duplex links, full-duplex links do not deteriorate as the network
load increases. All end-devices can transmit at will. However, if The MESH network is designed
with 100 Mb uplinks communication from these end-devices, this could cause a bottleneck due
to insufficient available network bandwidth
1
. Since The MESH network runs in full duplex
mode, both transmit and receive channels can run at 100 Mb simultaneously without degradation
on the adjacent channel.
The MESH network traffic rates are also affected by third-party applications or user applications
that generate high packet rates. Workstation to workstation operations on The MESH network,
such as copying extremely large files, can also result in a temporary high bandwidth usage up to
50% of the network:
Network loading will be represented by the percent of time that the network is in use over a given
period. By definition, individual Ethernet segments can only transmit one packet at a time. For
any given moment, the Ethernet segment is either at 100% loaded (transmitting a packet), or at
0% utilization (idle). The network loading percentage shows the percentage of time the network
is in use over a set period.
When calculating the network loading, you need to know how many bytes of network traffic are
being handled by the network over a set period. This involves totaling the input (or output) byte
counts for a set period, and dividing by the total capacity of the device interface for that period.
To determine the total number of bits received on the interfaces, each of the packet byte rates is
multiplied by 8.
%loading = (dataSent * 8) / (intSpeed * sampleTime) * 100
Where:
dataSent = total number of bytes sent
intSpeed = maximum connection speed
sampleTime = duration of time required for transmissions
For example, for a system having 6 x 1500 byte sends on 100Mb connection at maximum of 2Mb
(2 * 1,000,000 bps) every 500 milliseconds:
7.2% = (6 * 1500) * 8 / (2,000,000 bps * 0.50secs) * 100
Equipment Considerations
The MESH control network was designed and tested for operation with the Ethernet switches
listed in Reference Documents on page xii. The network may operate with similar, off-the-shelf
1.
Flow control and rate limiting are disabled, and full-duplex cables are used, allowing the controllers
transmitted packet to egress though the network without delaying or filtering. However, by doing
this, a packet on a heavily loaded network can cause traffic to slow. When using these features, the
switch acts as a traffic cop controlling the data to maximize the traffic flow. Disabling these features
allows the traffic to flow faster but can cause issues when traffic loads are high.
28

equipment, but Invensys Systems, Inc. is not responsible for any system malfunctions that may
occur if such equipment is used.
If you use your own network equipment:
1. You must meet the bandwidth requirements (1 Gb for uplink ports and 100 Mb for
ports) for the I/A Series equipment you have chosen.
2. There can be no Layer 3 inter-network devices (for example, routers) between any
I/A Series equipment.
Hardware Requirements
While switches with 100 Mb uplink ports are allowed on networks with standard configurations,
Security Enhanced Configurations require the use of 1 Gb uplink ports. However, 1 Gb uplink
connections should be used to interconnect switches in all configurations, if available.
Switch Utilization in the Standard Configurations
Any switches defined in the table Invensys-Supplied Ethernet Switches in the Introduction of
The MESH Control Network Operation, and Switch Installation and Configuration Guide
(B0700CA) can be used in Standard Configurations.
Note that C-Series switches (P0973BL/HA) are used only in a Standard Configuration and can-
not be used in the Security Enhanced Configuration.
Switch Utilization in the Security Enhanced Configurations
Refer to the Appendix table Qualified Switch Standard/Security Enhanced Configuration Com-
patibilities Matrix in The MESH Control Network Operation, and Switch Installation and Config-
uration Guide (B0700CA) for a list of allowed switch types in Security Enhanced Configurations.
Since the Security Enhanced Configurations deploy Loop Detection Policy algorithm (LDP)
methods to monitor and protect the network from RSTP or Data Loop (storm) failures, specific
switch types must be used in a variety of required locations within the network. The table listed
above identifies the switch types which fall into the categories Security Enhanced and LDP
Deployable.
In a linear topology in a Security Enhanced Configuration, both switches must be
from the LDP Deployable category.
A ring topology cannot be supported as Security Enhanced Configuration.
In a star topology in a Security Enhanced Configuration, both root and backup root
switches must be from the LDP Deployable category. All other switches within the
star network can be from the Security Enhanced category.
In an inverted tree topology in a Security Enhanced Configuration, all switches
within the network must be from the LDP Deployable category.
The modified inverted tree topology in a Security Enhanced Configuration requires
that all switches within the network core are LDP Deployable. All outer edge
switches within the tree network can be any switch type; however, the redundant
switches (A/B switches) must be deployed on different tiers.
29

Firmware Considerations
When planning which Invensys-supplied Ethernet switches to use when building The MESH
control network for your system, be aware that certain versions of these switchs firmware may be
incompatible with one another.
Refer to the Appendix Qualified Switch Firmware Compatibilities Matrix in The MESH Con-
trol Network Operation, and Switch Installation and Configuration Guide (B0700CA) for the
I/A Series qualified and compatible firmware for each switch.
The MESH Control Network Design Rules
When designing The MESH control network, the following rules/guidelines should be kept in
mind.
Standard Configuration Design Rules
The following rules apply to the standard design of The MESH control network:
1. No more than 1920 logical stations where:
Fault-tolerant stations count as one logical station
Workstations count as one logical station
Ethernet switches count as one logical station.
FCMs are not counted.
2. No more than 10,000 IP addresses where:
A fault-tolerant FCP270 counts as one address
A fault-tolerant ZCP270 counts as two addresses
An ATS station counts as two addresses
MESH workstations count as two addresses prior to I/A Series system version 8.2;
MESH workstations with I/A Series system V8.2 or later count as one address.
Nodebus workstations count as one address
Ethernet switches count as one address
FCMs count as one address.
3. No more than 250 managed switches are allowed.
4. There should be no more than seven switches in the path between any two devices in
The MESH control network.
5. Routers are not allowed in The MESH control network.
6. Switch-to-Switch connections (uplink ports) should be made using 1 Gb uplink ports
to allow enough bandwidth for network traffic of I/A Series equipment
7. Only two configured root bridges are allowed in the network.
8. Horizontal switch interlinks (links between switches of the same tier) are not allowed,
except between the root and backup root switches. (See Figure 2-1 on page 33).
9. The MESH control network must have redundant uplink connections between the
root and backup root switches.
30

10. The MESH control network will support up to six VLANs, one (1) of which must be
reserved for I/A Series system devices (VLAN 2 I/A Control Ports).
a. Only one I/A Series control system can be supported on The MESH control net-
work.
b. If VLANs are deployed, all I/A Series control devices shall be attached to VLAN 2
FE ports.
c. If VLANs are deployed, all switches in the network must have VLAN 2 enabled
d. VLANs other than VLAN 1 and 2 shall not utilize protocols other than TCP/IP
and/or UDP/IP.
e. No VLAN port shall exceed 50% loading
f. No end devices with transfer (Tx) and receive (Rx) speeds greater than 100Mb are
supported.
NOTE
It is recommended that one port on each managed Ethernet switch be reserved for
testing and diagnostic purposes. No device should be connected to this port.
Non I/A Series Equipment
The following rules apply to non I/A Series equipment:
1. Non I/A Series equipment (for example, routers, hubs and unmanaged switches) are
not allowed to be connected to The MESH control network.
2. A corporate or plant enterprise network should not be connected directly to The
MESH control network. The recommended way to communicate from a corporate or
plant enterprise network is by using an additional NIC on an I/A Series workstation.
Security Enhanced Configuration Design Rules
When designing the Security Enhanced Configuration, the following rules/guidelines must be
followed.
1. No more than 1920 logical stations where:
Fault-tolerant stations count as one logical station
Workstations count as one logical station
Ethernet switches count as one logical station.
FCMs are not counted.
2. No more than 10,000 IP addresses where:
A fault-tolerant FCP270 counts as one address
A fault-tolerant ZCP270 counts as two addresses
An ATS station counts as two addresses
MESH workstations count as two addresses prior to I/A Series system version 8.2;
MESH workstations with I/A Series system V8.2 or later count as one address.
31

Nodebus workstations count as one address
Ethernet switches count as one address
FCMs count as one address.
3. No more than 250 managed switches are allowed.
4. There should be no more than seven switches in the path between any two devices in
5. Routers are not allowed in The MESH control network.
6. Switch-to-Switch connections (uplink ports) must be made using 1 Gb uplink ports
to allow enough bandwidth for network traffic of I/A Series equipment.
7. Core switches must be only LDP-supported switches, as described in the Appendix
table Qualified Switch Standard/Security Enhanced Configuration Compatibilities
Matrix in B0700CA.
8. The root switch must have the lowest IP address in the network, or at least an address
lower than the backup root switch.
9. Only two configured root bridges are allowed in the network.
10. Do not use an application on the network which requires IGMP, such as IP video.
11. Horizontal switch interlinks (links between switches of the same tier) are not allowed,
except between the root and backup root switches. (See Figure 2-1).
12. The Circuit Breaker Policy (CBP) should be deployed at all uplinks ports facing the
outer edge (away from the root), ports that are not and will not be blocked by span-
ning tree.
13. The Beacon Priority Policy (BPP) should be deployed at all uplink ports facing
towards the root.
14. The Backplane Circuit Breaker Policy (BPCB) (used only for E-series bridge cards
P0973BS) should be used to detect RSTP failures on the E-series second and third
generation blades only (defined in the Appendix table Qualified Switch Stan-
dard/Security Enhanced Configuration Compatibilities Matrix in B0700CA). It is
not recommended that any other switches be interfaced to the second and third gener-
ation blades via uplink or data ports. Blades protected with the BPCB policy (i.e. E-
series second and third generation blades) should reside at the outer most edge of the
network. A detailed description of BPCB and other policies can be found in the
appendix section Deploying Loop Detection Policies in the B0700CA document.
15. All edge switch uplink ports facing the root require the spanning tree admin path cost
to be increased (200000), this ensures efficient port blocking at the edge.
16. Methods for active network monitoring (e.g. NetSight Policy Manager) are not
required but are recommended for medium to large networks.
17. The MESH control network must have redundant uplink connections between the
root and backup root switches.
18. The MESH control network will support up to six VLANs, one (1) of which must be
reserved for I/A Series system devices (VLAN 2 I/A Control Ports).
a. Only one I/A Series control system can be supported on The MESH control net-
work.
32

b. If VLANs are deployed, all I/A Series control devices shall be attached to VLAN 2
FE ports.
c. If VLANs are deployed, all switches in the network must have VLAN 2 enabled
d. VLANs other than VLAN 1 and 2 shall not utilize protocols other than TCP/IP
and/or UDP/IP.
e. No VLAN port shall exceed 50% loading
f. No end devices with transfer (Tx) and receive (Rx) speeds greater than 100Mb are
supported.
33

Figure 2-1. Large Network (Security Enhanced Configuration)
Uplink paths should
be gigabit paths
Interlink flows across the network, i.e. on the same tier, this is only recommended at the root tier
with Circuit Breaker PBQ and SBQ deployed.
Interlinks should flow
towards or away from root,
i.e. from one tier to the other.
Note: On N-series switches, the two interlinks between the root and backup should be on
separate blades.
34

Additional Guidelines for Planning Security Enhanced
Configuration
The design of each instance of The MESH control network is different, and depends on the needs
and requirements of the individual site. This section provides the information necessary to help
network designers to plan The MESH control network that meets the needs of their specific site.
The MESH Control Network with LDP (also referred to as the Security Enhanced Configura-
tion) isolates storming switches, and allows communications to continue on The MESH. In a
properly constructed network, all switches on the network are able to communicate with each
other with a high degree of reliability and redundancy. The MESH control network is con-
structed so that there are redundant links that form physical loops in the network which are con-
trolled (Blocked) by Rapid Spanning Tree Protocol (RSTP), creating a logical loop-free network.
In the event of a failure of this function the loop detection algorithm provides a backup to this
type of failure.
There are rules that must be followed when connecting switches to form The MESH control net-
work. The following section explains the dynamics affecting the physical topology and the traffic
considerations of The MESH control network.
Security Enhanced Configuration Design Considerations
Once the general plan for the Security Enhanced Configuration has been outlined, the physical
design of the network can be defined. Refer to B0700CA for the rules concerning network config-
uration/topology and list the specifications for the network devices (switches, converters, interface
modules) and cabling, and to choose the equipment that is needed to implement the network
plan.
NOTE
For a variety of questions to help you define both the Standard and Security
Enhanced Configurations, refer to Network Considerations on page 23.
When the physical design has been finalized, make a drawing or map of the network topology and
save it. The map should be updated whenever a physical change is made to The MESH control
network.
Spanning Tree Behavior
A Spanning Tree design must be implemented correctly to provide a loop-free network during
normal operations. When LDP is deployed, improper configuration can cause false triggers of
the policy which will disable active ports/switches.
Users should ensure that the edge switches keep their redundant connections blocked. The path
cost typically involves the sum of the path costs, that is, of the links that are traversed. This cost
can be adjusted by altering the default assigned path costs for individual links. An edge switch
may not block its ports in an expected order, though other switches may do so. Edge switches
must have their path cost administratively increased to get their local ports to block. This keeps
the Beacon packet from being forwarded to a port with the Circuit Breaker policy. Increasing
a path cost on a switch port, increases its likeliness to block all local ports on edge devices in a
MESH topology. This path cost is dependent on the switch interconnections between the switch
tiers. The edge switch uplink ports path cost must be increased to 200000. Ports designated as
the Beacon ports, must have their path cost set from default value of 0 to a value 400000.
35

The admin path cost is automatically adjusted when utilizing the I/A Series Switch Configurator
Application Software (Rev 1.1.4 or greater) discussed in The MESH Control Network Operation,
and Switch Installation and Configuration Guide (B0700CA). However, if other methods are used,
refer to the switch configuration documentation provided with the switch to perform this func-
tion.
IGMP Beacon Behavior
LDP functions utilize IGMP packets which act as a beacon. The IGMP Beacon is enabled only
on the root and backup root switches in the network when deploying LDP. Only the designated
root switch with the lowest IP address will send out a Beacon query packet. The Secondary
Beacon queryer (SBQ) acts as a backup and only sends out a Beacon query packet if the root
fails. The root switch must be configured with the lower IP address of the root and backup root
pair in order for LDP to operate properly. The switch with the lowest IP address and with its
IGMP Beacon enabled will be chosen as the Primary Beacon Queryer (PBQ). When configur-
ing the network in SysDef or IACC System Editor, the root switch should be designated with the
lowest switch IP address of the root/backup root pair.
In the event of a root switch failure the SBQ on the backup root will take over. Once the root
switch has been placed back online PBQ from the root will send out an IGMP Beacon packet
stopping the SBQ on the backup root from transmitting its Beacon packet.
CAUTION
!
When using Netsight Policy Manager to manage a Security Enhanced Configura-
tion switch, you must have Policy Manager 2.2.0 build 17 or greater installed. In
the event of a root switch failure, the LDP Beacon PBQ will become disabled and
the functions of the root switch will move to the backup root switch along with the
LDP Beacon SBQ. Once the root switch failure has been resolved and the switch
has been placed back on the network, the LDP Beacon PBQ will become enabled,
disabling the SBQ. Due to this event, the redundant root switch links between
the root and backup root will be viewed by LDP as a loop within the network,
which causes LDP to disable one of the links (the blocking port). To recover from
this rule hit, the root switch (switch with the lowest IP address) needs to regain all
root functions. Netsight Policy Manager 2.2.0 build 17 or greater has the correct
software required to recover from this rule hit. This event can be prevented if
VLAN 2 is deployed on the network, by moving the root switch host ports to a
secure VLAN VLAN 2 (host port moved to VLAN 2 is required for an I/A Series
system) will resolve the false port hits between the two root switches.
The default timing for the IGMP protocol causes one packet to be sent every 125 seconds. In
order to provide a Beacon packet that will allow a loop to be detected within one second the
IGMP timer is set to one second for LDP. Since IGMP packets are multicast packets they are for-
warded out from the root to the edge switches of the network. In the event of a loop, the packet
will be forwarded back towards the root where a port with a circuit breaker policy will detect it
and disable the port shutting down the loop.
Normally IGMP packets are used to sustain and prune multicast flows in a network and are used
for IP video. Any application which uses IGMP, such as IP video, cannot be used on a Security
Enhanced MESH Control Network since it will disrupt LDP operation.
36

The MESH Control Network Design
Once the general plan for The MESH control network has been outlined, the physical design of
the network can be defined. The following two sections describe the rules concerning network
configuration/topology and list the specifications for the network devices (switches, converters,
interface modules) and cabling. Use these sections to choose the equipment that is needed to
implement the plan that was derived from the answers to the questions at the beginning of this
chapter. When the physical design has been finalized, make a drawing or map of the network
topology and save it. The map should be updated whenever a physical change is made to The
MESH control network.
I/O Network Design Rules
The following rules apply to the I/O network design of The MESH control network:
1. Only Invensys-qualified switches supported for The MESH control network are per-
mitted on the I/O network.
2. The I/O network does not support redundant ISL links between switches. Only single
links between switches are allowed.
3. Routers are not allowed in the I/O network.
4. Switch-to-Switch connections (ISL uplink ports) must be made using 1 Gb uplink
ports to allow enough bandwidth for network traffic of the I/A Series equipment.
100 Mbps uplinks are not supported on the I/O network.
5. Configured root bridges are allowed in the I/O network, but not required.
6. The A Fieldbus switches must not be interlinked to the B Fieldbus switches (see
Figure 1-7 on page 14). The two Fieldbuses must maintain separation.
7. Only the linear topology is supported on the I/O network, with no more than seven
switches in series. It is recommended to use the fewest number of switches with which
you can configure the I/O network.
8. The same number of switches must be maintained on both the A side and the B
side of the I/O network.
NOTE
A dedicated I/O network switch can be configured with the Switch Configurator
Application Software (SCAS), as discussed in The MESH Control Network Opera-
tion, and Switch Installation and Configuration Guide (B0700CA).
NOTE
It is recommended that one port on each managed Ethernet switch be reserved for
testing and diagnostic purposes. No device should be connected to this port.
The MESH Control Network Topologies
A key feature of The MESH control network is that single points of failure will not prevent com-
munications between all the devices in the network. This is accomplished by using a MESH net-
work design in which each I/A Series station is ideally connected to two different Ethernet
37

switches. Each of the Ethernet switches is then connected to two other Ethernet switches. This
design provides redundant data paths so that the failure of a single device doesnt cause opera-
tional network problems.
Security Enhanced Configurations provide the loop detection algorithm known as LDP to pro-
tect against RSTP and/or data loop (storm) failures. When deployed on a Security Enhanced
Configuration, LDP allows for the isolation of a defective switch or port (depending on the type
of failure) during a network storm when caused by a spanning tree failure or a data looping fail-
ure. This function will prune the defective loops, maintaining network communications between
devices during a network storm.
NOTE
If the network is constructed with less than two connections between devices, data
traffic could be disrupted if any single device fails.
The MESH control network can be constructed using two network configuration/topology meth-
ods. The first is the Standard Configuration as described in this section under Standard Config-
urations below and the second is the Security Enhanced Configuration as described in Security
Enhanced Configurations on page 52. The topology chosen depends largely on the size, com-
plexity, site requirements, and network specifications of The MESH control network. The follow-
ing paragraphs provide some guidelines for choosing a network configuration (topology).
NOTE
All graphics of switches and media converters below are intended as generic illustra-
tions of networking concepts and do not necessarily reflect the currently offered
products.
Standard Configurations
There are five basic standard configurations that are supported by The MESH control network.
As well, the variations of these topologies listed below can be made as long as all design rules are
met. Be aware that some restrictions to these variations listed below may be warranted.
The basic standard configurations and their approved variations are as follows:
Linear
Ring
Star (Variation: Double Star)
Inverted tree
Modified inverted tree (Variation: Inverted Stepped Tier Tree)
Standard Small Networks
A small network, consisting of two switches, could be configured as in Figure 2-2.
38

Figure 2-2. Small Network (Standard Configuration)
In this example, two fiber optic Ethernet switches are used and the workstation and controller
have a 100 Mb connection to each of the switches. The connections between the switches can use
the switchs high speed uplink ports (1 Gb).
NOTE
Two connections between switches are required for proper redundancy.
Larger Chassis switches can be used instead of the non-Chassis Fiber switches. Larger Chassis
switches will allow hundreds of workstations, controllers and FCMs to be connected to The
MESH control network. This provides a small network but quite a large control system. Multiple
FCP270/FCM100s/ZCP270s, ATS modules and workstations can be connected to a redundant
switch.
Gigabit Ethernet over
Fiber Uplink Ports
FCP270
P92
Fiber Managed
Switches
39

Standard Medium Networks
A medium-sized network of three to seven switches can be configured as a ring. Figure 2-3 shows
an example of a ring topology containing six switches. In this example, all the switches are fiber
optic and each switch is connected to the two switches adjacent to it. If one of the switches should
fail, the ring is broken but RSTP (802.1w) reroutes data around the break and the network
remains operable. Typically a failover time of less than 1 second is achievable using switches qual-
ified and supplied by Invensys and configured in accordance with I/A Series documentation.
NOTE
The I/A Series system imposes a seven switch limit on the length of any data path.
This means that there can be no more than seven switches between any two devices
in the network.
Figure 2-3. Medium Network (Ring) (Standard Configuration)
A medium and large-size network can also be configured in a star topology as shown in
Figure 2-4. The star topology is the preferred topology for control systems. In star topology, two
switches make up the backbone of the network and should be configured as the root and backup
root switches. The other switches edge switches in the network are connected to both root
switches. If desired, other devices (workstations or controllers) can be connected directly to the
root switches.
A medium network Star Configuration has a limitation of 40 switches on the edges when Gold
Series blades are used and 166 when Platinum Series Blades are used. This limit is derived from
the maximum number of Gb ports on the Chassis switch.
Two ports are used for interconnection between the primary and backup root.
Fiber Uplink Ports
Primary Root
Backup Root
Fiber Managed
Switches
40

Figure 2-4. Medium or Large Network (Star) (Standard Configuration)
The Double Star Configuration (a variation of the Star Configuration, shown in Figure 2-5)
allows the user to benefit from the Star Configuration advantages while allowing the user to
deploy a much larger sized network. Bandwidth considerations between the Star topology
switches must be observed, to minimize bandwidth consumption. The majority of peer-to-peer
end device communications should reside local to the individual Star topologies.
Fiber Uplink Ports
Primary
Backup
Root
Root
Expandable
Chassis Switch
Fiber Managed
Switches
Fiber Managed
Switches
41

Figure 2-5. Medium or Large Network (Double Star) (Standard Configuration)
Standard Large Networks
Large networks should be designed to use an inverted tree topology or a modified inverted tree
topology.
For the inverted tree topology, Ethernet switches are arranged in tiers, with primary and backup
root switches in the top tier. Each switch is connected to two different switches in the tier above it
and end stations may be connected to any of the switches. A large network containing 12 switches
in four tiers is illustrated in Figure 2-6.
NOTE
There is a limit of four tiers on both inverted tree topology or a modified inverted
tree topology. This is to prevent the path between any two devices from containing
more than seven switches.
Fiber Uplink Ports
N-Series N-Series
Secondary Root Primary Root
Expandable Chassis Switches
Tier 1
Tier 2
Tier 3
A-, I- or V-Series Switches A-, I- or V-Series Switches
A-, I- or V-Series Switches
C-Series Switches
42

Figure 2-6. Large Network - All Blades (Standard Configuration)
Uplink paths should
be gigabit paths
43

The Inverted Stepped Tier tree topology, shown in Figure 2-7, is an example which demonstrates
that users can utilize a non-symmetrical tree topology.
In this topology, switches connected between every other tier cannot be separated by more than
100 meters due to the distance restrictions on the copper uplinks. Every other tier separation can
be greater than the 100 meter restriction, by using fiber Mini-GBICs. Separation greater than the
100 meter restriction can be between any Tier, but EMI noise should be taken in consideration
when using copper uplink ports.
In the example in Figure 2-7, the uplinks between tiers 2 and 3 are not required to be interfaced
to the switch directly above each. However, all switches in the network must be interfaced with
two (2) uplinks to the tier above them.
When using I-series switches at the out edge, fiber Mini-GBICs must be used to interface the
switches into The MESH control network.
If 100Mb uplinks are used instead of 1Gb uplinks, bandwidth consumption must be minimized
between the two tiers. To minimize bandwidth consumption as much as possible, the end device
peer-to-peer communications should reside local to the switch to which they are connected.
44

Figure 2-7. Large Network - Inverted Stepped Tier Tree - All Blades (Standard Configuration)
CAUTION
!
In this topology, switches connected between every other tier cannot be separated
by more than 100 meters due to the distance restrictions on the copper uplinks.
Every other tier separation can be greater than the 100 meter restriction by using
fiber Mini-GBICs. Separation greater than the 100 meter restriction can be
between any tier, but EMI noise should be taken in consideration when using cop-
per uplink ports.
Fiber Uplink Ports
N-Series N-Series
Tier 1
Tier 2
Tier 3
A-Series
Switches
Tier 4
A-Series
Switches
A-Series
Switches
45

The modified inverted tree topology is similar to the inverted tree topology but with consider-
ations to cost restraints. This topology utilizes low end, low cost switches as edge devices. In this
topology, the switches are arranged in tiers, with the root switches in the top tier and up to three
tiers below them (maximum of four tiers). The two root switches are connected to each other and
the other switches each have a connection to two of the switches in the tiers above and below
them. Low-end low-cost switches must be added to the outer edge at different tiers. Redundant
data paths allow the network to continue to operate if any one component fails.
A large network containing 10 switches in four tiers is illustrated in Figure 2-8.
I/A Series system requirement, limiting the number of switches between devices to seven.
Figure 2-8. Large Network - Blades and Low-Cost Switches (Standard Configuration)
46

When designing the physical layout of a large network, the following guidelines apply:
Switch-to-Switch connections (uplink ports) should be made using 1 Gb uplink ports
to allow enough bandwidth for network traffic of I/A Series equipment
There should be a primary and backup root switch on the network
There should be no more than four tiers of switches, including the primary and
backup root switches
There should be no horizontal connections between switches on the same tier except
the root and backup.
This minimizes the number of switch interconnections in order to facilitate faster
network respawning of the network tree.
This also reduces the likelihood of a loop occurring.
Each tier should contain an even number of switches
Each switch should be connected to two different switches in the tier above it
There should be two connections between the primary root and the backup root.
Example:
If it has been determined that the number of workstations, controllers and FCMs that must be
connected requires a network containing ten switches, the total number of switches needed will
be 12 (ten switches plus primary and backup root switches). Refer to Figure 2-6 for a diagram of
the finished network.
47

Step 1: Determine Physical Structure
The primary and backup root switches will be in the top tier, and the remaining ten switches will
be divided between the other three tiers. To determine the number of switches in the three tiers,
divide the ten switches by three. The result is three (with one left over), which suggests that there
should be approximately three switches per tier. Keeping in mind that there should be an even
number of switches in each tier, switches are added or subtracted until this guideline is met. In
this case we arrive at a solution that has two switches in the second tier, and four switches in each
of the remaining two tiers.
Figure 2-9. The MESH Control Network Tiers
Switch Switch
Switch Switch
Switch Switch
Switch Switch
Switch Switch Switch Switch
Backup Primary
Root Switches
2nd Tier
3rd Tier
4th Tier
48

Step 2: Connect Root Switches
Two connections are made between the primary and backup root switches. These connections
should be from gigabit uplink port to gigabit uplink port. Switch-to-Switch connections (uplink
ports) should be made using 1 Gb uplink ports to allow enough bandwidth for network traffic of
I/A Series equipment. Connecting a gigabit uplink port to a gigabit uplink port increases the
speed of the response.

Figure 2-10. Root Switches Connected
Switch Switch
Switch Switch
Switch Switch
Switch Switch
Backup Primary
Root Switches
2nd Tier
3rd Tier
4th Tier
49

Step 3: Connect Second Tier
Once the primary and backup root switches have been connected, separate connections should be
made from each second tier switch to both the primary and backup root switches.

Figure 2-11. Second Tier Connections
Switch Switch
Switch Switch
Switch Switch
Switch Switch
Backup Primary
Root Switches
2nd Tier
3rd Tier
4th Tier
50

Step 4: Connect Third Tier
Each switch on the third tier should have separate connections to two different switches in the tier
above it. In this case there are only two switches in the tier above, so a connection is made to each
of them.
Figure 2-12. Third Tier Connections
Step 5: Connect Fourth Tier
The final step is to connect each switch in the forth tier to two different switches in the tier above
it. Connections may be made to any two switches in the tier, but consideration should be made
with respect to keeping a manageable traffic load on each switch.

Figure 2-13. Fourth Tier Connections
Switch Switch
Switch Switch
Switch Switch
Switch Switch
Backup Primary
Root Switches
2nd Tier
3rd Tier
4th Tier
Switch Switch
Switch Switch
Switch Switch
Switch Switch
Backup Primary
Root Switches
2nd Tier
3rd Tier
4th Tier
51

Step 6: Label the Switches
Switches and cables should be labeled to aid in organizing the network connections. The strategy
illustrated here is one method. This example uses the A and B labels as an aid in migrating from
two separate nodebus networks to The MESH control network. Numeric superscripts are added
to designate the tier and the switch number within the tier. Any labeling strategy which helps to
keep the network connections organized may be used.

Figure 2-14. Labeling the Switches
Switch Switch
Switch Switch
Switch Switch
Switch Switch
Backup Primary
Root Switches
2nd Tier
3rd Tier
4th Tier
A
11
A
21
A
31
A
32
A
41
A
42
B
11
B
21
B
31
B
32
B
41
B
42
52

Security Enhanced Configurations
There are four basic Security Enhanced Configurations that are supported by The MESH control
network. These are:
Linear
Star (Variation: Double Star)
Inverted tree
The following diagrams provide examples of the different topologies deploying the loop detection
algorithm (LDP) as well as recommendations on where they might be used. All Security
Enhanced Configurations require specific switch models as the root or backup root. The inverted
tree requires switches which have LDP. The modified inverted tree requires all switches except the
outer edge switches to have LDP.
Refer to the LDP Deployable switches in the Appendix table Qualified Switch Standard/Secu-
rity Enhanced Configuration Compatibilities Matrix in The MESH Control Network Operation,
and Switch Installation and Configuration Guide (B0700CA) for the applicable switch models.
Linear Networks (Security Enhanced)
A small network, consisting of two switches, could be configured as shown in Figure 2-15.

Figure 2-15. Small Network (Security Enhanced Configuration)
NOTE
Two connections between switches are required for proper redundancy.
Larger Chassis switches can be used instead of the non-Chassis Fiber switches. Larger Chassis
switches will allow hundreds of workstations, controllers and FCMs to be connected to The
MESH control network. This provides a small network but quite a large control system. Multiple
FCP270/FCM100s/ZCP270s, ATS modules and workstations can be connected to a redundant
switch.
Fiber Uplink Ports
FCP270
P92
DFE Platinum
Switches
53

Star Topology (Security Enhanced)
The star topology is the preferred topology for all networks. It is the easiest to maintain, expand,
and deploy. In a star topology, the switches at the outside edge of the network have connections
to each of the two root switches. The two root switches are connected to each other and all edge
switches. Redundant data paths allow the network to continue to operate if any one component
fails.
Figure 2-16 illustrates a star network in the Security Enhanced Configuration. In a Security
Enhanced star topology, as many as 166 edge switches can be connected to the Chassis switch
using 1 Gb uplinks. In this configuration, if an edge switch spanning tree protocol fails, LDP will
remove the loop that was created by the failure. In the event the switch fails in a manner that
causes a network flood, LDP will remove the defective switch from the network, allowing its
redundant switch to take over. An inverted tree topology or modified inverted tree topology can
be considered if there is a larger number of edge switches required.
Figure 2-16. Star Topology (Security Enhanced Configuration)
54

The double star topology allows the user to benefit from the star topology advantages while allow-
ing the user to deploy a much larger sized network.
Bandwidth considerations between the star topology switches must be observed. To minimize
bandwidth consumption, the majority of peer-to-peer end device communications should reside
local to the individual star topologies.
Figure 2-17. Double Star Topology (Security Enhanced Configuration)
Fiber Uplink Ports
N-Series N-Series
Tier 1
Tier 2
Tier 3
N-Series
Chassis
Switch
N-Series
Chassis
Switch
55

Inverted Tree Topology (Security Enhanced)
The inverted tree topology is suited for very large networks with specific physical constraints. In
this topology the switches are arranged in tiers, with the root switches in the top tier and up to
three tiers below them (maximum of four tiers). The two root switches are connected to each
other and the other switches each have a connection to two of the switches in the tiers above and
below them. Redundant data paths allow the network to continue to operate if any one compo-
nent fails. The deployment of the loop detection algorithm allows for redundant network condi-
tion monitoring by RSTP and LDP.
NOTE
When utilizing an Inverted Tree Topology in the Security Enhanced Configura-
tion, all switches within the network must be switch types that support the loop
detection algorithm.
An inverted tree network topology is illustrated in Figure 2-18.
RSTP requirement, limiting the number of switches between devices to seven.
NOTE
The Stepped-Tier Tree variation example of this topology is not shown as a Security
Enhanced Configuration. However, this configuration can be utilized with the
following rule heeded:
All switches in the network must be interfaced with two (2) uplinks to the tier
above it, all uplinks attached to a tier above it must be attached to a switch that is
LDP deployable and if a switch is attached to a tier below it, it too must be LDP
deployable.
Refer to the LDP Deployable switches in the Appendix table Qualified Switch
Standard/Security Enhanced Configuration Compatibilities Matrix in The MESH
Control Network Operation, and Switch Installation and Configuration Guide
(B0700CA) for the applicable switch models.
56

Figure 2-18. Inverted Tree Topology (Security Enhanced Configuration)
57

Modified Inverted Tree Topology (Security Enhanced)
The modified inverted tree topology is similar to the inverted tree topology but with consider-
ations to cost restraints. The modified inverted tree topology is suited for very large networks.
However, this topology utilizes low end, low cost switches as edge devices. In this topology, the
switches are arranged in tiers, with the root switches in the top tier and up to three tiers below
them (maximum of four tiers). The two root switches are connected to each other and the other
switches each have a connection to two of the switches in the tiers above and below them. Low-
end low-cost switches can be added to the outer edge at different tiers. Redundant data paths
allow the network to continue to operate if any one component fails.
By deploying this, the loop detection algorithm allows for redundant network condition monitor-
ing by RSTP and LDP.
An inverted tree network topology is illustrated in Figure 2-19.
I/A Series system requirement, limiting the number of switches between devices to seven.
58

Figure 2-19. Modified Inverted Tree Topology (Security Enhanced Configuration)
I/O Network Topology Configurations
Only linear topologies are supported for use in a dedicated I/O network.
The Linear I/O Network
The linear I/O network can consist of a single A bus switch and a single B bus switch (two
switches total) or up to seven switches interlinked together via 1Gb (ISL) uplinks per bus. Addi-
tional switches should only be used when the total FCM and ZCP port count exceeds the switch
port count, or when the geographical conditions dictate the requirement for more than one
switch. When using Single Mode Fiber ISLs, the distance between two switches can be up to, but
not exceed, 80 km. When using Multi-Mode Fiber ISLs, the distance can be up to 2 km. No
59

crossover ISL links between the A and B buses are supported on the linear I/O network. Refer
to Figure 1-7 on page 14.
Switch and Fiber Cable Budget and Loss
Fiber Cable Budget Cable and Loss
The typical power budget for 100FX system of The MESH control network with 62.5/125 m
fiber is 11.0 db and worst case (includes aging) is 10.0 db.
Table 2-1 lists some typical and worst case power losses for fiber optic cable, splitter/combiners
and patch cords.
Fiber Optic Budgets
Refer to the documentation included with your Invensys qualified Ethernet switches for details of
their fiber optic budgets.
The MESH Control Network Cabling
The following sections provide the specifications and transmission distance capabilities for the
fiber optic, and twisted-pair cabling used in The MESH control network.
Switch and Converter Fiber Optic Cabling
Two types of fiber optic cable can be used to connect devices in The MESH control network:
multimode fiber optic cable and single mode fiber optic cable. Each cable type is used in different
applications and the devices that are connected must support the type of cable being used. The
following sections give the supported uses and maximum transmission distances for each type of
fiber optic cable.
Table 2-1. Fiber Cable Power Losses
Link Element Value Comments
Splitter/Combiner 4.5 db
MT-RJ Patch Cord 0.5 db 0.75 db worst case
LC, SC, and other
similar patch cords
0.15 db 0.5 db worst case
Singlemode fiber
cable (SMF)
0.5 db/km at 1300 nm wavelength
0.275 dB at 1550 nm wavelength
Multimode fiber cable
(MMF)
Splice loss 0.25 dB Mechanical splice
50/125 M mating
62.5/125 M loss
5.25 dB -5.0 dB mating loss and a -0.25 dB for
mechanical splice
Aging reserve 1.0 dB 62.5/125 micron cable
Patch Panel loss 2.0 dB average
60

Multimode Fiber Optic Cabling
Multimode fiber optic cable is used to connect stations to switches, media converters to other
devices, and, with the appropriate uplink module, Ethernet switches to other Ethernet switches.
Invensys recommends that the selected multimode fiber optic cabling have the following charac-
teristics:
62.5 micron core/125 micron cladding
Maximum allowable signal loss = 1 dB/km at a wavelength of 1300 nm
Maximum allowable signal loss = 3.5 dB/km at a wavelength of 850 nm.
Cables with different characteristics can be used, but maximum transmission distance may be
reduced. Cable requirements, such as flexibility, rodent protection, fire retardancy, and durability,
depend on the particular application. Check with your cable installer/vendor for a list of applica-
tion-specific cable characteristics.
Multimode fiber cable offered by Invensys that meet the recommended specifications are given in
Table 2-2, Table 2-3, and Table 2-4. Copper cable offered by Invensys that meet the recom-
mended specifications are given in Table 2-5.
Refer to the diagrams in the documentation included with your Invensys qualified Ethernet
switches for detailed information on cabling.
Table 2-2. Multimode Fiber Cables with LC Connectors
Cable Length
Connector
Type
Minimum
Bend
Radius
P0972UN 0.5 m (1.5 ft) (gray) LC
2 inches
P0972VG 0.5 m (1.5 ft) (orange) LC
P0972UJ 1 m (3 ft) LC
P0972TN 3 m (9.8 ft) LC
P0972TP 15 m (50 ft) LC
P0972TQ 50 m (165 ft) LC
P0972WX (50 micron/mode
conditioning cable)
3 m (10 ft) LC
Table 2-3. Multimode Fiber Cables with MT-RJ to ST Connectors
Cable Length
Connector
Type
Minimum
Bend
Radius
P0972KV 3 m (9.8 ft) MT-RJ
2 inches P0972KW 15 m (50 ft) MT-RJ
P0972KX 50 m (165 ft) MT-RJ
61

Single Mode Fiber Optic Cabling
Each switch must be equipped with an appropriate uplink module in order to use single mode
fiber optic cable. Invensys recommends that the selected single mode fiber optic cabling have the
following characteristics:
8-10 micron core/125 micron cladding
Typical allowable signal loss = 0.5 dB/km at a wavelength of 1300 nm or 0.275 dB at
a wavelength of 1550 nm.
Cables with different characteristics can be used, but maximum transmission distance might be
reduced. Cable requirements, such as flexibility, rodent protection, fire retardancy, and durability,
depend on the particular application. Check with your cable installer/vendor for a list of applica-
tion-specific cable characteristics. Invensys does not offer single mode fiber optic cables. SMF
cables are customer supplied.
Transmission distances for single mode fiber optic cable are given in Table 2-6.
Invensys does offer single mode fiber optic cables for use with patch panels. The MESH control
network uses cables with MT-RJ connectors which have a pair of fibers which crossover within
the cable (TX-to-RX). When patch panels are used on site, if they are constructed using MT-RJ
connectors, consider the number of panels in series between end devices which need to be built.
For instance, if a cable run is terminated with two patch panels and also connects to end devices,
and the patch panel was constructed so that the cable is straight-through, then using two cross-
over cables (one at each end) will not work.
Table 2-4. Multimode Fiber Cable with LC and SC Connectors
Cable Length Connector Type
P0972WW 3 m (9.8 ft) Two ceramic-type LC connectors on one
end, with SC connectors on the other end
Table 2-5. Copper Cables with RJ-45 Connectors
Cable Length
Connector
Type
P0972UB 0.5 m (1.5 ft) RJ-45
P0971XK 3 m (9.8 ft) RJ-45
P0971XL 15 m (50 ft) RJ-45
P0972MR 30 m (110 ft) RJ-45
P0971XM 50 m (165 ft) RJ-45
P0971XN 100 m (330 ft) RJ-45
Table 2-6. Single Mode Fiber Optic Cable - Maximum Transmission
Application Transmission Protocol Max. Distance
Switch to Switch 1000Base-LX 10000 m (32808 ft)
SMF cable is supplied by the customer
62

Connector Types for Switches
Refer to the documentation included with your Invensys qualified Ethernet switches for their
connector types and uplink ports.
When patch panels are used on site, and if they are constructed using MT-RJ connectors,
consider the number of panels in series between end devices which need to be built. For instance,
if a cable run is terminated with two patch panels and also connects to end devices, and the patch
panel was constructed so that the cable is straight-through, then using two crossover cables (one at
each end) will not work.
Patch Panels
The MT-RJ jumper cables go straight through and the connector on the switch serves as the
crossover. When patch panels are created using MT-RJ connectors, if the fiber run between two
patch panels has a crossover, then the jumpers supplied by Invensys will not work. See
Figure 2-20.
Table 2-7. Single Mode Fiber Optic Jumper Cables
Description Length Part Number
Single Mode Fiber (SMF) fiber optic patch cable w/ LC and
SC connectors. Cable to be configured as a null modem (i.e.,
input/output crossed)
3 m (9.8 ft) P0973EW
ST connectors.
3 m (9.8 ft) P0973EX
ST connectors.
15 m (50 ft) P0973FY
ST connectors.
50 m (165 ft) P0973FZ
Single Mode Fiber (SMF) fiber optic patch cable w/ LC - LC
connectors.
3 m (9.8 ft) P0973FV
connectors.
15 m (50 ft) P0973FW
connectors.
50 m (165 ft) P0973FX
63

Figure 2-20. Switch to Switch and Switch to Patch Panel Connections
FCP270, ZCP270, FCM100Et and FCM100E Fiber Signal Cabling
The cables listed in Table 2-8 are offered for use in making the 100 Mb MESH control network
fiber optic cable connections. Whether these items are shipped to your location depends on the
100 Mb network cabling configuration specified at the time of system purchase.
Fiber optic cabling connects to the Ethernet 100 Mb switches using one MT-RJ connector and
connects to the splitter/combiner for the FCP270, ZCP270, or FCM100Et using two ceramic
type LC fiber optic connectors. These same connectors can connect the switches directly to a
FCM100E. The maximum optical insertion loss though each connector must be equal to or less
than 0.5 db. For the fiber optic cable lengths, refer to Table 2-8. Figure 2-21 shows the cabling for
the FCP270, ZCP270, FCM100Et, FCM100E or patch panels.
Switch to Switch Connection
Switch to Patch Panel Connection
Switch
TX
RX
TX
RX
TX
RX
TX
RX
P1
P4
Switch
RX
TX
RX
TX
RX
TX
RX
TX
P4
P1
Switch
TX
RX
TX
RX
TX
RX
TX
RX
P1
P4
Switch
RX
TX
RX
TX
RX
TX
RX
TX
P4
P1
Patch
TX
RX
TX
RX
TX
RX
RX
TX
RX
TX
RX
TX
Patch
TX
RX
TX
RX
TX
RX
RX
TX
RX
TX
RX
TX
TX
RX
RX
TX
TX
RX
RX
TX
64

Table 2-8. Fiber Optic Cables
Part
Number Connector Length Color Use
Cable
Material
P0972TR LC to
MT-RJ
3 m (9.8 ft) No specified
color
Switch to Split-
ter/Combiner
FCP270/
ZCP270/
FCM100Et
or directly to
FCM100E
(without Split-
ter/Combiner)
MMF
62.5/125
micron
P0972TS 15 m (49.5 ft)
P0972TT 50 m (165 ft)
- Customer sup-
plied, over 50 m
(165 ft) to 2 km
(6600 ft) fiber
optic cable
P0972 UJ LC to LC 1.0 m (3 ft)
P0972TN 3 m (10 ft)
P0972TP 15 m (50 ft)
P0972TQ 50 m (165 ft)
P0972ZQ 0.25 m (1.0 ft)
P0972UN 0.5 m (2.1 ft) Gray color for
A Bus
P0972VG 0.5 m (2.1 ft) Orange color
for B Bus
P0972VD ST to
MT-RJ
3 m (9.8 ft) No specified
color
Interface
MT-RJ devices
to an ST-type
patch panel
MMF
62.5/125
micron
P0972VE 15 m (49.5 ft)
P0972VF 50 m (165 ft)
P0972WW LC to SC 3 m (9.8 ft) Interface legacy
P0972LQ
(GPIM-01) to
P0972WT
(MGBIC-LC01)
Mode condi-
tion MMF
P0972WX 3 m (9.8 ft)
P0972QP SC to SC 3 m (9.8 ft) SMF
P0973EW LC to SC 3 m (9.8 ft) Interface legacy
P0972LR
(GPIM-09) to
P0972WU
(MGBIC-LC09)
SMF 10
micron
P0973EX LC to ST 3 m (9.8 ft) Interface
P0972WU
(MGBIC-LC09)
to an ST patch
panel
65

Figure 2-21. FCP270, ZCP270, FCM100Et and FCM100E Signal Cabling

F
i
b
e
r

1
0
0
B
a
s
e
-
F
X

S
W
I
T
C
H

W
/
M
T
R
J

C
O
N
N
E
C
T
O
R
S
C
o
p
p
e
r

1
0
0
B
a
s
e
-
T
X

S
W
I
T
C
H

W
/
R
J
-
4
5

C
O
N
N
E
C
T
O
R
S
C
o
p
p
e
r

1
0
0
B
a
s
e
-
T
X

S
W
I
T
C
H

W
/
R
J
-
4
5

C
O
N
N
E
C
T
O
R
S

F
I
B
E
R

O
P
T
I
C
C
A
B
L
E

G
R
O
U
P

E

C
O
P
P
E
R
C
A
B
L
E

G
R
O
U
P

A
C
A
B
L
E

G
R
O
U
P

A
1
0
0
B
a
s
e
-
T
X
,

C
A
T

5
P
0
9
7
2
U
B

=

1

m
P
0
9
7
1
X
K

=

3

m
P
0
9
7
1
X
L

=

1
5

m
P
0
9
7
2
M
R

=

3
0

m
P
0
9
7
1
X
M

=

5
0

m
P
0
9
7
1
X
N

=

1
0
0

m
N
O
T
E
:

P
r
o
c
e
s
s
o
r

I
/
O

c
a
r
d
s

a
r
e

i
l
l
u
s
t
r
a
t
e
d

f
o
r

c
a
b
l
i
n
g

p
u
r
p
o
s
e
s

o
n
l
y
.
O
P
T
I
O
N
A
L

1

G
B
i
t

U
P
L
I
N
K
S
.
R
E
Q
U
I
R
E

U
P
L
I
N
K

G
B
I
C

M
O
D
U
L
E
S
(
P
0
9
7
2
W
T
,

P
0
9
7
2
W
U
,

P
0
9
7
2
Y
L
,
P
0
9
7
2
Y
Q
,

P
0
9
7
3
G
H
,

P
0
9
7
3
G
J
,
P
0
9
7
3
F
T
,

o
r

P
0
9
7
3
F
U
)

F
i
b
e
r

1
0
0
B
a
s
e
-
F
X

S
W
I
T
C
H

W
/
M
T
R
J

C
O
N
N
E
C
T
O
R
S
*

=

N
U
L
L

M
O
D
E
M

C
A
B
L
E
#

=

1
:
1

C
A
B
L
E
C
O
P
P
E
R

(
i
e
,

C
u
)

E
T
H
E
R
N
E
T

C
A
B
L
E
F
I
B
E
R

(
i
e
,

F
O
)

E
T
H
E
R
N
E
T

C
A
B
L
E
G
R
E
E
N
M
A
G
E
N
T
A
Z
C
P
2
7
0
Z
C
P
2
7
0
C
O
N
N
E
C
T
I
O
N

T
O

Z
C
P
2
7
0
F C P 2 7 0
( P 0 9 1 7 Y Z )
C
O
N
N
E
C
T
I
O
N

T
O

F
C
P
2
7
0
O P T I C A L
S P L I T T E R
O P T I C A L
S P L I T T E R
P
0
9
7
2
U
N
(
G
R
A
Y
)
P
0
9
7
2
V
G
(
O
R
A
N
G
E
)
P
0
9
2
6
A
H
F C P 2 7 0
( P 0 9 1 7 Y Z )
F C P 2 7 0
( P 0 9 1 7 Y Z )
O P T I C A L
S P L I T T E R
O P T I C A L
S P L I T T E R
P
0
9
7
2
U
N
(
G
R
A
Y
)
P
0
9
7
2
V
G
(
O
R
A
N
G
E
)
P
0
9
2
6
A
H
P
0
9
2
2
K
Y
1
0
/
1
0
0
B
a
s
e
-
T
X
E
T
H
E
R
N
E
T

P
C
I

C
A
R
D
P
0
9
2
2
K
Y
1
0
/
1
0
0
B
a
s
e
-
T
X
E
T
H
E
R
N
E
T

P
C
I

C
A
R
D
F
I
B
E
R

C
A
B
L
E

G
R
O
U
P

E
M
M
F

w
i
t
h

M
T
-
R
J
/
L
C

C
A
B
L
E
S
P
0
9
7
2
T
R

=

3

m
P
0
9
7
2
T
S

=

1
5

m
P
0
9
7
2
T
T

=

5
0

m
C
U
S
T
O
M
E
R

S
U
P
P
L
I
E
D

=

>
5
0
m
,

<

2

k
m
T W O S I N G L E
E T H E R N E T C A R D ( S )
O P T I C A L
S P L I T T E R
P
0
9
2
6
A
H
C
O
N
N
E
C
T
I
O
N

T
O

F
C
M
1
0
0
E
t

o
r

F
C
M
1
0
0
E
O P T I C A L
S P L I T T E R
F
B
M
F
B
M
F
B
M
F
B
M
P
0
9
7
2
U
N
(
G
R
A
Y
)
P
0
9
7
2
V
G
(
O
R
A
N
G
E
)
D I N R A I L
M O U N T E D
T
H
E

M
E
S
H
C
O
N
T
R
O
L
N
E
T
W
O
R
K
I/A
S
e
r
ie
s
I/A
S
e
r
ie
s
I/A
S
e
r
ie
s
I/A
S
e
r
ie
s
I/A
S
e
r
ie
s
I/A
S
e
r
ie
s
I/A
S
e
r
ie
s
S
I
N
G
L
E

F
C
M
A
F C M 1 0 0 E * *
( P 0 9 7 2 Z A )
I/A
S
e
r
ie
s
B
F C M 1 0 0 E * *
( P 0 9 7 2 Z A )
AB
F C M 1 0 0 E t * *
( P 0 9 2 6 G S )
R
E
D
U
N
D
A
N
T

P
A
I
R
S
I
N
G
L
E

F
C
P
ABA
AB
B
F
A
U
L
T
T
O
L
E
R
A
N
T

P
A
I
R
S
W
I
T
C
H

G
R
O
U
P

A
S
W
I
T
C
H

G
R
O
U
P

A
S
W
I
T
C
H

G
R
O
U
P

B
S
W
I
T
C
H

G
R
O
U
P

B
AB
F C M 1 0 0 E t * *
( P 0 9 2 6 G S )
O P T I C A L
S P L I T T E R
O P T I C A L
S P L I T T E R
I/A
S
e
r
ie
s
F
B
M
I/A
S
e
r
ie
s
F
B
M
P
R
O
C
E
S
S
O
R

C
A
R
D
S
(
W
o
r
k
s
t
a
t
i
o
n
)
P
0
9
7
2
Z
Q

P
0
9
7
2
Z
Q

P
0
9
7
2
Z
Q

*
*

A
l
s
o
,

F
C
M
1
0
0
E

m
o
d
u
l
e
s

(
P
0
9
7
2
Z
A
)
m
a
y

b
e

u
s
e
d

i
n

p
l
a
c
e

o
f

F
C
M
1
0
0
E
t
s
.
T
h
e

F
C
M
1
0
0
E

m
o
d
u
l
e
s

d
o

n
o
t

r
e
q
u
i
r
e
s
p
l
i
t
t
e
r
/
c
o
m
b
i
n
e
r
s

f
o
r

c
o
n
n
e
c
t
i
o
n

t
o

t
h
e

s
w
i
t
c
h
e
s
.
66

Twisted-Pair Cabling
Category 5 (CAT5), shielded twisted-pair (STP) copper cable is used in The MESH control net-
work for connecting switches to RJ-45 ports on other devices. CAT5 cable can be used, with the
appropriate uplink module, to connect copper switch ports to other copper switch ports or media
converters.
Transmission distances for CAT5 copper cable are given in Table 2-9.
Table 2-10 lists the prefabricated CAT5 STP cables with RJ-45 connectors that are offered by
Invensys.
NOTE
Plenum grade jackets are supplied on cables over three meters in length that are
offered by Invensys. This applies to both fiber optic and CAT5 copper cables.
Null Hub
A null hub is a very short cable that has a male RJ-45 connector on one end and a female RJ-45
connector on the other. The transmit and receive wires are reversed, so when it is connected to
the end of a straight-through cable, the resulting cable system acts as a crossover cable. The null
hub is used to connect switches that do not have an MDI crossover port or an auto MDI/MDI-X
port.
Table 2-9. CAT5 Cable - Maximum Transmission Distance
Application Transmission Protocol Max. Distance
Media converter to RJ-45 port,
switch to switch, port to port
100Base-TX and
1000Base-T
100 m (328 ft) for all
applications
Table 2-10. Prefabricated CAT5 STP Cables with RJ-45 Connectors
CAT5 Cable Length
Connector
Type
P0972UB 1 m (3.3 ft) RJ-45
P0971XK 3 m (10 ft) RJ-45
P0971XL 15 m (50 ft) RJ-45
P0972MR 30 m (100 ft) RJ-45
P0971XM 50 m (165 ft) RJ-45
P0971XN 100 m (330 ft) RJ-45
Table 2-11. Null Hub
Null Hub Part Number Length
P0971PK 0.3 m (1 ft)
67

3. Installation (Cabling)
This chapter describes the steps necessary to install the cabling that enables The MESH control
network equipment to communicate.
Installation of The MESH control network consists of locating the individual components and
then making connections between them using fiber optic or copper cable. The following para-
graphs provide the information necessary for the cabling installation of the high performance sys-
tem network.
NOTE
Refer to the documentation included with your Invensys qualified Ethernet
switches to install these switches and the media converters as part of The MESH
control network.
Connecting The MESH Control Network Components
Fiber Optic Cabling Guidelines
The following guidelines should be followed when making fiber optic connections between
devices in The MESH control network:
Minimum bend radius - Fiber optic cable can be damaged if it is bent too sharply.
Maintain a minimum bend radius of two inches when using fiber optic cable supplied
by Invensys. Follow the cable manufacturers guidelines if third-party fiber optic cable
is used.
Cable straps - Cable straps, if used, should be designed for use with fiber optic cable.
Hard plastic cable ties can damage fiber optic cable and are not recommended.
Dust caps - Dust caps should remain in place to protect the polished cable ends over
the cable ends and in the switches while cables are being routed. Dust caps should be
installed on unused switch ports and device ports.
Interconnecting Ethernet Switches
Depending on which switch model has been selected as the root network switch, additional
switches may be connected in one of several ways. They are summarized in Table 3-1 below. Refer
to the Ethernet Switch Interconnection Diagrams in the documentation included with your
Invensys qualified Ethernet switches for detailed cabling diagrams.
B0700AZ Rev L 3. Installation (Cabling)
68

CAUTION
!
If Fast Ethernet Device ports are used as uplink ports between switches on an N-
Series chassis switch configuration, it is very important that the switch spanning
tree protocol settings be configured correctly. Failure to do so will cause system deg-
radation during switch failover, causing excessive packet flooding possibly resulting
in device failures. (Refer to The MESH Control Network Operation, and Switch Instal-
lation and Configuration Guide (B0700CA).)
CAUTION
!
Do not make direct interconnection between ports of the same switch (i.e. loop-
back). Loopbacks can result in network failures upon a loss or failure of RSTP.
Also, refer to page 61 for a list of single mode fiber optic cables for use at patch panels.
NOTE
All graphics of switches and media converters below are intended as generic illustra-
tions of networking concepts and do not necessarily reflect the currently offered
products.
NOTE
These sections pertain to both standard switches and chassis switches.
Uplink Port to Uplink Port
The Ethernet switches can be connected to each other through the use of an optional uplink port.
Depending upon which uplink module is installed, the switch is able to communicate over single
mode or multimode fiber optic cable, or CAT5 copper cable. Transmission via uplink ports is
made using the Gigabit Ethernet protocol. The fiber optic cable uses LC type connectors and
should be designed for the mode (single mode vs. multimode) of uplink module that is installed
in the Ethernet switchs interface slot. Figure 3-1 shows a connection between two Ethernet
switches using their fiber optic uplink ports.
Table 3-1. Methods of Connecting Ethernet Switches
Root Switch Port End Switch Port
Uplink port (Gigabit Ethernet) copper and
fiber
Uplink port (Gigabit Ethernet)
Fiber optic port (100Base-FX) Fiber optic port (100Base-FX)
Fiber optic port (100Base-FX) RJ-45 (100Base-TX)
RJ-45 (10Base-T/100Base-TX) RJ-45 (10Base-T/100Base-TX)
RJ-45 (10Base-T/100Base-TX) Fiber optic port (100Base-FX)
3. Installation (Cabling) B0700AZ Rev L
69

Figure 3-1. Switch-to-Switch Fiber via Uplink Port
Managed switches can be connected directly to other switches connecting each switchs fiber optic
ports with fiber optic cable.
NOTE
Make sure that the transmit (TX) port of one switch is connected to the receive
(RX) port of the other switch. The standard Invensys LC to LC cable ensures proper
TX to RX connections.
Fiber optic cable can be used to connect fiber switches to other fiber switches using each devices
100Base-FX fiber ports. Multimode fiber cable with MT-RJ connectors is connected between a
port on each switch. Figure 3-2 gives an example of a port-to-port connection via fiber optic
cable. However, even though this method can be used for uplink ports it is not recommended due
to the 100 Mb port's bandwidth capabilities. This method of uplinks must not be used for the
Security Enhanced Configuration topology designs. Normally 1 Gb uplink connections should be
used to interconnect switches in all standard or enhanced network designs.
Figure 3-2. Port-to-Port Connection via Fiber Optic Ports
Fiber Switch
62.5/125 micron Single/Multimode
w/LC-type Connectors
Fiber Optic Cable
1000Base-SX, 1000Base-LX or 1000Base-ZX
Fiber Switch
Fiber Switch
Fiber Switch
62.5/125 micron Multimode
w/LC Connectors
Fiber Optic Cable
100Base-FX
B0700AZ Rev L 3. Installation (Cabling)
70

CAUTION
!
If Fast Ethernet Device ports are used as uplink ports between switches, it is very
important that the switch spanning tree edge-port protocol settings be configured
correctly. Failure to do so will cause system degradation during switch failover, caus-
ing excessive packet flooding possibly resulting in device failures. (Refer to The
MESH Control Network Operation, and Switch Installation and Configuration Guide
(B0700CA).)
RJ-45 Port to RJ-45 Port
Switches can be connected to other switches through an RJ-45 port on each switch. It is necessary
that the first switchs transmit signal is connected to the second switchs receive line. The same is
true for the second switchs transmit and receive signals. However, even though this method can
be used for uplink ports it is not recommended due to the 100 Mb port's bandwidth capabilities
and the RJ-45 auto-negotiation characteristics. This method of uplinks is not recommended for
the enhanced network topology designs. Normally 1 Gb uplink connections should be used to
interconnect switches in all standard or enhanced network designs. Figure 3-3 shows a connection
between a copper switches with auto MDI/MDI-X ports.

Figure 3-3. Connecting Switches via RJ-45 Ports
In the event that neither switch has an MDI/MDI-X port, a separate crossover cable (P0971PK -
with a 1 ft cable), called a null hub, must be used between the switches.
Switch Configuration
Refer to The MESH Control Network Operation, and Switch Installation and Configuration Guide
(B0700CA) and the documentation included with your Invensys qualified Ethernet switches for
instructions on configuring them for The MESH control network.
Copper Switch
100Base-TX
Auto
MDI/MDI-X
Auto
MDI/MDI-X
Copper Switch
71

4. Maintenance
This chapter provides information on identifying the cause of problems with The MESH
control network.
The MESH Control Network Addresses
All I/A Series equipment IP addresses for The MESH control network are 151.128.81.x (where x
is a number between 1 and 254 that identifies the specific Ethernet switch). IP addresses assigned
to the primary port of controllers (FCP270, ZCP270), FCM100Ets, FCM100Es, and worksta-
tions are 151.128.y.z (where y is a number between 152 and 191, and z is a number between 1
and 254). The IP addresses for the alternate ports of FCM100Ets, FCM100Es or workstations are
151.128.w.z (where w is derived from y above as w = y - 128, and z is the same value as above).
The IP addresses for the primary I/O controller ports of ZCP270 are 151.128.v.z (where v is
derived from y above as v = y + 64, and z is the same value as above). The IP addresses for the
alternate Input/Output Controller (IOC) ports of ZCP270 are 151.128.u.z (where u is derived
from y above as u = y - 64, and z is the same value as above). See Table 4-1.
Fault tolerant FCP270s and ZCP270s use the same IP addresses for both modules of the fault tol-
erant pair. Additionally, the ZCP270 requires a third and fourth IP address for its Input/Output
Controller (IOC) ports. These also are derived and assigned according to Table 4-1.
IP addresses for all devices on The MESH control network (workstations, controllers, managed
switches, FCM100Ets and FCM100Es) are assigned by the system configurator (SysDef or
IACC). For information on using the system configurator, refer to:
I/A Series System Definition: A Step-by-Step Procedure (B0193WQ) or
I/A Series Configuration Component (IACC) Users Guide (B0400BP).
Table 4-1. IP Address Assignments
I/A Series Release Port IP Address
v8.0 or later Network Monitoring Devices 151.128.82.1 through 151.128.82.254
v8.0 or later Primary Port 151.128.152.1 through 151.128.191.254
v8.0 to v8.1.x Alternate Port
1
1.
At v8.2, the intermediate driver was changed so that the Alternate Port IP is never used.
151.128.24.1 through 151.128.63.254
v8.0 or later Primary IOC Port 151.128.216.1 through 151.128.255.254
v8.0 or later Alternate IOC Port 151.128.88.1 through 151.128.127.254
v8.0 or later Switches 151.128.81.1 through 151.128.81.254
Pre-v8.0 Single Node 151.128.8.65 through 151.128.8.126
Pre-v8.0 Multi-Node 151.128.16.65 through 151.128.23.254
Pre-v8.0 Switches 151.128.79.1 through 151.128.79.254 or
151.128.80.1 through 151.128.80.254
B0700AZ Rev L 4. Maintenance
72

Once the configurator has assigned the primary IP address, each Ethernet switch must be config-
ured with its assigned IP address using the Command Line Interface (CLI) Entries or I/A Series
Switch Configurator Application Software (discussed in the switch configuration section of the
documentation included with your Invensys qualified Ethernet switches). One IP address is
assigned to each non-Chassis Ethernet switch in The MESH control network. For the N1 Chassis
switch (P0973AR), the chassis is assigned one IP address. For the N3 Chassis switch (P0973AS),
the chassis is assigned one IP address for each of its three blades. For the N7 Chassis switch
(P0972YE), the chassis is assigned one IP address for each of its seven blades.
The system configurator (SysDef or IACC) assigns a MAC (Media Access Control) address for
each FCP270, ZCP270, FCM100Et, and FCM100E on the network. The primary port MAC
addresses are assigned from the range, 00006CC00000 to 00006CC03FFF. The alternate port
MAC addresses are derived from the primary port MAC addresses by ORing the primary net-
work MAC address with 000000004000, yielding a range of 00006CC04000 to
00006CC07FFF. The MAC addresses for the primary I/O controller ports of ZCP270 are derived
from the primary port MAC addresses by ORing the primary network MAC address with
000000008000, yielding a range of 00006CC08000 to 00006CC0BFFF. The MAC addresses for
the alternate I/O controller ports of ZCP270 are derived from the primary port MAC addresses
by ORing the primary network MAC address with 00000000C000, yielding a range of
00006CC0C000 to 00006CC0FFFF.
Workstations, FCP270, ZCP270, FCM100Et, FCM100E and FBMs are assigned letterbugs to
identify the module by the system configurator. FBMs are not assigned IP or MAC addresses. Let-
terbugs and IP addresses can be changed using the system configurator
General Troubleshooting Guidelines
When a problem occurs with The MESH control network, it is usually best to take an organized
approach to diagnosing the cause. A random or scattered approach generally takes longer and can
make it more difficult to track down the cause of the problem by introducing additional
unknowns. The following sections will help to determine the cause of a network problem.
Characterize the Problem
The initial step in diagnosing network problems is to understand what kind of problem exists.
Has the problem always existed or has it just started?
New problems are sometimes caused by a change in The MESH control network con-
figuration. Check to see whether there have been recent changes to The MESH con-
trol hardware, software, or configuration files.
Is the problem constant or intermittent?
Constant problems are often the result of a component failure or a change to The
MESH control network configuration. Intermittent problems can be caused by envi-
ronmental factors such as excess heat, electrical noise, poor contacts, or high attenua-
tion.
What kind of error indication is occurring?
Device communication failure - inability to communicate with one or more devices
on The MESH control network. This happens when the information path has been
broken or interrupted. Possible causes are damaged or misconnected cabling, a failure
in a network device, or high attenuation.
4. Maintenance B0700AZ Rev L
73

Data Loss - incorrect data is arriving at one or more devices. The data path is intact,
but the data is being corrupted along the way. This can be caused by failing network
devices, environmental factors, or exceeding transmission distance limits.
Error message - software has detected a problem. This can be caused by any number
of things. The text of the error message indicates what kind of problem has been
detected.
Determine Which Devices are Affected
Identifying the devices that are affected helps to determine where the cause of the problem lies.
For example, if all the unreachable devices are connected to the same Ethernet switch, then you
should probably start troubleshooting at that switch.
When troubleshooting a Security Enhanced Configuration with LDP deployed, it is highly rec-
ommended that you refer to the section Loop Detection Monitoring and Maintenance in The
MESH Control Network Operation, and Switch Installation and Configuration Guide (B0700CA)
before continuing.
Follow the steps below to identify which device or devices are affected.
1. Are the problems limited to specific devices (stations, switches), or are they general in
nature?
2. Are the problems constant or are they intermittent?
3. Can the devices be reached using the ping command?
4. Does the ping -s command report that packets are being dropped?
5. What does System Management Displays indicate? Refer to System Management
Displays on page 74.
6. Use a network topology map to determine the physical location of the affected
devices.
Troubleshoot the Affected Devices
1. Check activity and status indicators on the affected devices.
2. Verify that the devices are getting power (check Status LEDs).
3. Check that the devices are correctly cabled and that the cables are not damaged (swap
a suspect cable with a known good cable).
4. Verify that the limit on distance between devices has not been exceeded. Distance is
dependent on transmission mode and cable type.
5. If fiber optic cable is being used, verify that the optical budget (total signal loss - dB),
bandwidth, and dispersion are within specification.
6. Verify that traffic rates to the affected devices are not excessive.
7. Check System Management error counters. Refer to System Management Displays
(B0193JC).
8. Verify that the devices are properly configured. Also check that the adjacent device in
the network is correctly configured as well.
74

NOTE
Before removing a switch from the network, be sure to record its existing configura-
tion. This information is necessary when the replacement switch is configured.
NOTE
Make sure a replacement switch is correctly configured before adding it to the net-
work. Any uplink ports should be installed before the switch is configured. Refer to
the documentation included with your Invensys qualified Ethernet switches for
more information.
System Management Displays
The I/A Series System Management Display Handler (SMDH) obtains current and historical
information about the system, and displays this information in System Management displays.
SMDH uses SNMP to access the switches to obtain status information from each switch. With
regard to The MESH control network and associated Ethernet switches, SMDH provides the fol-
lowing displays:
A Switched Network View of all the configured switches
A Switch Domain display that shows all The MESH control network stations in the
selected Switchs domain.
A Switch Ports Display which shows each port of the switch that was selected on the
Domains display
Detailed equipment information (EQUIP INFO) and equipment change (EQUIP
CHG) displays for each port of the selected Ethernet switch.
Accessing SMDH Switch Network Displays
System Management Displays are accessed from the FoxView application.
To access System Management on Windows based systems or Solaris based systems with the
FoxView application, refer to System Management Displays (B0193JC).
The initial System Management Display, System Monitor Domains, appears. From this display
you can select a system monitor and navigate through the Domain Display(s).
All switches are shown in the system monitor domain as stations designated by their letterbug. To
access displays for The MESH control network switches, select (click-on) the menu bar soft keys
indicated in Figure 4-1.
75

Figure 4-1. Accessing SMDH Switched Network Displays
Domains
Switched
Network Display
Click Switch
Equipment
Change Display
System Monitor
Click SW NTWRK
Domains
Click EQUIP CHG
Click EQUIP INFO
Click CONFIG INFO
Click NEXT LEVEL
Switch
Domain Display
Click Switch
Click EQUIP CHG
Click EQUIP INFO
Click CONFIG INFO
Click NEXT LEVEL
Equipment
Information Display
Equipment
Configuration Display
Switch
Ports Display
Click Port
Click EQUIP CHG
Click EQUIP INFO
Equipment
Change Display
Equipment
Information Display
(Figure 4-2)
(Figure 4-6)
(Figure 4-7)
(Figure 4-3)
(Figure 4-4)
(Figure 4-5)
(Figure 4-8)
(Figure 4-9)
76

Switched Network Display
A typical Switched Network display is shown in Figure 4-2. The display is accessed from the Sys-
tem Monitor Domains display by clicking on the SW NTWRK soft key.
It displays the letterbug system name of every switch in the network. Note that each card (blade)
in an E7 Chassis switch is assigned a letterbug, where the N7 Chassis switch is only assigned one
letterbug. Media converters and extenders are not assigned letterbugs.
From the Switched Network Display you can select a switch (click on the letterbug of a particular
switch of interest) and then click on the EQUIP CHG, EQUIP INFO, CONFIG INFO, or
NEXT LEVEL. Clicking NEXT LEVEL brings up the Switch Domain Display.
Figure 4-2. SMDH Switched Network Display
77

Switch Equipment Change Display
A typical Switch Equipment Change display is shown in Figure 4-3. The display is accessed from
the Switched Network display or the Switch Domain display.
The Switch Equipment Change display enables/disables station (switch) alarms and reports from
being propagated to the I/A Series system. Table 4-2 describes only the available text fields in the
order that they appear on the display pages, from left column to right column.
Figure 4-3. Switch Equipment Change Display
Table 4-2. Switch Equipment Change Display Actions
Action Description
ENABLE STATION
ALARMING
Enables device status to affect the overall system status.
INHIBIT STATION
ALARMING
Inhibits device status to affect the overall system status.
ENABLE
ALL REPORTS
Enables reports between the device and the I/A Series System Man-
agement.
DISABLE
ALL REPORTS
Disables reports between the device and the I/A Series System Man-
agement.
78

Switch Equipment Information Display
A typical Switch Equipment Information display is shown in Figure 4-4. The display is accessed
from the Switched Network display or the Switch Domain display. It shows the name, type, and
status of the switch.
Table 4-3 describes only the available text fields in the order that they appear on the display pages,
from left column to right column.
Figure 4-4. Switch Equipment Information Display
Table 4-3. Switch Equipment Information Display Fields
Field Description
STATION NAME
Name (letterbug) of the selected station.
RUN MODE
On-line or Off-line (default) is displayed.
FAIL ACK STATE
Acknowledged (default) or Not Acknowledged is displayed. If the FAIL
STATE value changes from Not Failed to Failed, the FAIL ACK STATE
value changes to Not Acknowledged to indicate this transition, and
remains until you acknowledge the switch failure.
FAIL DEV ACK
Acknowledged (default) or Not Acknowledged is displayed. If any of
the switch ports fail and become unacknowledged, Not Acknowledged
is displayed.
MAC ADDRESS
The MAC Address of the station.
TYPE
The number of ports for a particular switch: Sw w/16 ports, Sw w/24
ports, Sw w/48 ports. For the N1, N3, N7 Chassis switches, displays
Ethernet Switch. The ports for all switches are displayed on the
Switch Ports Display.
79

FAIL STATE
Fail or Not Failed is displayed.
ALARMING STATE
Alarming State indicates whether alarming is enabled or inhibited for
the switch. When alarming is inhibited, the System Monitor continues
to indicate overall system and network health (a green SYS BAR).
FAIL DEV ATT
Yes is displayed if one or more switch ports has failed; otherwise, No is
displayed.
MT REPORT STATE
Master Timekeeper Report State. GPS Not Configured is always dis-
played for the switch equipment information display.
Table 4-3. Switch Equipment Information Display Fields (Continued)
Field Description
80

Switch Configuration Information Display
A typical Switch Configuration Information display is shown in Figure 4-5. The display is
accessed from the Domains display.
Table 4-4 describes the available text fields in the order that they appear on the display pages,
from left column to right column.

Figure 4-5. Switch Configuration Information Display
Table 4-4. Switch Configuration Information Display Fields
Field Description
STATION NAME
Name (letterbug) of selected station.
SMON NAME
System Monitor name for the selected station.
IP ADDRESS
IP Address for the selected station.
SW CONNECTION
Lists all of the switches to which this switchs ports are connected such
as; SW001B, SW003A, SW004B.
TYPE
The number of ports for a particular switch: Sw w/16 ports, Sw w/24
ports, Sw w/48 ports, up to a Sw w/672 ports.
SMON AP NAME
The name (letterbug) of workstation that is hosting SMON.
BOOT HOST
Name (letterbug) of boot host for the switch.
81

Switch Domain Display
The Switch Domain display for a switch is shown in Figure 4-6. The display is similar for all
switches.
From the Switch Domain Display you can select a switch (click on the letterbug of the switch)
and then click on the EQUIP CHG, EQUIP INFO, CONFIG INFO, or NEXT LEVEL. The
EQUIP CHG, EQUIP INFO, CONFIG INFO displays are the same as those that can be
selected from the Switched Network Display. Clicking NEXT LEVEL brings up the Switch
Switch Ports Display.
Figure 4-6. Switch Domain Display
The SWITCH DOMAIN DISPLAY indicates the status of The MESH control network Ethernet
cables to/from the modules connected to the switch. If a fault in cable A or cable B is detected, the
faulty Ethernet cable is identified by the mnemonic RA (Receive A), RB (Receive B), or RAB
(Receive A and Receive B) appearing next to the receive cable as shown below:
RA RB
82

Switch Ports Display
A typical Switch Ports Display for a 16-port switch is shown in Figure 4-7. The Switch Ports Dis-
play, displays a maximum of 30-ports on one screen. Paging is enabled so that more than 30-ports
can be shown on the display.
The carrot < next to the port number in Figure 4-7 indicates that the port alarming has been
inhibited (see Switch Port Equipment Change Display on page 83). All unused ports must be
inhibited, disabling switch alarming for that port. This stops false alarms from propagating
upward in the I/A Series System Management.
The asterisk * adjacent to the port number in Figure 4-7 indicates that the port has failed Lost
Link and should be investigated to determine the cause of the port failure. Refer to General
Troubleshooting Guidelines on page 72 to aid in the isolation of the failure.
Figure 4-7. Switch Ports Display - Typical
83

Switch Port Equipment Change Display
The Equipment Change display for a switch is shown in Figure 4-8. The display is similar for all
switch ports. Table 4-5 describes the available text fields in the order that they appear on the dis-
play pages, from left column to right column.
Figure 4-8. Switch Equipment Change Display
Table 4-5. Switch Equipment Change Actions
Action Description
ENABLE DEVICE ALARMING
Enables switch alarms to propagate upward in the I/A Series
System Management.
INHIBIT DEVICE ALARMING
Inhibits switch alarms from propagating upward in the
I/A Series System Management. All non-used Ethernet ports
must be inhibited for proper device monitoring.
84

Switch Port Equipment Information Display
An Equipment Information display for a port of an Ethernet switch is shown in Figure 4-9. The
display is similar for all switches. Table 4-6 describes the available text fields in the order that they
appear on the display pages, from left column to right column.
Figure 4-9. Switch Port Equipment Information Display - Typical
Table 4-6. Switch Port Equipment Information Display Fields
Field Description
NAME
The port number of the switch displayed in SMDH as P01 through
P378.
RUN MODE
On-line or Off-line (default) is displayed.
FAIL ACK STATE
Acknowledged (default) or Not Acknowledged is displayed. If the
DEVICE STATE value changes from Not Failed to Failed, the FAIL ACK
STATE value changes to Not Acknowledged to indicate this transition,
and remains until you acknowledge the port failure.
WARNING STATE
Yes is displayed if the device has a non-fatal error condition; otherwise,
No is displayed.
NUMBER
The port number 1 through 64.
CURRENT STATE
Port running or port failed.
TYPE
The switch port or the uplink of the switch.
DEVICE STATE
Failed is displayed if one or more devices connected to the switch has
failed; otherwise, Not Failed is displayed.
85

Indicators
Ethernet Switches
For the location and meaning of the indicators on a particular Ethernet switch, refer to the docu-
mentation provided with the switch.
Media Converter
For the location and meaning of the indicators on a particular media converter, refer to the docu-
mentation provided with the device.
Fiber Optic Cable Handling and Cleaning
Fiber optics communication relies on a clear path for its signals. Make every effort to install
splices, connectors, and terminations as clean as possible, thus reducing their effects on optical
data transmission.
Handling Fiber Optic Cable
Consult the cable specifications for the cable you are installing. Mishandling the cable may cause
damage that can alter its transmission characteristics requiring replacement of the cable. When
handling fiber optic cabling:
ALARMING STATE
Alarming State indicates whether alarming is enabled or inhibited for
the device. When alarming is inhibited, the System Monitor continues
to indicate overall system and network health (a green SYS BAR) while
equipment is Failed or Off-line.
COMM STATE
OK or Fail.
ADMIN STATE
Port Enabled or Port Disabled.
INTERFACE STATE
Interface state indicates the communications state of the port. Ethernet-
csmacd or software Loopback are normally displayed for The MESH
control network. Other states that can be displayed are: other,
regular1822, hdh1822, ddn-25, rfc877-x25, iso88023-csmacd,
iso88024-tokenbus, iso88025-tokenRing, iso88026-man, starLan, pro-
teon-10 Mb, proteon-80Mb, hyperchannel, fddi, lapb, sdlc, dsl, el,
basic ISDN, proppointToPointSerial, ppp, eon, ethernet-3Mb, nsip,
slip, ultra, ds3, sip, or fram-relay. These other states can be shown
depending on the type of switch and communications employed.
Table 4-6. Switch Port Equipment Information Display Fields (Continued)
Field Description
86

WARNING
!
Never look directly into the end of a fiber cable or bulkhead adapter. Eye damage
may result. Laser light can damage your eyes. Laser light is invisible. Viewing it
directly does not cause pain. The iris of the eye will not close involuntarily as when
viewing a bright light. Consequently, serious damage to the retina of the eye is pos-
sible. Never look into the end of a fiber or at a connector, or a connector in an
inspection microscope which may have a laser coupled to it. Should accidental eye
exposure to laser light be suspected, arrange for an eye examination immediately.
Turn off power to the equipment before cleaning or looking at fiber cable ends.
Always use dust covers on the end of the fiber cable connector (see Figure 4-10,
Figure 4-11 and Figure 4-12) or any bulkhead (including unused bulkheads)
Do not pull or kink the cable as the glass strand in the middle might become damaged
or broken
Do not walk-on, step-on, or crush the cable as the glass strand in the middle might
become damaged or broken
Keep bend radii no less than two inches
Use specialized optical cable raceways and plenums whenever available
Never use tie wraps as you would with electrical cables
When using optical connectors, insert or remove the ferrule straight into the sleeve.
Minimize wiggling the connection as this may loosen the tight fit
For SC connectors, orient the prominent key on the connector body with the slot in
the bulkhead adapter. Push the connector until it clicks. To remove, pinch the con-
nector body between your thumb and finger, and gently pull straight out (see
Figure 4-10)
Figure 4-10. SC Connector, Typical
For MT-RJ connectors, orient the connector body with the slot in the bulkhead
adapter. Push the connector until it clicks. To remove, push the release tab between
your thumb and finger, and gently pull straight out (see Figure 4-11)
Release Tabs
Dust Covers
87

Figure 4-11. Multimode MT-RJ Connector
For LC connectors, orient the connector body with the slot in the bulkhead adapter.
Push the connector until it clicks. To remove, push the release tab between your
thumb and finger, and gently pull straight out (see Figure 4-12).
Figure 4-12. Multimode Duplex LC Connector
Cleaning Fiber Optic Cable
Contamination of Fiber Optic Connectors and Sockets
Fiber optic cable connector parts are particularly sensitive to contamination which reduces the
light transfer from one side of the interface to the other. Finger oils, dust, fuzz, and so forth, can
attenuate the photon transfer across the cable connector to socket interface. Experience has shown
that minute quantities of contaminants can have major effects on the communication between
controllers and workstations.
The first line of defense is to prevent contamination from building up on the sensitive fiber optic
surfaces. The second approach is to remove the contamination once it has occurred. Prevention of
contamination is easier and cheaper than cleaning up after the fact.
Release Tab
Dust Cover
Release Tab
Dust Covers
88

Contamination Prevention
The easiest way to prevent contamination from affecting fiber optic sockets is to keep the protec-
tive covers (see Figure 4-10, Figure 4-11 and Figure 4-12) on all sockets which are not currently
in use. You should retain these dust covers (rubber plugs) in a place where they can be retrieved or
maintain a ready supply of covers so that they may be inserted into sockets when cables are
removed. All fiber optic switches, control modules (for example, FCP270 and ZCP270), and
interface modules (for example FCM100Et, FCM100E, Fiber NIC, ATS) come with protective
plugs installed in each socket. Figure 4-13 shows an example of microscopic lint which has lodged
in a fiber optic LC socket. The contaminant in this example totally prevented the port from func-
tioning. This type of contamination similarly affects MT-RJ and SC connectors, especially when
static charges on the connectors attract charged particles.
Figure 4-13. Lint in Fiber Optic LC Socket
Cleanliness is equally necessary with FO cable connectors plugs. Finger oils as well as dust and
lint can negatively affect the cable plugs transmission abilities. As with the sockets, the easiest
defense is to keep the dust covers, which come shipped on the connectors, in place until just
before insertion into a socket.
Contamination Removal
In spite of following the best contamination prevention procedures, you will eventually encounter
a dirty connection, in which case you must clean the connection. There are basically three
methods to remove contamination from FO connections; blowing, chemical washing, and abra-
sion.
The blowing method consists of removing dust or lint from a connector with a blast of dusting
gas such as Tetrafluoroethane, usually from an aerosol can. This is effective for removing material
in depressions (such as in a socket) which is loosely coupled to the surface, for example, as by
static electricity. Because dust can be transferred into connector sockets through the normal inser-
tion of cable plugs, it is good practice to dust off the connector before plugging it in.
Chemical washing basically consists of flushing or wiping the face of a fiber with isopropyl alco-
hol. The alcohol can be wiped on the fiber face by applying the fluid to a lint-free wipe or lint-free
swab, or by using pre-saturated wipes. These alcohol saturated wipes are most useful in cleaning
the connectors on the ends of cables where the exposed fibers are most easily accessible. The alco-
hol saturated swabs are needed for the female sockets in which the fiber is located in deep recesses.
89

The abrasion technique is useful for cleaning cable connectors such as MT-RJ style; it is not used
to clean sockets. It is a dry method which does not require isopropyl alcohol. It is generally avail-
able in the form of a fabric tape housed in a dispenser which exposes a short length of tape over
which you wipe the end of the connector. The fabric tape acts as a fine abrasive which removes
contamination from the glass fiber end.
Examples of cleaning products are listed below. It is suggested that the user of fiber optic cables
keep on hand similar items so you can quickly get back on-line when a contamination situation
arises. It is also highly recommended that you observe the storage and use suggestions provided by
the vendor of these cleaning products since some are classified as hazardous material. Obtain and
use the Material Safety Data Sheets (MSDS) and U.S. Department of Transportation DOT-E
10232 literature associated with these products as handling guides.
Examples of Fiber Optic Connector Cleaning Products
The following products are available through various distributors:
Techspray Fiber Optic Cleaning Kit, Part number 1602 (see: www.techspray.com)
Contains: Isopropyl alcohol, alcohol saturated wipes, aerosol duster, swabs, and wipes.
Techspray Aerosol Duster, Part number 1671-15S (see: www.techspray.com)
Contains an Aerosol duster (can of compressed Tetrafluoroethane gas).
Cletop S Cassette cleaner, P/N 14110611 (see: www.cletop.com)
An abrasive-type cable connector cleaner.
Cletop 1.25 mm Q-tip (swab) stick cleaner, P/N 14100401(see: www.cletop.com)
Corning Cable Systems, P/N 2104359-01(see: www.corning.com/cablesystems)
An abrasive-type cleaning cassette.
90

91

5. Combining Two or More MESH
Control Networks
This chapter provides information on combining two or more instances of The MESH control
network.
Overview
When combining two or more MESH networks, many variables must be taken into consider-
ation. Combining MESH networks actually refers to combining I/A Series systems. When
I/A Series systems are combined, a new committal must be generated which contains all the sta-
tions from the combined systems. All workstations must be recommitted and any station that
receives a new letterbug, NSAP, MAC address or IP address will have to be rebooted.
The following examples cover most of the basic situations and list the essential principles which
must be followed. Care must be taken when combining networks in order to prevent system or
network problems and even possible system shutdown.
Rules:
1. A complete inventory of all device IP, MAC and NSAP addresses on all networks must
be performed to ensure no duplicate addresses are present.
2. Establish the topology of both networks obtaining the following required informa-
tion.
a. Which switches are designated as the root bridge switches (Root/Back-up root) on
all networks
b. What topology configuration are the networks (Star, Tree, Ring, etc.), and are
they compatible with each other (with the consideration of the step c below)?
c. When the two networks become, as one, will any two switch communications
exceed the maximum limit of seven hops?
d. If VLANs are deployed on one or more networks do the VLAN assignments
match between the networks?
e. If Loop Detection Policy (LDP) is deployed on one or more networks, how will
they function between each other when considering the rule requirements for
LDP?
3. When combining two or more live networks, the process can be dangerous. It is
very important that all network and device components attached to the network be
100% functional and the functioning status of all switches must be known (root,
back-up root, uplink port status, configuration and location).
The following is the process by which two or more networks are combined.
Planning Stage
One must assume if two or more networks are being combined that there is more than one
I/A Series system in operation. To combine two or more I/A Series systems, the systems in whole
B0700AZ Rev L 5. Combining Two or More MESH Control Networks
92

must be re-committed. When re-committing the systems, the Bridge switch functionality must be
considered - see below.
Station Addressing
1. If duplicate IP, MAC and NSAP addresses on I/A Series devices or stations exist
within the networks, then those stations or devices will have to have new addresses
assigned.
2. Stations and devices with new addressing will have to be marked PCHANG and
rebooted after the whole system has been committed.
3. If duplicate IP addresses exist on a network switch, the switch must be re-configured
with a new IP address and the switch host workstation must be re-committed to
acknowledge the new IP address of the switch.
Bridge Switch Functionality
1. Establish which pair of switches will be the root and the backup root of the new com-
bined network.
a. The root and backup root should be the switch within the network that has the
largest throughput capability.
b. The root and back-up root should have the lowest set of IP addresses among the
network switches. (This is not mandatory but it is highly recommended.)
c. Any former root/backup switches will need the priority of their Bridge IDs
changed.
Topology Constants
When merging two or more networks, consider the physical layout of the topology. For example,
if a four-layer Tree topology is to be combined with a Star topology, the Tree cannot be added to
the Star. However, the Star can be added to the Tree but only at the root or layer one switches.
Adding the Tree to the Star results in a five-layer Tree, exceeding the limit of seven switches (hops)
between any two devices on the network. Adding the Star to the Tree's root or at the layer two
switches only results in a broader Tree which does not exceed the seven hop limit.
There are an unlimited number of ways in which network topologies can be combined. However,
the rules established in this document must be followed. Because of the unlimited ways of com-
bining networks, take each case with its own considerations. Although this document cannot
cover every possibility, the following are a few methods in which networks should be combined.
5. Combining Two or More MESH Control Networks B0700AZ Rev L
93

Combining Star Network Topologies
In Figure 5-1 below, A1 and B1 are root switches with the A2 and B2 as backup root switches
within their respective Star topology networks. To combine the two Star networks with minimum
impact to the physical layout of switch and cabling, the two Star networks are combined to form
a three layered Tree topology, shown in Figure 5-2 below.
Figure 5-1. Combining Two Star Network Topologies Into One Tree Network Topology
(Before)
Figure 5-2. Combining Two Star Network Topologies Into One Tree Network Topology (After)
To perform this, switch B2 must be taken offline, removed from the B network and configured as
an edge switch in the A network. (Refer to The MESH Control Network Operation, and Switch
Installation and Configuration Guide (B0700CA) for the definition of an edge switch.)
It is assumed that A1 and B1 are the root bridge switches within their respective networks and A2
and B2 are the back-up root bridges within their respective networks.
To do this, the following actions are required:
1. Verify that the A1 and A2 switches have a Bridge Priority value at least two (2) less
than the B1 switch. If they do not, reconfigure the A1 and A2 switches with a lower
priority value.
2. Power down the B2 switch and physically remove it from the B network, removing all
cables attached to this switch.
A1 A2
Star
B1 B2
Star
A1 A2
B1 B2
94

3. The B2 switch must be re-configured as an edge switch in the A network. (Refer to
The MESH Control Network Operation, and Switch Installation and Configuration
Guide (B0700CA) for configuration information.)
4. With the B2 switch powered down, physically connect the B2 switch to the A1 and
A2 switches via uplink (Trunk) cables one for each A root switch. The A root switch
uplink (Trunk) port must be configured to accept the new cables (if previously dis-
abled or if 100Mb ports are to be used).
5. Remove the previously cross-linked cables between the B1 and B2 switches from the
B1 switch. Reconnect the B network edge switches to the B2 switch.
6. Power up the B2 switch. When this is completed, the B1 switch will re-span, releasing
its root bridge function to the A1 switch. All devices on the B network have now been
migrated to the A network.
7. Power down the B1 switch and physically remove it from what is now the A network,
removing all cables attached to this switch.
8. The B1 switch must now be re-configured as an edge switch in the A network. (Refer
to The MESH Control Network Operation, and Switch Installation and Configuration
Guide (B0700CA) for configuration information.)
NOTE
If the two networks are to be separated for maintenance at some point, insert a
lower bridge priority value (i.e. 32000) for this B1 switch. At the time of separation,
the switch will re-establish its root functions. The same can be done for the B2
switch as well, but with a higher bridge priority value (i.e. 32001).
9. The B1 switch must be added to the A network via uplink (Trunk) cables - one for
each A root switch. The A root switch uplink (Trunk) ports must be configured to
accept the new cables (if previously disabled or if 100Mb ports are to be used).
10. Make the physical connections to the A root switches utilizing the old cross-linked
root connections to the B2 switch.
11. Reconnect the B1 switch to the former B network edge switches.
12. Power up the B1 switch.
You have just converted two Star network topologies into one Tree network topology.
If two Star network topologies are to be combined and it is desired that a Star topology be main-
tained (see Figure 5-7 on page 97), you must also apply the process of eliminating the seconded
pair of root switches, discussed in Combining Tree Network Topologies on page 96.
95

Combining Star with Tree Network Topology
The process for combining a Star network Topology with a Tree network Topology is the same as
for the previous procedure, Combining Star Network Topologies on page 93. See Figure 5-3
and Figure 5-4 below.
Figure 5-3. Combining a Star Network Topology with a Tree Network Topology Into One Tree
Network Topology (Before)
Figure 5-4. Combining a Star Network Topology with a Tree Network Topology Into One Tree
Network Topology (After)
A1 A2
Tree
B1 B2
Star
A1 A2
B1
B2
96

Combining Tree Network Topologies
The process for combining two Tree network topologies is the same as for the above two Star net-
work procedures. Combining two Tree network topologies is possible if one or more of the topol-
ogies has less than four tiers (layers). If two or more Tree network topologies has four tiers, then
this combination process is not possible. A tier (layer) must be removed from one of the topolo-
gies, as only one topology can have four tiers. This will cause an increase in effort and a higher risk
in having a network failure.
Figure 5-5. Combining Two Tree Network Topologies Into One Tree Network Topology
(Before)
Figure 5-6. Combining Two Tree Network Topologies Into One Tree Network Topology
(After)
For two four-tier topologies, the root and back-up root switches of the second network must be
removed (eliminated) and the second tier switches of this network must be connected to the root
and back-up root of the first topology (see Figure 5-7).
A2
Tree
B1 B2
Tree
A1
A1 A2
B1
B2
97

Figure 5-7. Combining Two (4) Tier Tree Network Topologies Into One (4) Tier Tree Network
Topology (After)
Combining Ring Network Topologies
Ring topologies are not recommend to be combined since the seven hop limit restricts the size of
the network topology to a maximum of seven switches in the network Ring topology. When com-
bining Ring topologies, it is recommended that the Ring topology migrate to the Star or Tree
topology that best suites the physical layout of the site requirements. Because of the potential
excessive amount of re-cabling, it is recommended that a plant shutdown be considered. Even
combining a Ring with a Star or Tree topology, it is recommended that at least the Ring topology
network side be shut down.
A Ring topology can migrate to a Star topology before combining the networks without requiring
a plant shutdown. To migrate from a Ring topology to a Star topology, first determine which
switches are the root and backup switches. If a root and backup are not configured yet, they
should be configured first. A Star topology can be created by manipulating the uplink Trunk
cables as shown in Figure 5-8 below. If there are no extra 1Gb uplink ports, the 100Mb uplink
ports of the B1/B2 switches can be used. Ideally, if two rings are to be combined, the final root
and backup root switches should be replaced with C2 Series or N-Series switches so that all
uplinks utilize 1Gb connections.
By using Figure 5-8 below and Figure 5-7 above (which combines two stars), two rings could be
converted into stars and then combined.
A1 A2
B1 B2
98

Figure 5-8. Combining Ring Network Topologies
Move cables one at a time
B1 B2
Root and Backup
B1 B2
B1 B2
Added cables
99

Appendix A. COMEX Fault
Handling on The MESH Control
Network
This chapter provides information on Communication Executive (COMEX) fault handling in
COMEX Layers
COMEX Applications Layer
This layer provides connectionless and exposed LLC communications services. Connectionless
messages are contained in unit data packets.
There are 2 types of connectionless messages - acknowledged and unacknowledged. For acknowl-
edged communications, there is a 2-way acknowledgement (handshake), where the receiver of the
unit data packet replies with an ack1 packet to the initial sender, which then responds with an
ack2 packet.
No replies (acks) are returned for unacknowledged message types. However, these message packets
are sent twice, first out the primary port, and then 1 second later, out the secondary port. The
Invensys multicasts are unacknowledged connectionless messages. The Invensys Broadcast is
actually one of the Invensys Multicasts.
Cable selection is based on an array in COMEX. The number of transmits is used as an index into
this table to select the cable. Seven (7) transmission attempts (Versatile Real-Time Executive
(VRTX) devices) are allowed, with the number of transmits starting at 1. Therefore, the first entry
in the array (index 0) is a filler and not used. The remaining entries indicate the cable selection as
follows: Primary, Primary, Secondary, Secondary, Primary, Primary, and Secondary. On the third
transmission attempt, a cable test is requested before attempting to send the message. When the
number of transmits exceeds the maximum (7 for VRTX devices), the message is flushed. For the
I/A Series MESH and 7.x Control Network stations (Windows XP
workstations and Nucleus Plus

devices), this maximum is also 7.
Application Layer Protocol and Timers
A transmitted acknowledged connectionless Unit Data packet requires an Ack1 response from the
receiver. Upon receiving the Ack1 response, the initiator responds with an Ack2 packet.
A transmitted unacknowledged connectionless Unit Data packet is directed to the Primary cable,
and requires no response. Rather, after waiting 1 second, the packet is re-directed to the Second-
ary cable. (Since the Data Link Layer of the I/A Series MESH and 7.x Control Network systems
determines the active communications port, the above cable directions are ignored.) A reference
delay is used to time the receipt of Ack1's and Ack2's and the duplicate transmission of unac-
knowledged packets. This time delays are as follows:
round trip time delay - 1.01 seconds
B0700AZ Rev L Appendix A. COMEX Fault Handling on The MESH Control
100

This is a basic delay period for stations on the same Node, for which some other
delays are based. For stations on another Node on the same LAN, 1 is added to this
value. For stations on another LAN on the same Site, 2 is added to this value. Note
that the Site in the comms hierarchy was never implemented.
acknowledged UD packet transmitted (retransmit time-out) - 1 second time-out for
receipt of Ack1 response
Failure to receive response within the time-out period results in the UD packet being
re-transmitted, up to the limit of retries. This value is subject to ongoing optimization in
The MESH network devices, and may change without notice.
Ack1 response sent - Time-out period is taken from the original acknowledged UD
packet (its retransmit time-out) and modified, for receiving the Ack2 response.
The time-out period is multiplied by the maximum number of connectionless retries
allowed and 1 added. Then, the round trip delay is added to the time - 1.01 seconds
for same node, 2.01 seconds for same LAN, and 3.01 seconds for same site. There-
fore, the reference delay is between 9.01 and 11.01 seconds, for VRTX stations.
(Because, the site hierarchy part of the communications system was never implemented, the
same LAN would be the furthest round trip delay - 10.01 seconds.) Failure to receive the
Ack2 response is the same as if the Ack2 was received - resources are released.
unacknowledged UD packet transmitted - 1 second time-out.
Timer expiration results in sending duplicate unacknowledged UD packet on other
cable. This is always the case, since no response is expected that would stop the timer.
unacknowledged UD packet received
The time-out period is taken from the original unacknowledged UD packet and mul-
tiplied by 2. Then, the round trip delay is added to the time - 1.01 seconds for same
node, 2.01 seconds for same LAN, and 3.01 seconds for same site. Therefore, the ref-
erence delay is between 3.01 and 5.01 seconds. (Because, the site hierarchy part of the
communications system was never implemented, the same LAN would be the furthest
round trip delay - 4.01 seconds.) When the duplicate packet is received, or with timer
expiration, receive resources are released.
Transport Layer
This layer provides connected communications services. The protocol of this layer is based on a
reduced subset of what appears to be class 4, as defined in RFC 905, ISO Transport Protocol Spec-
ification, ISO DP 8073, dated April 1984. The message is contained in a data transfer packet
(DT). Connections are set up by sending a Connection Request packet (CR). The response is a
Connection Confirm packet (CC) to accept the connection, or a Disconnect Request (DR) to refuse
the connection. When the initiator receives an acceptance (CC packet), an Ack (AK) packet is sent
to acknowledge the connection. DT packets are acknowledged with an AK packet. In addition, if
either side of the connection has not sent a packet in a certain time-out period, an AK packet is
then sent as a heartbeat to show that the connection is still active. Disconnects are initiated with
a DR packet. The disconnect is accepted by replying with a Disconnect Confirm (DC) packet.
DT packets are sequenced and use a window controlling the number of outstanding DT pack-
ets before an AK is received. The sequence number starts at 0 and is incremented for successive
DT packets. The receiver of the DT packet responds with an AK, indicating the sequence number
of the next expected packet. If a DT packet is received in error, then the AK will contain the
sequence number of that packet. A receive credit value is given in the CR, CC, and AK packets,
Appendix A. COMEX Fault Handling on The MESH Control Network B0700AZ Rev L
101

indicating the size of the window for the receiver. In effect, the transmitter can have no more
than the receive credit number of transmitted packets outstanding with the receiver.
When a connection is requested, (or any packet expecting an ack response) the initiator will wait
the Retransmit Timer period for the response, before retransmitting the packet. When a connec-
tion is established, DT packets or heartbeats (AK's) will be sent every Window Timer period. A
DT or AK packet is expected to be received within the Inactivity Timer period, in order to main-
tain the connection. When a DT packet is received, the receiver will wait no longer than the AK
Timer period before transmitting a response (AK).
Messages could be received in multiple data packets. This is called a multi-packet receive. The
packet indicates if it is the last packet of a transmission (EOT bit of the sequence number octet).
Multi-packets need to be linked into a single message before being delivered to IPC. In addition,
during multi-packet receives, the receive credit get boosted above normal and an AK set to change
the credits with the sender. With the last packet, the receive credit gets set back to normal, and an
AK is then sent to change the credit back to normal with the sender. (At the sender, it becomes his
transmit credit.)
When a connection exists, all messages to be transmitted are placed in the Wait Ack Queue
(WAKQ) to wait for their acknowledgment. Received packets (DTs or ACKs) that are out of
sequence with what is expected are handled according to the following table:
Transmissions expecting a response are attempted 8 times (7 retries), before the packet is dis-
carded. For the I/A Series MESH and 7.x Control Network stations, this limit is 8 attempts for both
Windows XP workstations and Nucleus Plus devices (7 retries). Transmissions not expecting a
response (AK and DC packets) are only sent once. The cable selection is governed by an array.
Table A-1. Out of Sequence DT and Ack Packets
Received Packet Action Retransmit Timer Ack Timer
ACK seq < outstand-
ing DT pkt's last seq.
Acknowledged packets
removed from WAKQ,
if present. Available new
packets sent up to xmit
credit limit.
Restarted if any packets
on WAKQ acknowl-
edged and new ones
sent. If WAKQ empty,
timer stopped.
-
ACK seq same as last
ACK seq received.
Ignored. No action. -
ACK seq > outstand-
ing pkts last seq + 1
Out of sequence
counter incremented.
ACK ignored.
No action. -
DT seq < next
expected - 1.
Dumped. - Start timer if not
already running.
DT seq = next
expected - 1
Dumped and ACK
sent.
- (This packet was the
last one acknowl-
edged and timer
should already be
stopped.)
DT seq > next
expected
Dumped. - No action.
102

The number of retries is used as an index into the array. Cables in this array are selected as follows:
primary, primary, secondary, secondary, primary, secondary, primary, secondary. Packets expecting
a response will use this array. Packets not expecting a response (AK and DC), use another array.
The result of using this array, is that the cable is swapped between the primary and secondary,
with every transmit. Because AK and DC packets use private resources that is never released (these
packets are created in the TL), the transmit count will keep incrementing, so that these packets
alternate cables.
The retry count is for an individual message (not a response) on a given connected channel. The
transmit count is only for responses being sent using the private resources. Individual messages are
retried. Responses (AK, DC) are not retried, because they do not get a response.
When the retry count is 2, a cable test is requested. When the retry count exceeds the maximum,
the message is flushed.
Transport Layer Protocol and Timers
The timers used by the Transport Layer are:
Retransmit Timer - 0.5 seconds for VRTX devices (for the I/A Series MESH and 7.x
Control Networks. this value is 0.5 seconds for Nucleus Plus devices and 0.5 seconds
for Windows XP workstations)
Maximum time allowed between sending a packet and receiving an AK for it. This
timer automatically restarts itself. Timer expiration causes a transmission reattempt of
all packets in the Wait Ack Queue, up to the retry limit. When outstanding packets
are AK'ed, this timer is stopped (and packets removed from the queue).
Inactivity Timer - 10 seconds
Maximum time allowed to receive heartbeat AK's (or DT's). This is a one-shot timer.
It is restarted whenever a packet is received. At timer expiration, the connection is
aborted.
Window Timer - 2 seconds for VRTX devices (for the I/A Series MESH and 7.x Con-
trol Networks, this value is 2 seconds for Nucleus Plus devices and 2 seconds for
Windows XP workstations) Maximum time allowed between sending heartbeat AK's
(or DT's)
This timer automatically restarts itself. It is also restarted whenever an AK or DT is
sent. At expiration, an AK packet is sent.
AK Timer - 75 msecs
Maximum time allowed between the first DT received and transmitting its AK. This
is a one-shot timer. At expiration, an AK packet is sent.
Normally, when the number of received DT packets equals the limit, an AK packet is
sent. In this case, the timer provides a cap on the time between received data and its
AK transmission. The limit is 1.
When a connection is requested with a CR packet, a CC packet response is generated by the
receiving end. The Retransmit Timer is used to time the response. If the timer expires before get-
ting the response, the CR packet is resent, up to the maximum retries, before the packet and the
connection are aborted. If the protocol is successful, a connected channel is created. Then the
Inactivity Timer is used by both ends of the channel for receiving packets from the other end of
the channel. These packets can be either DT packets or Acks (DT acknowledgements or heart-
beats). If the Inactivity Timer expires, the connection is aborted.
Appendix A. COMEX Fault Handling on The MESH Control Network B0700AZ Rev L
103

With a connection established, and a DT packet sent, an Ack response is expected. The Retrans-
mit Timer is used to time the response. If the timer expires before getting the response, the DT
packet is re-sent, up to the maximum retries, before the packet and the connection are aborted.
With a connection established, and a DR packet sent, a DC response is expected. The Retransmit
Timer is used to time the response. If the timer expires before getting the response, the DR packet
is re-sent, up to the maximum retries, before the packet and the connection are aborted.
An aborted connection causes a DR packet to be sent. The expected response and timers are the
same as previously described. The difference is that instead of IPC requesting the disconnect, the
request is initiated by the Transport Layer.
Network Layer
The Network Layer provides routing to the Nodebus, to another Nodebus (through the LAN
Interface), and back into the box (upper layers).
Operation
There are no states in the Network Layer operation, because no protocol is performed. The pri-
mary responsibility of this layer is routing the packets. Packets are routed to the local node, to the
LAN Interface in VRTX devices for other nodes, and back to the upper layers in the same station.
In The MESH network devices, off-node packets are routed to a specific Address Translation Sta-
tion (ATS), rather than to a LAN Interface.
Point-to-point packets are routed to the upper layers when the destination NSAP (site, LAN, and
node ID's) is the same as the source NSAP. The destination and source are on the same station
and the packet is routed to the upper layers as if it was a received packet.
Multicast packets are always sent out the station. They are also routed to the upper layers of the
source station, if the Reroute bit (in the packet) is set and the multicast group is active in the sta-
tion.
Logical Link Control Layer
The implementation of this layer is based on IEEE 802.2, class 1 (Type 1 Operation only), using
only unnumbered UI commands (information). No other U commands are supported. There-
fore, the poll/final bit is not used (it is not valid for UI commands).
Transmit Operation
There are no states used in an LLC transmit operation. The LLC header is simply added to the
packet and it is passed on to the MAC layer.
Receive Operation
The LLC distributes received packets according to whether they are normal data packets, diagnos-
tic packets, or exposed LLC packets. Several additional packet types are defined for The MESH
network devices.
The routing of connected and connectionless packets is based on the Network Layer destination
LSAP (Link Service Access Point). These packets specify network routing (Network Layer LSAP),
providing for routing by the destination LLC Layer to the destination Network Layer.
For exposed LLC packets, 2 LSAPs are supported. The Application Layer 802.1 individual LSAP is
used for downloading station images, and the Application Layer 802.1 group LSAP is used for
104

cable test messages (diagnostics). These packets are distributed directly to the Application Layer.
The diagnostic packet used for a cable test is sent by NFD with a Station Manager destination
LSAP, via exposed llc (through IPC and the Application Layer). Received Station Manager packets
are passed by the LLC Layer directly to SMD (station manager diagnostics), which echoes them
back directly through the LLC Layer, as an Application Layer 802.1 group destination LSAP. These
are then passed by the receiver LLC Layer directly to the Application Layer (and thus, back to
NFD).
The MESH network devices provide for additional destination LSAPs:
NFD PROXY for The MESH network to Nodebus NFD cable test proxy packet.
MAC QUERY for a Mac Query packet.
(Other LSAPs are used by The MESH network comms drivers and ATS devices for validating net-
work integrity.)
MAC Layer
The MAC layer is designed for the individual hardware for which it interfaces.
105

Appendix B. The MESH Network
Fault Handling
This chapter provides information on fault handling in The MESH control network.
Fault handling on The MESH network is handled on a per station basis. This error handling pro-
cess is protected by a patent. This description applies to all dual port COMEX stations which
have two roles for their ports, including Workstations, FCP, ZCP, and ATS Stations:
an active port based on the system address for that station.
a standby port that only is used for testing with a different MAC address until it is
determined that the active port is not functioning correctly.
COMEX implements seven retries on the active port spaced over 3.5 seconds for all connected
messages. Connectionless acknowledged messages perform seven retries over seven seconds.
COMEX relies on lower level mechanisms to select the best possible port as the active port. The
port selecting behavior of the stations is as follows.
LINK
The first line of defense is the Link Integrity Signal. This signal is derived by the PHY Interface
hardware and is indicative of the presence of IDLE symbols received. If a port does not have
LINK, it cannot send or receive. If the Link signal goes inactive, the port is not usable and the
driver will switch the station address on the module to the other port if it does have a LINK sig-
nal. Typically switches stop sending if they detect no activity on a port (to conserve power) so link
becomes a bi-directional indication. Loss of LINK switchover will be indicated to the customer as
PORT A (or B) failure in the smon_log and SMDH.
Causes of loss of link:
1. Switch power down
2. Switch failure
3. Cable failure of either the transmit or receive fiber
4. Dirty cable
5. Module hardware failure
PORT TEST Packets
If both LINK signals are active two PORT TEST packet are sent by the driver every 300 milli-
seconds, one PORT TEST packet is sent from the A port to the B port and one PORT
TEST packet from the B to the A. In the body of the packet is the text Prim Port Test and
Alt port Test. Reception of these packets in both directions indicate that both ports have equal
connectivity to the rest of the network. As long as LINK is good and the PORT TEST pack-
ets succeed, no port switching will occur.
B0700AZ Rev L Appendix B. The MESH Network Fault Handling
106

A single port test packet failure is flagged and the Excessive Collision Counter is incremented but
no switchover occurs until two PORT TEST in a row have experienced a failure. A PORT
TEST failure is defined as a packet in either direction failing.
If two successive PORT TEST tests fail due to the inability to receive both packets the module
will initiate the PING test. The failure of this test indicates that there is a fault in The MESH
which prevents normal communications. The port test packets will cease for 400 milliseconds
while the PING test attempts to detect the best way to survive the problem. After the
llc_ping test runs the module goes back to sending port tests until the network is healthy.
Causes of PORT TEST failure:
1. Switch failure causing islanded switches
2. Message storm substantially interfering with normal communications
3. Internal switch failure of the switch connected to the A port or the B port
4. Module hardware failure
5. Broken or damaged cable
LLC_PING Test
This test involves sending a PING packet to up to seven stations that have previously been in
communication with this controller. It is used to allow the module to select the port with the
highest connectivity with its most recent COMEX partners based on a most recently received
table. This IS NOT the standard IP PING packet instead it is a COMEX llc_ping type
(DSAP = 0x0A, SSAP = 0x0A). Only COMEX stations are sent this message to guarantee that the
targeted station is running the SW that will reply to this PING. The packet indicates that this is
a llc_ping request and is sent on both the A port and the B port. When any COMEX sta-
tion receives this request it responds with a LLC_PING_response packet with the same format
(DSAP = 0x0A, SSAP = 0x0A) to the senders MAC address. The station who initiated the
llc_ping message then tallies the responses and make a decision whether to switch ports or not.
1. IF all responses are received on both cables this is a Good case and no switchover is
performed and no messages are sent.
2. IF more responses (for example 4 responses on alternate and 2 on Primary) are
received on the Alternate port a port switch is performed (no Hot-Remarry should
occur on FT stations). This port switch places the Alternate as the Primary port and
re-assigns the MAC addresses of the ports. A broadcast is sent to update all switches
with the new MAC address location. A message is sent to the alarm printer and the
smon_log if enabled.
PING Test Sent = 7 Replies = Pri 2 Alt 4 new cable = B (or A)
NOTE
A workstation does not currently send any messages to the alarm printer/smon_log.
Workstations run in promiscuous mode. Upon a Primary port swap, the NIC is not
reprogrammed but the software does use the Primary port Mac address on the Alter-
nate port when a swap occurs.
3. IF AN EQUAL number of responses is received and the number is less than the total
sent a one-time switchover is performed. This switchover is to resolve situations with
Appendix B. The MESH Network Fault Handling B0700AZ Rev L
107

an equal number of local and remote stations causing islanding. It is done to cascade
all stations over to a healthy path. No more switchovers on tie situations are allowed
until successful A to B port tests are detected indicating a return to healthy
MESH. A message is sent to the alarm printer
PING Test Sent = 7 Replies = Pri 3 Alt 3 new cable = B (or A)
4. IF more responses are received on the Primary port, no switching is performed and a
PORT FAILURE message for the Alternate port is sent to the alarm printer.
ZCP-FCM Communications
All communications between the ZCP and the FCM use raw IP packets. The ZCP sends an
alternating request/reply transaction on each Ethernet cable to talk to the FCM pair at all times.
On discovery of any failure, the field connection has its reliability score reduced. The ZCP uses
the more reliable cable exclusively while background mechanisms try to rehabilitate the failed
cable.
The ZCP maintains a quality of connection for each FCM separately. If one FCM has a problem
on channel A, that FCM will prefer to use channel B. All others would still use A or B normally.
There are no retries. All Ethernet, IP, and port addresses are unique to the particular network.
NOTE
If the user has required the ZCP to use the A Ethernet cable and the A cable is in
fact failed, the ZCP will blindly follow the requirement to use the A cable even
though it does not work. This is done to aid debugging of problems.
NOTE
Two FCMs can have the same addresses if they are on separate networks; for exam-
ple, behind different ZCPs and not on The MESH.
There is no LLC3 protocol on the I/O side between the ZCP and the FCM.
ZCP and FCM communication behaves as discussed below in the following cases:
Case 1: The ZCP fails to receive a response on the primary port to an FCM and/or
switch.
Behavior: The ZCP retains a health score on each of its connections. It maintains com-
munications on the A or B bus, whichever is more healthy. However, the FCM is also
trying to recover the bad cable, and is sending health packets on the bad cable. A success-
fully delivered packet raises the ZCPs health score, so the bad cables health score may
increase. Multiple successes raise the cables health score to completely good. When the
score is equal that of the other cable, the ZCP will start to alternate between cables again
for its communication to the FCM.
Case 2: The ZCP communicates with several FCMs and does not receive a response on a
port.
Behavior: The ZCP maintains a quality of connection for each FCM separately. If one
FCM has a problem on the A bus, that FCM will switch to the B bus. All other FCMs
108

would still use A or B normally. A message indicating this change should be generated
by the ZCP on the FCM's behalf.
DIAGNOSTIC Information
Diagnosing MESH network failures is complicated but here is some general information
1. PORT FAILURE MESSAGE
a. Most likely caused by loss of the LINK signal.
b. Most likely caused by a cable problem or dirty (very common) Fiber connections.
c. Loss of link may be reported by both the switch and the station PORT FAIL-
URE message and the SMDH port field.
d. Port Failure is also indicated by a persistent path failure from the Primary port to
the Alternate port. The Alternate port will display the port failure message.
2. FT LINK Problems
a. On FT station both stations must have a common link on at least one cable to
stay married.
b. If the Primary loses both links it will reboot and the Shadow will become Primary.
c. Modules will not marry without at least one common link. If no common link is
found a module will go RED/GREEN.
3. ZCP LINK problems
a. For ZCPs link failure on the Control network are handled in the same fashion as
failure on the Application network.
b. For ZCP I/O functionality as of 8.2 there must be a path from the A port of the
ZCP to the A port of the FCM or the B port to B port in order for the ZCP
to maintain comms. The ZCP cannot talk from the A port to the FCM B port
MAC address. This means that cross-wired systems have multiple single points of
failures as follows:
ZCP port A to Switch 1 port 1
ZCP port B to Switch 2 port 1
FCM port A to Switch 2 port 2
FCM port B to Switch 1 port 2
Switch 1 port 3 to Switch 2 port 3
If this configuration was built the ZCP to FCM communications would work fine
with all hardware functional. Any of the following failures would cause loss of I/O
communications:
Switch 1 -power down or total failure
Switch 2 -power down or total failure
Switch 1 port 3 or switch 2 port 3
Cable Switch 1 port 3 to Switch 2 port 3
4. SMDH counters
a. MAC resets indicate either Hot-Remarries or Port problems.
Appendix B. The MESH Network Fault Handling B0700AZ Rev L
109

b. Excessive Collisions indicate either continuous (evenly incrementing approxi-
mately 1 per second) or intermittent MESH problems (incrementing at less than
1 per second) causing A to B failures.
c. At the present time, workstations do not indicate A to B port test failures
unless a port switch occurs.
110

111

Index
C
Cable
fiber optic 60, 61
specifications 59
twisted-pair 66
Cabling
fiber optic 63
Cleaning
fiber optic cable 87
D
Display, Switch Configuration Information 80
Display, Switch Domain 81
Display, Switch Port Equipment Change 83
Display, Switch Port Equipment Information 84
Display, Switch Ports 82
Display, Switched Equipment Information 78
Display, Switched Network 76, 77
E
Ethernet switches
connecting 67
indicators 85, 87
F
Fiber optic cable 18, 60, 61
cleaning 87
handling 85
Fiber optic cabling 63
G
Gigabit Ethernet uplink ports 68
H
Handling
Fiber optic cable 85
I
I/O network 12
design rules 36
B0700AZ Rev L Index
112

illustration 14
topology configuration 58
Indicators
Ethernet switches 85, 87
media converter 85
IOC (I/O Controller) 12
M
Multimode fiber. See Fiber optic cable, multimode
N
Network components connections 67
Network topology 36
Null hub 18, 66
P
Prefabricated cables 66
S
SCAS - Switch Configurator Application Software 17
Single mode fiber. See Fiber optic cable, single mode
Site planning 23
SMDH 74
Switch Configuration Information display 80
Switch Domain display 81
Switch Equipment Information display 78
Switch Port Equipment Change display 83
Switch Port Equipment Information display 84
Switch Ports display 82
Switched Network display 76, 77
System Management displays 74
T
Topology, network 36
Troubleshooting 72
Twisted-pair cable 66
U
Uplink port 68
V
VLANs 3, 23, 30
Index B0700AZ Rev L
113

B0700AZ Rev L Index

Printed in U.S.A. 1009
IPS Corporate Headquarters
5601 Granite Parkway Suite 1000
Plano, TX 75024
United States of America
www.ips.invensys.com
Foxboro Global Client Support
Inside U.S.: 1-866-746-6477
Outside U.S.: 1-508-549-2424 or
contact your local Foxboro representative.
Facsimile: 1-508-549-4999

Mesh Control

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Mesh Control

Uploaded by

Copyright:

Available Formats

B0700AZ

workstations and Nucleus Plus

You might also like