PowerHA SystemMirror 7.

1 and Multicasting Setup Dated: 1st Nov 2011

Table of Contents

1 Introduction ................................................................................................................. 1
1.1 Terminology........................................................................................................ 1
1.2 Reference ............................................................................................................ 2
2 PowerHA SystemMirror 7.1 and Multicasting Setup ................................................. 2
1.3 Multicasting Concepts ........................................................................................ 2
1.3.1 Multicast packet communication .................................................................2
1.3.2 Network Switches ........................................................................................2
1.3.3 Internet Group Management Protocol (IGMP) ............................................3
1.3.4 IGMP Snooping ...........................................................................................4
1.3.5 Multicast routing ..........................................................................................4
1.4 Multicast guidelines for PowerHA SystemMirror 7.1 ........................................ 4
1.4.1 Test Multicasting in Network ......................................................................4
1.4.2 Troubleshooting guidelines..........................................................................5

1 Introduction
PowerHA SystemMirror 7.1 Standard Edition High Availability solution implements
clustering using multicast (IP based multicast) based communication between the
nodes/hosts in the cluster. Multicast based communication provides for optimized
communication method to exchange not only heartbeats, but also allows clustering
software to communicate critical events, cluster coordination messages etc in 1 to N
method instead of communication 1 to 1 between the hosts.

Multicast communication is a well established mode of communication in the world of

TCP/IP network communication. However in some cases, the network switches used in
the communication path need to be reviewed and enabled for multicast traffic to flow
between the cluster hosts through them. This document explains some of the network
setup aspects that may need to be reviewed before the PowerHA SystemMirror 7.1
cluster is deployed.

1.1 Terminology
Multicast Address range
Multicast routing (mrouted)
L2 Multicast Address
224.0.0.0 through 239.255.255.255
Multicast router protocol. This router handles the multicast packets and
routes as necessary. Some switches are capable of routing multicast
packets between switches in the cascaded environments (See reference
#2)
IPv4 Multicast packets map to a predefined IEEE MAC Address that
has the format: 01.00.5e.xx.xx.xx

Page 1 of 5
PowerHA SystemMirror 7.1 and Multicasting Setup Dated: 1st Nov 2011

1.2 Reference
1. IP multicasting Information:
http://www.ietf.org/rfc/rfc2365.txt?number=2365
2. Multicast address range & Assignments:
http://www.iana.org/assignments/multicast-addresses/multicast-
addresses.txt
3. Multicast Router Information: http://www.rfc-editor.org/rfc/rfc1075.txt
4. Multicast: http://www.rfc-editor.org/rfc/rfc1054.txt
5. Switch Guidelines(Vendor)
a. Information on multicasting setup:
http://www.cisco.com/en/US/products/hw/switches/ps708/products_tec
h_note09186a008059a9df.shtml
b. Switch Support information
http://www.cisco.com/en/US/tech/tk828/technologies_tech_note09186a
0080122a70.shtml#NetPro
c.

2 PowerHA SystemMirror 7.1 and Multicasting

PowerHA SystemMirror 7.1 uses a new re-architected cluster health management layer
embedded as part of the Operating System called Cluster Aware AIX (CAA). CAA uses
kernel level code to exchange heartbeats over network, SAN fabric (when right Fiber
Channel adapters are deployed) and as well as a disk based messaging through the
Central repository.

Cluster Aware AIX exploits multicast IP based network communication mechanism to

communicate between the various nodes in the cluster within a site. Administrator can
manually configure a multicast address to be used for the cluster communication or allow
PowerHA SystemMirror/CAA to choose a multicast address.

Note that multicast communication is used during the initial discovery phase when the
cluster is being created, but also during the normal operations of the cluster. Hence it is
extremely important that the multicast traffic to flow between the cluster hosts in the
datacenter before the cluster formation can be attempted. Please plan to test and verify
the multicast traffic flow between the would-be cluster nodes before even attempting to
create the cluster. Review the guidelines in the following sections to test the multicast
packet flow between the hosts.

1.3 Multicasting Concepts

1.3.1 Multicast packet communication

Multicasting is a form of addressing, where a group of hosts form a group and exchange
messages. A multicast message sent by one in the group is received by all in the group.
This allows for efficient cluster communication where many times messages need to be
sent to all the nodes in the cluster. For example a cluster member may need to notify the
rest of the nodes about a critical event and can accomplish the same by sending a single
multicast packet with the relevant information.

1.3.2 Network Switches

Page 2 of 5
PowerHA SystemMirror 7.1 and Multicasting Setup Dated: 1st Nov 2011

Hosts communicate over the network fabric that could consist of many switches and
routers.
Switch connects different hosts and network segments and allows for network traffic to
get sent to the right place. Switch refers to a multi-port network bridge that processes and
routes data at the data link layer (layer 2) of the OSI model. Some Switches can also
process data at the network layer (Layer 3).

Typically a datacenter networking environment consists of hosts connected through

network fabric that consists of Ethernet or such cabling and switches. Many times
switches are interconnected to form the fabric between the hosts. When switches cascade,
multi cast packet would have to flow from Host in the cluster to a switch and then
through the other switches to finally reach the destination Host in the cluster. Since
switches review multicast packets differently as compared regular network
communication, switch to switch communication might not happen for multicast packets
if the setup is incorrect.

Cluster

Host A Host B

Switch 1 Switch 2

Figure 1: Hosts connected through cascaded switches

Fig 1 shows one such configuration where Host A that would be part of the cluster is
connected to Host B through two switches (Switch 1 and Switch 2). Network link
between the two switches shown in Red in the picture should allow for the multicast
traffic to flow for the cluster to be formed and operational.

1.3.3 Internet Group Management Protocol (IGMP)

IGMP is a communications protocol that enables Hosts (receivers) to inform a multicast

router (IGMP querier) of the Hosts intention to receive particular multicast traffic. So
this is a protocol that runs between a router and Hosts and allows:

Routers to ask Hosts if they need a particular multicast stream (IGMP query)

Page 3 of 5
PowerHA SystemMirror 7.1 and Multicasting Setup Dated: 1st Nov 2011

Hosts to tell or respond to the router if they seek a particular multicast stream
(IGMP reports)

So IGMP communication protocol is used by the Hosts and the adjacent routers on IP
networks to interact and establish ground rules for multicast communication in particular
establish multicast group membership. Switches featuring IGMP snooping derive useful
information by observing these IGMP transactions between the Hosts and Routers. This
enables the switches to correctly forward the multicast packets when needed to the next
switch in the network path.

1.3.4 IGMP Snooping

IGMP snooping is an activity performed by the switches to track the IGMP

communications related packet exchanges and adapt the same in regards to filtering the
multicast packets. Switches monitor the IGMP traffic and allow out the multicast packets
only when necessary. Switch typically builds an IGMP snooping table that has a list of all
the ports that have requested a particular multicast group and uses this table to allow or
disallow the multicast packets to flow.
Switches allow many cases to be configured around this IGMP snooping

1.3.5 Multicast routing

The network entities that forward multicast packets using special routing algorithms are
referred to as mrouters. Also, router vendors may implement multicast routing, so the
customer should also refer to the router vendor's documentation and guidance. Hosts and
other network elements implement mrouters and allow for the multicast network traffic to
flow appropriately. Note that some traditional routers would also support multicasting
packet routing.
When switches are cascaded(chained), it may be necessary to setup the switch to forward
the packets as necessary to implement the mrouting. However this could be one of the
possible approaches to solving multicast traffic flow issues in the environment. Refer to
the Switch Vendors documentation and guidance in regards to setting up the switches for
multicast traffic.

1.4 Multicast guidelines for PowerHA SystemMirror 7.1

In this section set of guidelines will be provided in regards to multicast setup in your
environment. However note that these guidelines are generic in nature and the
configuration of the switches would depend on your network environment and switch
type and capabilities.

1.4.1 Test Multicasting in Network

Do not attempt to create the cluster until multicast traffic flows without any interruptions
between the nodes that will be part of the cluster. From PowerHA SystemMirror 7.1.1
release onwards, clustering will not continue if mping test fails. If there are problems

Page 4 of 5
PowerHA SystemMirror 7.1 and Multicasting Setup Dated: 1st Nov 2011

with the multicast communication in your network environment, contact the network
administrator and review the switches involved and the setup needed. Once the setup is
complete, retest the multicast communication.

One of the simplest method to test end to end multicast communication is to use the
mping command available on AIX. In Fig 1, start the mping command in receive mode
on one Host (Say Host A) and then use mping command to send packets from the other
Host (Host B). If multiple hosts will be part of the cluster, test end to end mping
communication from each host to the other.

Command mping can be invoked with a particular multicast address or it will choose a
default multicast address. Following illustrates an invocation of mping command for a
multicast address 228.168.101.43:

Host A: mping Packet Receiver

Host A> mping -v -r -c 5 -a 228.168.101.43

Host B: mping Packet Sender

Host B> mping -v -s -c 5 -a 228.168.101.43

Use the actual multicast address that will be used during clustering (later) as the address
input to mping. Note that CAA creates a default multicast address if one is not specified
during cluster creation. This default multicast address is formed by combining (ORing)
228.0.0.0 with lower 24 bits of the IP address of the Host. Example: If the Hosts IP
address is 9.3.199.45, then the default multicast address will be chosen as 228.3.199.45

1.4.2 Troubleshooting guidelines

If mping command fails to receive packets from Host to Host in the network
environment, there could be some issue in the network path in regards to multicast packet
flow. Follow some of the general guidelines below to troubleshoot the issue:
1. Review the switch vendors documentation for guidelines in regards to switch
setup. Some of the known switch guideline links are included in the reference.
2. Disable IGMP snooping on the switches. Most switches will allow for disabling
IGMP snooping. If your network environment allows, disable the IGMP snooping
and allow all multicast traffic to flow without any problems across switches.
3. If your network requirements does not allow snooping to be disabled: Debug the
problem by disabling the IGMP snooping and then adding network components
one at a time for snooping
4. Debug if necessary by eliminating the cascaded switch configurations (having
only one switch between the Hosts).

Page 5 of 5