CH A P T E R 11

Configuring PowerPath Multipathing for Linux

This chapter contains the following sections:

• Configuring Boot over Multipathing, page 11-1
• Preparing the System and Performing PowerPath Installation, page 11-1
• Setting Up Files and Directories, page 11-2
• Copying Required Objects and Configuration Files, page 11-2
• Modifying Files, page 11-4
• Archiving the bootdisk Directory, page 11-6

Configuring Boot over Multipathing

The following steps configure Boot over IB using PowerPath, using Linux RHEL3 Update7 on a X86_64
(AMD) system as an example.

Preparing the System and Performing PowerPath Installation

To prepare the system and install PowerPath, follow these steps:

Step 1 Set up the system for boot over SRP. When the system can boot over SRP, proceed with the steps below
to get the system boot over PowerPath.
Step 2 Install PowerPath on the system. Follow the usual EMC PowerPath installation procedure to install
PowerPath. Refer to the PowerPath installation document for instructions.
pm -ivh EMCpower.LINUX-4.5.1-022.rhel.x86_64.rpm
/etc/init.d/PowerPath start
emcpreg -install (register powerpath)

Step 3 After PowerPath is installed properly and started, enter the fdisk -l command. You should see output
similar to the example below. In this case, only one LUN is assigned to this host, and the LUN is
partitioned accordingly.
[root@sw-n4300-1 ~]# fdisk -l
Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System

Cisco VFrame Third Party Integration Guide

OL-12603-01 11-1
 Chapter 11 Configuring PowerPath Multipathing for Linux
Setting Up Files and Directories

/dev/sda1 * 1 38 305203+ 83 Linux

/dev/sda2 39 293 2048287+ 82 Linux swap
/dev/sda3 294 2610 18611302+ 83 Linux
Disk /dev/emcpowera: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/emcpowera1 * 1 38 305203+ 83 Linux
/dev/emcpowera2 39 293 2048287+ 82 Linux swap
/dev/emcpowera3 294 2610 18611302+ 83 Linux

PowerPath is now up and running. The emcpowera* shows the PowerPath pseudo devices.

Setting Up Files and Directories

To set up files and directories, follow these steps:

Step 1 Create a new directory called bootdisk.

Step 2 Copy the initrd file from the /boot to the new bootdisk directory.
-cp -p /boot/initrd-2.6.9-34.ELsmp.img bootdisk/

Step 3 Unzip the initrd-2.6.9-34.ELsmp.img file in the same bootdisk directory.

zcat initrd-2.6.9-34.ELsmp.img > initrd-2.6.9-34.ELsmp

Step 4 Copy the initrd directory structure and all the associated files into the same bootdisk directory, as follows
-cpio -i -I initrd-2.6.9-34.ELsmp

Step 5 Verify that the bootdisk directory now contains the following files and directories, in addition to
initrd-2.6.9-34.ELsmp.img and initrd-2.6.9-34.ELsmp files.
bin dev etc init lib loopfs proc sbin sys sysroot

Step 6 Delete the initrd-2.6.9-34.ELsmp.img and initrd-2.6.9-34.ELsmp files from the bootdisk directory.
Step 7 Create two more directories in the bootdisk directory and call them lib64 and usr.
Step 8 Verify that the following files and directories are in the bootdisk directory:
bin dev etc init lib lib64 loopfs proc sbin sys sysroot usr

Copying Required Objects and Configuration Files

To copy the requireed PowerPath objects and configuration files, follow these steps:

Step 1 Copy the PowerPath objects and configuration files into the new bootdisk directory. Some Linux objects
also need to be copied.

Cisco VFrame Third Party Integration Guide

11-2 OL-12603-01
Chapter 11 Configuring PowerPath Multipathing for Linux
Copying Required Objects and Configuration Files

The entire list is displayed in the table below.

• Note that it is not required to copy the IB module since it should already be in the /lib directory in
the original initrd file.
• The /lib64 directory is required since it is a x86_64 system. You will copy all the required library
files to either the /lib or /lib64 directory.
• Use the “ldd” command to obtain the required library, as shown in this example:
[root@host-1 bin]# ldd /bin/mkdir
libselinux.so.1 => /lib64/libselinux.so.1 (0x0000003977b00000)
libc.so.6 => /lib64/tls/libc.so.6 (0x0000003976800000)
/lib64/ld-linux-x86-64.so.2 (0x0000003976600000)
[root@host-1 bin]# ldd /sbin/powermt
linux-gate.so.1 => (0xffffe000)
libc.so.6 => /lib/tls/libc.so.6 (0x00adf000)
/lib/ld-linux.so.2 (0x00ac6000)

Table 11-1 Copy the PowerPath Objects and Configuration Files to the Directory

Notes Copy To Bootdisk Directory

/bin/ash ----> /sbin/ash
/bin/gawk ----> /sbin/gawk
/bin/mkdir ----> /sbin/mkdir
/sbin/powermt ----> /sbin/powermt
/sbin/rmmod ----> /sbin/rmmod
/bin/mount ----> /sbin/mount
/bin/umount ----> /sbin/umount
/bin/cat ----> /sbin/cat
/bin/mknod ----> /sbin/mknod
/sbin/pivot_root ----> /sbin/pivot_root
/bin/sleep ----> /sbin/sleep
Copy the whole directory /etc/emc ----> /etc/emc
/etc/emcp_devicesDB.dat ----> /etc/emcp_devicesDB.dat
/etc/emcp_devicesDB.idx ----> /etc/emcp_devicesDB.idx
/etc/emcp_registration ----> /etc/emcp_registration
This directory contains /etc/opt/emcpower ----> /etc/opt/emcpower
/etc/opt/emcpower/powercf and
/etc/opt/emcpower/emcpmgr object.
These objects are used in the init file.

/etc/fstab ----> /etc/fstab

Copy the whole PowerPath module. /lib/modules/2.6.9-34.ELsmp/extr ----> /etc/lib
a/emc*.ko
/usr/lib/libemcp_core.so ----> /lib/libemcp_core.so
/usr/lib/libemcp_lam.so ----> /lib/libemcp_lam.so
/usr/lib/libemcp_lic_rtl.so ----> /lib/libemcp_lic_rtl.so
/usr/lib/libemcp_mp_rtl.so ----> /lib/libemcp_mp_rtl.so

Cisco VFrame Third Party Integration Guide

OL-12603-01 11-3
 Chapter 11 Configuring PowerPath Multipathing for Linux
Modifying Files

Table 11-1 Copy the PowerPath Objects and Configuration Files to the Directory

Notes Copy To Bootdisk Directory

/usr/lib/libemcpmp.so ----> /lib/libemcpmp.so
/usr/lib/libemcp.so ----> /lib/libemcp.so
/lib/ld-linux.so.2 ----> /lib/ld-linux.so.2
/lib/tls/libm.so.6 ----> /lib/libm.so.6
/lib/tls/libc.so.6 ----> /lib/libc.so.6
/lib64/libdl.so.2 ----> /lib64/libdl.so.2
/lib64/tls/libm.so.6 ----> /lib64/tls/libm.so.6
/lib64/tls/libc.so.6 ----> /lib64/tls/libc.so.6
/lib64/ld-linux-x86-64.so.2 ----> /lib64/ld-linux-x86-64.so.2
/lib64/libselinux.so.1 ----> /lib64/libselinux.so.1
/lib64/tls/librt.so.1 ----> /lib64/tls/librt.so.1
/lib64/tls/libpthread.so.0 ----> /lib64/tls/libpthread.so.0
/lib/libpn.so ----> /lib/libpn.so
/usr/sbin/chroot ----> /usr/sbin/chroot

Modifying Files
To modify the /etc/fstab file, which you previously copied, follow these steps:

Step 1 Modify the /etc/fstab file to reflect the PowerPath devices:

In this example, /dev/emcpowera3 is the root partition and /dev/emcpowera1 is the boot partition.
# This file is edited by fstab-sync - see ‘man fstab-sync’ for details
/dev/emcpowera3 / ext3 defaults 1 1
/dev/emcpowera1 /boot ext3 defaults 1 2
none /dev/pts devpts gid=5,mode=620 0 0
none /dev/shm tmpfs defaults 0 0
none /proc proc defaults 0 0
none /sys sysfs defaults 0 0
/dev/emcpowera2 swap swap defaults 0 0
/dev/hdc /media/cdrom auto
pamconsole,exec,noauto,managed 0 0
/dev/fd0 /media/floppy auto
pamconsole,exec,noauto,managed 0 0

Step 2 Modify the default init file to the file below:

#!/bin/ash

mount -t proc /proc /proc

echo Mounted /proc filesystem

echo Mounting sysfs
mount -t sysfs none /sys

Cisco VFrame Third Party Integration Guide

11-4 OL-12603-01
Chapter 11 Configuring PowerPath Multipathing for Linux
Modifying Files

echo Creating /dev

mount -o mode=0755 -t tmpfs none /dev
mknod /dev/console c 5 1
mknod /dev/null c 1 3
mknod /dev/zero c 1 5
mkdir /dev/pts
mkdir /dev/shm
echo Starting udev
udevstart
echo -n “/sbin/hotplug” > /proc/sys/kernel/hotplug
echo “Loading mlxsys.ko module”
insmod /lib/mlxsys.ko
echo “Loading mod_vip.ko module”
insmod /lib/mod_vip.ko
echo “Loading mod_thh.ko module”
insmod /lib/mod_thh.ko
echo “Loading mod_rhh.ko module”
insmod /lib/mod_rhh.ko
echo “Loading ts_kernel_services.ko module”
insmod /lib/ts_kernel_services.ko
echo “Loading ts_ib_packet_lib.ko module”
insmod /lib/ts_ib_packet_lib.ko
echo “Loading ts_ib_core.ko module”
insmod /lib/ts_ib_core.ko
echo “Loading ts_ib_mad.ko module”
insmod /lib/ts_ib_mad.ko
echo “Loading ts_ib_tavor.ko module”
insmod /lib/ts_ib_tavor.ko
echo “Loading scsi_mod.ko module”
insmod /lib/scsi_mod.ko
echo “Loading sd_mod.ko module”
insmod /lib/sd_mod.ko
echo “Loading ts_kernel_poll.ko module”
insmod /lib/ts_kernel_poll.ko
echo “Loading ts_ib_client_query.ko module”
insmod /lib/ts_ib_client_query.ko
echo “Loading ts_ib_sa_client.ko module”
insmod /lib/ts_ib_sa_client.ko
echo “Loading ts_ib_dm_client.ko module”
insmod /lib/ts_ib_dm_client.ko
echo “Loading ts_ib_cm.ko module”
insmod /lib/ts_ib_cm.ko
echo “Loading ts_srp_host.ko module”
insmod /lib/ts_srp_host.ko
echo “Loading emcplib.ko module”
insmod /lib/emcplib.ko
echo “Loading emcp.ko module”
insmod /lib/emcp.ko
echo “Loading emcpmp.ko module”
insmod /lib/emcpmp.ko
echo “Loading emcpmpc.ko module”
insmod /lib/emcpmpc.ko
echo “Loading emcpmpaa.ko module”
insmod /lib/emcpmpaa.ko
echo “Loading emcpmpap.ko module”
insmod /lib/emcpmpap.ko
echo “Loading emcphr.ko module”
insmod /lib/emcphr.ko
echo “Loading emcpioc.ko module”
insmod /lib/emcpioc.ko
echo “Removing module emcpioc.ko”
rmmod emcpioc
echo “Loading mptbase.ko module”
insmod /lib/mptbase.ko

Cisco VFrame Third Party Integration Guide

OL-12603-01 11-5
 Chapter 11 Configuring PowerPath Multipathing for Linux
Archiving the bootdisk Directory

echo “Loading mptscsih.ko module”

insmod /lib/mptscsih.ko
echo “Loading jbd.ko module”
insmod /lib/jbd.ko
echo “Loading ext3.ko module”
insmod /lib/ext3.ko
udevstart

rm -f /dev/emcpower
mmaj=`cat /sys/class/misc/emcpower/dev | gawk -F : '{print $1}'`
mmin=`cat /sys/class/misc/emcpower/dev | gawk -F : '{print $2}'`
mknod /dev/emcpower c $mmaj $mmin

sleep 2

echo “Start the configuration of powerpath”

powermt config
powermt load
/etc/opt/emcpower/emcpmgr map

sleep 7

/etc/opt/emcpower/powercf -C powermt save

powermt register

sleep 2

echo “Creating root device”

echo “mkrootdev /dev/root” | /sbin/nash

umount /sys
echo “Mounting root filesystem”
mount -o defaults --ro -t ext3 /dev/root /sysroot
mount -t tmpfs --bind /dev /sysroot/dev

umount /proc
cd /sysroot
echo “Switching to the new root”
pivot_root . initrd
exec chroot . /sbin/init <dev/console >dev/console 2>&1

echo “Unmount the old dev now”

umount /initrd/dev

Archiving the bootdisk Directory

To archive the bootdisk directory, follow these steps:

Step 1 Archive the bootdisk directory now that all the required files are copied into it.
In this example the archive file is put at /root/
a. cd to the bootdisk directory.
b. Locate and copy the file.
find . | cpio -v -c -o > /root/initrd-2.6.9-34.ELsmp

c. Compress the file.

Cisco VFrame Third Party Integration Guide

11-6 OL-12603-01
 Chapter 11 Configuring PowerPath Multipathing for Linux
Archiving the bootdisk Directory

gzip -9 initrd-2.6.9-34.ELsmp

d. Move the directory.

mv initrd-2.6.9-34.ELsmp.gz initrd-2.6.9-34.ELsmp.img

Completing the PowerPath Configuration

To complete the PowerPath configuration, follow these steps:

Step 1 Move the initrd-2.6.9-34.ELsmp.img file to the /boot directory.

Step 2 Change the bootloader append for the root parameter of the kernel to the appropriate PowerPath pseudo
device.
In this example the PowerPath pseaudo device was root=/dev/emcpowera3
Step 3 Reboot the server. It will boot the root filesystem on a PowerPath device instead of a sd device.

Cisco VFrame Third Party Integration Guide

OL-12603-01 11-7
 Chapter 11 Configuring PowerPath Multipathing for Linux
Archiving the bootdisk Directory

Cisco VFrame Third Party Integration Guide

11-8 OL-12603-01