3par RedHat 45 Implementation Guide

3PAR Confidential
3PAR RedHat 4 & 5 and OEL

(Oracle Enterprise Linux)
Implementation Guide
3PAR, Inc.
4209 Technology Drive
Fremont, CA 94538 USA
Part No. 320-200268 Rev A
December 2010

3PAR Confidential
Revision Notice
This is the first release of this manual. A complete revision history is provided at the end of this document.
Changes
The material in this document is for information only and is subject to change without notice. While reasonable efforts have been
made in the preparation of this document to assure its accuracy, 3PAR Inc. assumes no liability resulting from errors or omissions in
this document or from the use of the information contained herein.
3PAR reserves the right to make changes in the product design without reservation and without notification to its users.
Updates to the Documentation at 3PAR Central
Any updates to this document, or to other 3PAR technical documents, can be found by logging in to 3PAR Centrals Document
Control System from 3PARs Support page at http://support.3PAR.com.
3PAR Technical Support and Services
Contact your local service provider for technical support and services at: http://www.3PAR.com/services.html.
Sales and Ordering Information
For sales and ordering information, contact:
3PAR Inc.
4209 Technology Drive
Fremont, CA 94538 USA
Telephone: 510-413-5999
Fax: 510-413-5699
Email: salesinfo@3PAR.com
Reader Comments and Suggestions
Please email your comments and suggestions about this document to ReaderComments@3PAR.com.
Copyrights
2010 3PAR Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any
form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written consent of 3PAR Inc.,
4245 Technology Drive, Fremont, CA 94538. By way of exception to the foregoing, the user may print one copy of electronic material
for personal use only.
Trademarks
3PAR, InServ, InForm, InSpire and Serving Information are registered trademarks of 3PAR Inc.
Intel and Pentium are registered trademarks of Intel Corporation.
Linux is a registered trademark of Linus Torvalds.
RedHat is a registered trademark of RedHat, Inc.
SuSE is a registered trademark of Novell, Inc.
Microsoft, Windows, and Windows NT are either registered trademarks or trademarks of Microsoft Corporation.
Oracle is a registered trademark of Oracle Corporation.
Sun, Solaris, and Java are trademarks of Sun Microsystems, Inc.
UNIX is a registered trademark of The Open Group.
All other trademarks and registered trademarks are owned by their respective owners.
3PAR Confidential
iii Table of Contents
3PAR RedHat 4 and RedHat 5 Implementation Guide
Table of Contents
1 Introduction
1.1 Supported Configurations 1.2
1.2 InForm OS Upgrade Considerations 1.2
1.3 Audience 1.2
1.4 Related Documentation 1.3
1.5 Organization 1.4
1.6 Typographical Conventions 1.5
1.7 Advisories 1.6
2 Configuring the InServ Storage Server for Fibre Channel
2.1 Configuring the InServ Storage Server Running InForm OS 2.3.X 2.2
2.1.1 Configuring Ports for a Direct Connection 2.2
2.1.2 Configuring Ports for a Fabric Connection 2.3
2.1.3 Creating the Host Definition 2.5
2.3 Connecting the InServ Storage Server to the Host 2.8
iv Table of Contents

3PAR Confidential
2.4 Setting Up and Zoning the Fabric 2.8
2.4.1 Configuration Guidelines for Fabric Vendors 2.9
2.4.2 Target Port Limits and Specifications 2.10
2.4.3 Configuration Diagrams for Zoning 2.10
3 Configuring the InServ Storage Server for iSCSI
3.1 Configuring Ports for an iSCSI Connection 3.1
3.2 Creating the iSCSI Host Definition 3.3
4 Connecting a Host Server with Fibre Channel
4.1 Checking the Host for Required Packages 4.2
4.2 Installing the Emulex HBA 4.2
4.2.1 Building the Emulex Driver 4.3
4.2.2 Modifying the etc/modprobe.conf file and Building the Ramdisk 4.4
4.3 Installing the QLogic HBA 4.8
4.3.1 Building the QLogic Driver 4.8
4.3.2 Modifying the /etc/modprobe.conf file and Building the Ramdisk 4.9
4.4 Setting the SCSI Timeout 4.12
4.4.1 Using UDEV Rules to Set the SCSI Timeout 4.13
4.4.2 Using QLogic Scripts to Set the SCSI Timeout 4.15
4.4.3 Using Emulex Scripts to Set the SCSI Timeout 4.16
4.5 Setting Up Multipathing Software 4.18
4.5.1 Setting Up Device Mapper 4.18
4.5.2 Setting Up Veritas DMP Multipathing 4.22
4.5.3 Installing the 3PAR Host Explorer Package 4.24
5 Connecting a Host Server with iSCSI
5.1 Setting Up the Ethernet Switch and Network Interface Card 5.2
5.2 Configuring RedHat 4 for iSCSI 5.2
5.2.1 Installing iSCSI on RedHat 4 5.2
5.2.2 Setting Up iSCSI for RedHat 4 5.3
5.2.3 Configuring RedHat 4 iSCSI Settings with Device Mapper Multipathing 5.4
v Table of Contents

3PAR Confidential
5.2.4 Starting the iSCSI Daemon for RedHat 4 5.5
5.3.5 Creating the iSCSI Connection in RedHat 5 Using the iscsiadm Command 5.11
5.4 Configuring CHAP for the iSCSI Host 5.14
5.4.1 Setting the Host CHAP Authentication on the InServ Storage Server 5.14
5.4.2 Setting Up the Bidirectional CHAP on the InServ Storage Server 5.17
6 Allocating Storage for Access by the RedHat Host
6.1 Creating Storage on the InServ Storage Server 6.2
6.1.1 Creating Virtual Volumes 6.2
6.1.2 Creating Thin Provisioned Volumes 6.3
6.2 Exporting LUNs to the Host 6.3
6.3 Restrictions on Volume Size and Number 6.5
6.4 Discovering Devices with an Emulex HBA 6.6
6.4.1 Scan Methods for LUN Discovery 6.6
6.4.2 Verifying Devices Found by the Host Using the Emulex HBA 6.7
6.5 Discovering Devices with a QLogic HBA 6.9
6.5.2 Verifying Devices Found by the Host Using the QLogic HBA 6.12
6.6 Discovering Devices with an iSCSI Connection 6.14
6.6.1 Discovering Devices with RedHat 4 6.14
7 Modifying 3PAR Devices on the Host Server
7.1 Creating Device Mapper Devices 7.1
7.2 Displaying Detailed Device Mapper Node Information 7.5
7.3 Partitioning Device Mapper Nodes 7.6
vi Table of Contents

3PAR Confidential
7.4 Creating Veritas Volume Manager Devices 7.8
7.5 Removing a Storage Volume from the Host 7.9
8 Booting the Host from the InServ Storage Server
8.1 Inserv Storage Server Setup Requirements 8.1
8.2 RedHat Host HBA BIOS Setup Considerations 8.2
8.2.1 Booting from the InServ Storage Server Using QLogic HBAs 8.2
8.2.2 Booting from the InServ Storage Server Using Emulex HBAs 8.3
8.3 Installation from RedHat 5 Linux CDs 8.4
8.4 Modifying the /etc/multipath.conf file 8.5
8.5 Changing the Emulex HBA Inbox Driver Parameters 8.6
8.6 Installing the New QLogic Driver 8.6
9 Using Veritas Cluster Servers
10 Using RedHat Xen Virtualization
11 Using RedHat Cluster Services
12 Using RedHat KVM Hypervisor
A RedHat iscsiadm Utility Usage
B Setting up the NVRAM and BIOS with QLogic HBA
C Configuring the QLogic HBA Using the SCLI Utility
RH Revision History
1.1 Introduction
3PAR Confidential
1
Introduction
In this chapter
1.3 Audience 1.2
1.4 Related Documentation 1.3
1.5 Organization 1.4
1.6 Typographical Conventions 1.5
1.7 Advisories 1.6
This implementation guide provides the information you need to configure a 3PAR InServ
Storage Server with RedHat 4 and RedHat 5. General information is also provided on the basic
steps required to allocate storage on the InServ Storage Server that can then be accessed by
the RedHat host.
The information contained in this implementation guide is the outcome of careful testing of
the 3PAR InServ Storage Server with as many representative hardware and software
configurations as possible.
REQUIRED: For predictable performance and results with your
3PAR InServ Storage Server, the information in this guide must be used in concert
with the documentation set provided by 3PAR for the InServ Storage Server and
the documentation provided by the vendor for their respective products.
1.2
Supported Configurations
3PAR Confidential
1.1 Supported Configurations
The following types of host connections are supported between the InServ Storage Server and
hosts running Linux OS:
Fibre Channel
iSCSI
For complete details on supported host configurations, consult the latest 3PAR InForm
Configuration Matrix that is available on DCS.
1.2 InForm OS Upgrade Considerations
Refer to the InForm OS Upgrade Pre-Planning Guide for information and planning of an online
InForm OS upgrade.
1.3 Audience
This implementation guide is intended for system and storage administrators who monitor and
direct system configurations and resource allocation for 3PAR InServ Storage Servers. The tasks
described in this manual assume that the administrator is familiar with RedHat 4/RedHat 5 and
the 3PAR Inform OS. Although this guide attempts to provide the basic information that is
required to establish communications between the InServ Storage Server and the RedHat host,
and to allocate the required storage for a given configuration, the appropriate 3PAR
documentation must be consulted in conjunction with the RedHat host and HBA vendor
documentation for specific details and procedures.
NOTE: This Implementation Guide does NOT intend to reproduce any third-party
product documentation. For details about devices such as host servers, HBAs,
fabric and Ethernet switches, and non-3PAR software management tools, consult
the appropriate third-party documentation.
1.3
Related Documentation
3PAR Confidential
1.4 Related Documentation
The following documents also provide information related to InServ Storage Servers and the
InForm Operating System:
For information about Read the
Specific platforms supported 3PAR InForm Configuration
Matrix
CLI commands and their usage InForm OS Command Line
Interface Reference
Using the InForm Management
Console to configure and
administer InServ Storage Servers
3PAR InForm OS Management
Console Online Help
InServ Storage Server concepts
and terminology
3PAR InForm OS Concepts Guide
Determining InServ Storage
Server hardware specifications,
installation considerations,
power requirements, networking
options, and cabling
3PAR InServ S-Class/T-Class
Storage Server Physical Planning
Manual or the 3PAR InServ E-
Class/F-Class Storage Server
Planning Manual
Identifying storage server
components and detailed alert
information
3PAR InForm OS Messages and
Operators Guide
Using 3PAR Remote Copy 3PAR Remote Copy Users Guide
Using 3PAR CIM 3PAR CIM API Programming
Reference
1.4
Organization
3PAR Confidential
1.5 Organization
This guide is organized as follows:
Chapter 1, Introduction (this chapter), provides an overview of this guide, including
information on audience, related documentation, and typographical conventions.
Chapter 2, Configuring the InServ Storage Server for Fibre Channel, provides instructions
for configuring the InServ Storage Server for a Fibre Channel connection.
Chapter 3, Configuring the InServ Storage Server for iSCSI. provides instructions for
configuring the InServ Storage Server for an iSCSI connection.
Chapter 4, Connecting a Host Server with Fibre Channel, provides instructions for preparing
the RedHat host for connection to the InServ Storage Server over a Fibre Channel network.
Chapter 5, Connecting a Host Server with iSCSI, provides instructions for preparing the
RedHat host server for connection to the InServ Storage Server over an iSCSI network.
Chapter 6, Allocating Storage for Access by the RedHat Host, provides instructions for
creating storage on the InServ Storage Server, exporting LUNs to the RedHat host, and
performing LUN discovery from the RedHat host server.
Chapter 7, Modifying 3PAR Devices on the Host Server, provides instructions for modifying
3PAR devices on a RedHat host with a Fibre Channel or iSCSI connection.
Chapter 8, Booting the Host from the InServ Storage Server, provides instructions for
configuring a RedHat host for booting from the InServ Storage Server.
Chapter 9, Using Veritas Cluster Servers, indicates support and where to find for
configuring the Veritas Cluster Server with RedHat host.
Chapter 10, Using RedHat Xen Virtualization, indicates support for the RedHat host for use
as virtual machine (Xen).
Chapter 11, Using RedHat Cluster Services, indicates support for the RedHat Cluster
Services.
Chapter 12, Using RedHat KVM Hypervisor, indicates support for using the RedHat host as a
Kernel Virtual Machine (KVM).
Appendix A, RedHat iscsiadm Utility Usage, provides examples of iscsiadm commands.
Appendix B, Setting up the NVRAM and BIOS with QLogic HBA, provides instructions for
setting NVRAM BIOS for the QLogic HBA.
1.5
Typographical Conventions
3PAR Confidential
Appendix C, Configuring the QLogic HBA Using the SCLI Utility, provides instructions for
configuring the QLogic HBA using SCLI utility.
This guide also contains a revision history for your reference.
1.6 Typographical Conventions
This guide uses the following typographical conventions:
Typeface Meaning Example
ABCDabcd Used for dialog
elements such as
titles, button
labels, and other
screen elements.
When prompted, click Finish
to complete the installation.
ABCDabcd Used for paths,
filenames, and
screen output.
Open the file
\ os\ wi ndows\ set up. exe
ABCDabcd Used to
differentiate user
input from screen
output.
# cd \opt\3par\console
<ABCDabcd> Used for variables
in filenames,
paths, and screen
output.
Modify the content string by
adding the - P<x>option
after - j ar i nf or m. j ar
<ABCDabcd> Used for variables
in user input.
#.\java -jar inform.jar
-P<x>
1.6
Advisories
3PAR Confidential
1.7 Advisories
To avoid injury to people or damage to data and equipment, be sure to observe the cautions
and warnings in this guide. Always be careful when handling any electrical equipment.
NOTE: Notes are reminders, tips, or suggestions that supplement the procedures
included in this guide.
CAUTION: Cautions alert you to actions that can cause damage to equipment,
software, or data.
REQUIRED: Requirements signify procedures that must be followed as directed in
order to achieve a functional and supported implementation based on testing at
3PAR.
WARNING: Warnings alert you to actions that can cause injury to people or
irreversible damage to data or the operating system.
2.1 Configuring the InServ Storage Server for Fibre Channel
3PAR Confidential
2
Configuring the InServ
Storage Server for Fibre
Channel
In this chapter
2.3 Connecting the InServ Storage Server to the Host 2.8
2.4 Setting Up and Zoning the Fabric 2.8
2.4.1 Configuration Guidelines for Fabric Vendors 2.9
2.4.2 Target Port Limits and Specifications 2.10
2.4.3 Configuration Diagrams for Zoning 2.10
2.2
Configuring the InServ Storage Server Running InForm OS 2.3.X
3PAR Confidential
This chapter describes how to establish a connection between InServ Storage Server and
RedHat host using Fibre Channel and how to set up the fabric when running InForm OS 2.3.X
and InForm OS 2.2.X. For information on setting up the physical connection for a particular
InServ Storage Server, see the appropriate 3PAR installation manual.
2.1 Configuring the InServ Storage Server Running InForm OS 2.3.X
This section describes how to configure InServ Storage Server running InForm OS 2.3.X.
2.1.1 Configuring Ports for a Direct Connection
To configure InServ Storage Server ports for a direct connection to the RedHat host, complete
the following steps:
1 To set up the InServ Storage Server ports for a direct connection, issue the following set of
commands with the appropriate parameters for each direct connect port:
a cont r ol por t of f l i ne [ node: sl ot : por t ]
b cont r ol por t conf i g host ct l oop [ node: sl ot : por t ]
where - ct l oop specifies a direct connection.
c cont r ol por t r st [ node: sl ot : por t ]
Example:
REQUIRED: If you are setting up a fabric along with your installation of the
InServ Storage Server, consult 2.4 Setting Up and Zoning the Fabric on page 2.8
before configuring or connecting your InServ Storage Server.
REQUIRED: The following setup must be completed before connecting the InServ
Storage Server port to a device.
# cont r ol por t of f l i ne 1: 5: 1
# cont r ol por t conf i g host - ct l oop 1: 5: 1
# cont r ol por t r st 1: 5: 1
2.3
3PAR Confidential
2 After all ports have been configured, verify that the ports are configured for a host in a
direct connection by issuing the showpor t - par command on the InServ Storage Server.
2.1.2 Configuring Ports for a Fabric Connection
To configure InServ Storage Server ports for a fabric connection, complete the following steps
for each port connecting to a fabric.
1 To determine if a port has already been configured for a host port in fabric mode, issue
showpor t - par on the InServ Storage Server.
# showport -par
N: S: P Connmode ConnType Cf gRat e MaxRat e Cl ass2 Uni qNodeWwn VCN I nt Coal
0: 0: 1 di sk l oop aut o 2Gbps di sabl ed di sabl ed di sabl ed enabl ed
0: 4: 1 host poi nt aut o 4Gbps di sabl ed di sabl ed di sabl ed enabl ed
0: 5: 2 host l oop aut o 2Gbps di sabl ed di sabl ed di sabl ed enabl ed
1:5:1 host loop auto 4Gbps disabled disabled disabled enabled
CAUTION: Before taking a port offline in preparation for a fabric connection, you
should verify that the port has not been previously defined and that it is not
already connected to a host as this would interrupt the existing host connection.
If an InServ port is already configured for a fabric connection, you can ignore
step 2 on page 2.4 as you do not have to take the port offline.
2.4
3PAR Confidential
2 If the port has NOT been configured, take the port offline before configuring it for
connection to a host server. To take the port offline, issue the CLI command cont r ol por t
of f l i ne [ node: sl ot : por t ] .
3 To configure the port to the host server, issue cont r ol por t conf i g host ct
poi nt [ node: sl ot : por t ] , where - ct poi nt indicates that the connection type
specified is a fabric connection.
4 Reset the port by issuing the cont r ol por t r st [ node: sl ot : por t ] command.
#showport -par
N: S: P Connmode ConnType Cf gRat e MaxRat e Cl ass2 Uni qNodeWwn VCN I nt Coal
1:5:1 host point auto 4Gbps disabled disabled disabled enabled
# cont r ol por t of f l i ne 1: 5: 1
# cont r ol por t conf i g host ct poi nt 1: 5: 1
# cont r ol por t r st 1: 5: 1
2.5
3PAR Confidential
2.1.3 Creating the Host Definition
Before connecting the RedHat host to the InServ Storage Server, create a host definition that
specifies a valid host persona for each InServ Storage Server that is to be connected to a host
HBA port through a fabric or a direct connection.
1 To create host definitions, issue the cr eat ehost [ opt i ons] <host name>
[ <WWN>. . . ] command. For example:
2 To verify that the host has been created, issue the showhost command.
# cr eat ehost - per sona 1 r edhat host 1122334455667788 1122334455667799
Set t i ng def aul t host per sona 1 ( Gener i c)
# showhost
I d Name Per sona - WWN/ i SCSI _Name- Por t
6 r edhat host Gener i c 1122334455667788 - - -
1122334455667799 - - -
NOTE: 3PAR recommends Host persona 1 for hosts running RHEL 4 update 6 and
above RHEL 5.0 and above to enable Host_Explorer functionality.
However, host persona 6 is automatically assigned following a rolling upgrade
from 2.2x. If appropriate, you can change host persona 6 after an upgrade to
host persona 1.
Host Persona 1 enables two functional features: Host_Explorer, which requires the
SESLun element of host persona 1, and the UARepLun, which notifies the host of
newly exported vLUNs and should trigger a LUN discovery request on the host,
making the vLUN automatically available. Currently, none of the supported RHEL
versions use the UARepLun, so you must manually scan the newly exported vLUNs.
NOTE: See the 3PAR InForm OS Command Line Reference or the IMC help for
complete details on using the cont r ol por t , cr eat ehost and showhost
commands.
2.6
3PAR Confidential
2.2 Configuring the InServ Storage Server Running InForm OS 2.2.X
This section describes how to configure InServ Storage Server running OS 2.2.X.
2.2.1 Configuring Ports for a Direct Connection
To configure the InServ Storage Server ports for a direct connection, complete the following
steps.
1 Set each InServ Storage Server port to port personality 1 by issuing cont r ol por t per sona
1 <X: X: X>, where <X: X: X>is the port location, expressed as node:slot:port.
2 Issue cont r ol por t vcn di sabl e - f <X: X: X>.
3 Verify that each port has the appropriate personality defined:
2.2.2 Configuring Ports for a Fabric Connection
To configure the InServ Storage Server ports for a fabric connection, complete the following
steps.
1 Set each storage server port that will connect to a fabric to port persona 7 by issuing
cont r ol por t per sona 7 <X: X: X>, where <X: X: X> is the port location, expressed as
node:slot:port.
2 Issue cont r ol por t vcn di sabl e - f <X: X: X> for each port.
3 Verify each port has the appropriate persona defined:
REQUIRED: The following setup must be completed before connecting the InServ
Storage Server port to a device.
# showpor t - par
N: S: P ConnTypeCf gRat eCl ass2 VCN - - - - - - - - - - - Per sona- - - - - - - - - - - - I nt Coal
4: 0: 2 l oop aut o di sabl e di sabl e *( 1) g_ven, g_hba, g_os, 0, DC enabl ed
# showport -par
N: S: P ConnType Cf gRat e Cl ass2 VCN - - - - - - - - - - - Per sona- - - - - - - - - - - - I nt Coal
4: 0: 2 poi nt aut o di sabl e di sabl e *( 7) g_ven, g_hba, g_os, 0, FA enabl ed
2.7
3PAR Confidential
2.2.3 Creating the Host Definition
Before connecting the RedHat host to the InServ Storage Server, create a host definition that
specifies a valid host persona for each InServ Storage Server that is to be connected to a host
HBA port through a fabric or a direct connection.
1 To create host definitions on the InServ Storage Server, issue cr eat ehost [ opt i ons]
<host name> [ <WWN>] .
2 To verify the host definition, issue showhost :
# cr eat ehost r edhat host 1122334455667788 1122334455667799
# showhost
2 r edhat host 1122334455667788 4: 0: 1
1122334455667799 5: 0: 1
2.8
Connecting the InServ Storage Server to the Host
3PAR Confidential
2.3 Connecting the InServ Storage Server to the Host
During this stage connect the InServ Storage Server to the host server directly or to the fabric.
This set of tasks includes physically cabling the InServ Storage Server to the host server or
fabric.
2.4 Setting Up and Zoning the Fabric
Fabric zoning controls which devices have access to each other on the fabric. The required use
of single initiator to single target zoning isolates the host server and InServ Storage Server
ports from Registered State Change Notifications (RSCNs) that are irrelevant to these ports.
You can set up fabric zoning by associating the device World Wide Names (WWNs) or ports
with specified zones in the fabric. Although you can use either the WWN or the port zoning
methods with the InServ Storage Server, the WWN zoning method is recommended because
the zone survives the changes of ports when cables are reconnected on a fabric.
Use the methods provided by the switch vendor to create relationships between host server
HBA ports and storage server ports before you connect the host server HBA ports or InServ
Storage Server ports to the fabric.
After connecting each host server HBA port and InServ Storage Server port to the fabric(s),
verify the switch and zone configurations using the InForm CLI showhost command, to ensure
that each initiator is zoned with the correct target.
In the following explanations an initiator port (initiator for short) refers to a host server HBA
port and a target port (target for short) refers to an InServ Storage Server HBA port.
REQUIRED: When you establish zoning with the InServ Storage Server, there must
only be a single initiator zoned with a single target. If an issue occurs using
another zoning approach, 3PAR may require that you implement this zoning
approach as part of troubleshooting and/or corrective action.
2.9
Setting Up and Zoning the Fabric
3PAR Confidential
2.4.1 Configuration Guidelines for Fabric Vendors
Use the following fabric vendor guidelines before configuring ports on fabric(s) to which the
InServ Storage Server connects.
Brocade switch ports that connect to a host server HBA port or to an InServ Storage Server
port should be set to their default mode. On Brocade 3xxx switches running Brocade
firmware 3.0.2 or later, verify that each switch port is in the correct mode using the Brocade
telnet interface and the por t cf gshowcommand as follows:
McData switch or director ports should be in their default modes as type GX-Port with a
speed setting of Negotiate.
Cisco switch ports that connect to InServ Storage Server ports or host HBA ports should be
set to AdminMode = FX and AdminSpeed = auto port, with the speed set to auto
negotiate.
QLogic switch ports should be set to port type GL-port and port speed auto-detect.
QLogic switch ports that connect to the InServ Storage Server should be set to I/O Stream
Guard disable or auto, but never enable.
br ocade2_1: admi n> por t cf gshow
Por t s 0 1 2 3 4 5 6 7
- - - - - - - - - - - - - - - - - +- - +- - +- - +- - +- - - - +- - +- - +- -
Speed AN AN AN AN AN AN AN AN
Tr unk Por t ON ON ON ON ON ON ON ON
Locked L_Por t . . . . . . . . . . . . . . . .
Locked G_Por t . . . . . . . . . . . . . . . .
Di sabl ed E_Por t . . . . . . . . . . . . . . . .
wher e AN: Aut oNegot i at e, . . : OFF, ??: I NVALI D.
2.10
3PAR Confidential
2.4.2 Target Port Limits and Specifications
To avoid overwhelming a target port and ensure continuous I/O operations, refer to the
following limitations on a target port:
Maximum of 64 host server ports per InServ Storage Server port, with a maximum total of
1,024 host server ports per InServ Storage Server.
I/O queue depth on each InServ Storage Server HBA model as follows:
QLogic 2G: 497
LSI 2G: 510
Emulex 4G: 959
3PAR HBA 4G: 1638
The I/O queues are shared among the connected host server HBA ports on a first-come,
first-served basis
When all queues are in use and a host HBA port tries to initiate I/O, it receives a target
queue full response from the InServ Storage Server port. This condition can result in erratic
I/O performance on each host server. If this condition occurs, each host server should be
throttled so that it cannot overrun the InServ Storage Server port's queues when all host
servers are delivering their maximum number of I/O requests.
2.4.3 Configuration Diagrams for Zoning
This section describes various zoning configurations.
NOTE: When host server ports can access multiple targets on fabric zones, the
assigned target number assigned by the host driver for each discovered target can
change when the host server is booted and some targets are not present in the
zone. This situation may change the device node access point for devices during a
host server reboot. This issue can occur with any fabric-connected storage, and is
not specific to the InServ Storage Server.
NOTE: In the examples in the following sections, a fabric can be one or more Fibre
Channel switches or directors.
2.11
3PAR Confidential
2.4.3.1 Single Initiator to Single Target Zoning No Fan-In No Fan-Out
With a single initiator to single target zoning, in a no fan-in, no fan-out configuration, each
HBA port is connected to only one InServ Storage Server port, as shown in Figure 2-1.
Figure 2-1. Single Initiator to Single Target Zoning No Fan-In/No Fan-Out
0004_IG_R1
Node 0 Node 1
Host Server A Host Server B
InServ
Fabric Fabric
A single initiator to
a single target in a zone
to target port
(InServ)
to initiator port
(host)
port
port
Zone
2.12
3PAR Confidential
2.4.3.2 Single Initiator to Single Target Zoning with Fan-Out from One
InServ Storage Server Port to Multiple Host Server Ports
With single initiator to single target zoning, in a fan-out configuration, the InServ Storage
Server port is connected to more that one host port, as shown in Figure 2-2.
Figure 2-2. Single Initiator to Single Target Zoning with Fan-Out
NOTE: A maximum of 64 host server ports can fan-out from a single
InServ Storage Server port.
0003_IG_R1
Node 0 Node 1
InServ
Fabric
Fabric
2.13
3PAR Confidential
2.4.3.3 Single Initiator to Single Target Zoning with Fan-In from Multiple
InServ Storage Server Ports to One Host Server Port
With single initiator to single target zoning, in a fan-in configuration a host server port
connects to many InServ Storage Server ports, as shown in Figure 2-3.
Figure 2-3. Single Initiator to Single Host Target Zoning with Fan-In
0002_IG_R1
Node 0 Node 1
Host Server
InServ
Fabric Fabric
NOTE: A maximum of four InServ Storage Server ports can fan-in to a single host
server port.
2.14
3PAR Confidential
2.4.3.4 Single Initiator to Single Target Zoning with Mixed Fan-In and Fan-Out
Configurations
With single initiator to single target zoning, in a fan-in and fan-out configuration, one InServ
Storage Server connects to multiple host servers.
Figure 2-4. Single Initiator to Single Target Zoning with Fan-In and Fan-Out
0005_IG_R1
Node 0 Node 1
InServ
Fabric Fabric
2.15
3PAR Confidential
2.4.3.5 Non-Compliant Zoning Examples
In the following example, the zoning rule of one initiator zoned to one target is not respected,
as shown in Figure 2-5.
Figure 2-5. Non-Compliant Zoning
0001_IG_R1
Node 0
No zoning
Node 1
Host Server A
Fabric A
Host Server B
C
B
A
Multiple initiators to a single target B
Single initiator to multiple targets C
InServ
Non-Compliant Zoning
Fabric
2.16
3PAR Confidential
3.1 Configuring the InServ Storage Server for iSCSI
3PAR Confidential
3
Configuring the InServ
Storage Server for iSCSI
In this chapter
3.1 Configuring Ports for an iSCSI Connection 3.1
3.2 Creating the iSCSI Host Definition 3.3
3.1 Configuring Ports for an iSCSI Connection
To configure an iSCSI target port on the InServ Storage Server for connection to an iSCSI
Initiator, complete the following steps:
1 Check the current settings of the iSCSI ports by issuing showpor t - i scsi .
2 Set up the IP address and netmask address of the iSCSI target ports by issuing
cont r ol i scsi por t addr <i paddr > <net mask> [ - f ] node: sl ot : por t .
# showpor t - i scsi
N: S: P St at e I PAddr Net mask Gat eway TPGT MTU Rat e DHCP i SNS_Pr i mi SNS_Sec i SNS_Por t
0: 3: 1 of f l i ne 0. 0. 0. 0 0. 0. 0. 0 0. 0. 0. 0 31 1500 n/ a 0 0. 0. 0. 0 0. 0. 0. 0 3205
0: 3: 2 l oss_sync 0. 0. 0. 0 0. 0. 0. 0 0. 0. 0. 0 32 1500 n/ a 0 0. 0. 0. 0 0. 0. 0. 0 3205
1: 3: 1 of f l i ne 0. 0. 0. 0 0. 0. 0. 0 0. 0. 0. 0 131 1500 n/ a 0 0. 0. 0. 0 0. 0. 0. 0 3205
1: 3: 2 l oss_sync 0. 0. 0. 0 0. 0. 0. 0 0. 0. 0. 0 132 1500 n/ a 0 0. 0. 0. 0 0. 0. 0. 0 3205
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
4
# cont r ol i scsi por t addr 10. 100. 0. 101 255. 0. 0. 0 - f 0: 3: 1
3.2
Configuring Ports for an iSCSI Connection
3PAR Confidential
3 Verify the changed settings by issuing showpor t - i scsi .
4 Issue the pi ng command to verify that the Ethernet switch ports where the InServ iSCSI
target ports and iSCSI Initiator host connect are visible to each other.
# showpor t - i scsi
N: S: P St at e I PAddr Net mask Gat eway TPGT MTU Rat e DHCP i SNS_Pr i mi SNS_Sec i SNS_Por t
0: 3: 1 r eady 10. 100. 0. 101 255. 0. 0. 0 0. 0. 0. 0 31 1500 1Gbps 0 0. 0. 0. 0 0. 0. 0. 0 3205
0: 3: 2 l oss_sync 0. 0. 0. 0 0. 0. 0. 0 0. 0. 0. 0 32 1500 n/ a 0 0. 0. 0. 0 0. 0. 0. 0 3205
1: 3: 1 r eady 10. 101. 0. 201 255. 0. 0. 0 0. 0. 0. 0 131 1500 1Gbps 0 0. 0. 0. 0 0. 0. 0. 0 3205
1: 3: 2 l oss_sync 0. 0. 0. 0 0. 0. 0. 0 0. 0. 0. 0 132 1500 n/ a 0 0. 0. 0. 0 0. 0. 0. 0 3205
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
4
3.3
Creating the iSCSI Host Definition
3PAR Confidential
3.2 Creating the iSCSI Host Definition
This section describes how to create an iSCSI host definition.
1 You can verify that the iSCSI Initiator is connected to the iSCSI target port by using the CLI
showhost command.
2 Create an iSCSI host definition entry by using the CLI cr eat ehost - i scsi <host name>
<host i SCSI name> command. For example:
NOTE: You must configure the InServ iSCSI target port(s) and establish an iSCSI
Initiator connection/session with the iSCSI target port from the host to be able to
create a host definition entry. For details, see 5.3.5 Creating the iSCSI Connection
in RedHat 5 Using the iscsiadm Command on page 5.11.
# showhost
I d Name Per sona - - - - - - - - - - WWN/ i SCSI _Name- - - - - - - - - - - Por t
- - i qn. 1994- 05. com. r edhat : a3df 53b0a32d 0: 3: 1
i qn. 1994- 05. com. r edhat : a3df 53b0a32d 1: 3: 1
NOTE: 3PAR requires Host persona 1 for hosts running RHEL 4 update 6 and
above RHEL 5.0 and above to enable Host_Explorer functionality.
However, host persona 6 is automatically assigned following a rolling upgrade
from 2.2x. It is required to change host persona 6 after an upgrade to host
persona 1.
Host Persona 1 enables two functional features: Host_Explorer, which requires the
SESLun element of host persona 1, and the UARepLun, which notifies the host of
newly exported vLUNs and should trigger a LUN discovery request on the host,
making the vLUN automatically available. Currently, none of the supported RHEL
versions use the UARepLun, so you must manually scan the newly exported vLUNs.
# cr eat ehost - i scsi r edhat host i qn. 1994- 05. com. r edhat : a3df 53b0a32d
Set t i ng def aul t host per sona 1 ( Gener i c)
3.4
Creating the iSCSI Host Definition
3PAR Confidential
3 Verify that the host entry has been created.
Example of showhost output for InServ Storage Server running OS 2.2.X:
NOTE: For RedHat 4, you can get the iSCSI initiator name from / et c/
i ni t i at or name:
# cat / et c/ i ni t i at or name. i scsi
I ni t i at or Name=i qn. 1987- 05. com. ci sco: 01. 4766d09183f 3
# showhost
0 r edhat host Gener i c i qn. 1994- 05. com. r edhat : a3df 53b0a32d 0: 3: 1
NOTE: For Inserv running OS 2.2.X, the output of showhost appears differently
since there are no Persona fields.
# showhost
I d Name - - - - - - - - - - - WWN/ i SCSI _Name- - - - - - - - - - - - Por t
0 l i nux i qn. 1994- 05. com. r edhat : a3df 53b0a32d 0: 3: 1
4.1 Connecting a Host Server with Fibre Channel
3PAR Confidential
4
Connecting a Host Server
with Fibre Channel
In this chapter
4.1 Checking the Host for Required Packages 4.2
4.2 Installing the Emulex HBA 4.2
4.2.1 Building the Emulex Driver 4.3
4.2.2 Modifying the etc/modprobe.conf file and Building the Ramdisk 4.4
4.3 Installing the QLogic HBA 4.7
4.3.1 Building the QLogic Driver 4.7
4.3.2 Modifying the /etc/modprobe.conf file and Building the Ramdisk 4.8
4.4 Setting the SCSI Timeout 4.12
4.4.1 Using UDEV Rules to Set the SCSI Timeout 4.13
4.4.2 Using QLogic Scripts to Set the SCSI Timeout 4.15
4.2
Checking the Host for Required Packages
3PAR Confidential
4.5 Setting Up Multipathing Software 4.18
4.5.1 Setting Up Device Mapper 4.18
4.5.2 Setting Up Veritas DMP Multipathing 4.22
4.5.3 Installing the 3PAR Host Explorer Package 4.24
This chapter describes the tasks necessary for connecting the host to Fibre Channel. When tasks
are specific to the version of the RedHat OS, headings refer to RedHat 4 or RedHat 5.
4.1 Checking the Host for Required Packages
If you are installing and building the Emulex driver, make sure the Developmental Tool
package that contains the gcc compiler is installed on the RedHat server. If not, install them
from the RedHat installation CD. After installation, verify the following gcc packages were
installed. Some gcc packages may not be needed.
The following example shows gcc compilers installed for RedHat 4 Updat e 6 Li nux.
4.2 Installing the Emulex HBA
Install the Emulex host bus adapter(s) in the host server according to the documentation
provided with the HBAs and host server.
# rpm -qa | grep gcc
gcc- j ava- 3. 4. 6- 9
gcc- 3. 4. 6- 9
compat - gcc- 32- c++- 3. 2. 3- 47. 3
gcc- c++- 3. 4. 6- 9
compat - l i bgcc- 296- 2. 96- 132. 7. 2
l i bgcc- 3. 4. 6- 9
gcc- g77- 3. 4. 6- 9
l i bgcc- 3. 4. 6- 9
4.3
Installing the Emulex HBA
3PAR Confidential
4.2.1 Building the Emulex Driver
If you are installing the Emulex driver instead of using the in-box Emulex driver that was
already installed by the RedHat installation, follow these steps:
1 Download the driver package from www.emulex.com and extract the driver contents by
issuing t ar xvzf l pf c_<ker nel ver si on>_dr i ver _ki t - <ver si on>. t ar . gz.
Example:
2 Change to the driver source directory by issuing cd l pf c_<ker nal ver si on>
_dr i ver _ki t - <ver si on>. For example:
3 Run the l pf c- i nst al l script that builds and installs the l pf c driver. Check the installed
README for more details.
# . / l pf c- i nst al l
The script performs the following:
a The driver source is installed at / usr / sr c/ l pf c from the installed rpm packages
l pf cdr i ver - <ker nal ver si on>_<dr i ver ver si on>. For example:
b The l pf c driver parameters are added to / et c/ modpr obe. conf .
NOTE: Use this section only if you are installing and building the Emulex driver. If
you are using the Emulex driver that was installed by the RedHat installation, skip
to 4.2.2 Modifying the etc/modprobe.conf file and Building the Ramdisk on
page 4.4.
# tar xvfz lpfc_2.6_driver_kit-8.0.16.40-2.tar.gz
l pf c_2. 6_dr i ver _ki t - 8. 0. 16. 40- 2/
l pf c_2. 6_dr i ver _ki t - 8. 0. 16. 40- 2/ l pf cdr i ver _2. 6- 8. 0. 16. 40- 1. noar ch. r pm
l pf c_2. 6_dr i ver _ki t - 8. 0. 16. 40- 2/ l pf c- i nst al l
l pf c_2. 6_dr i ver _ki t - 8. 0. 16. 40- 2/ README
l pf c_2. 6_dr i ver _ki t - 8. 0. 16. 40- 2/ el xl pf c
# cd l pf c_2. 6_dr i ver _ki t - 8. 0. 16. 40- 2
# ls /usr/src/lpfc/lpfcdriver*
l pf cdr i ver - 2. 6- 8. 0. 16. 40- 2
4.4
3PAR Confidential
c The newly built Emulex driver l pf c. ko is copied to / l i b/ modul es/ <uname - r >/
ker nel / dr i ver s/ scsi / l pf c. The current l pf c driver is saved at / usr / sr c/ l pf c/
savedf i l es.
d A new ramdisk is created and the currently running ramdisk is copied as / boot /
i ni t r d- <uname - r >. i mg.
The items in bold were added by the l pf c- i nst al l script to the / et c/ modpr obe. conf
configuration file for a dual ported HBA:
4.2.2 Modifying the etc/modprobe.conf file and Building the Ramdisk
This section describes how to modify the etc/modprobe.conf file to set Emulex HBA parameters
and build the ramdisk.
1 Before building the ramdisk, add the following HBA parameters to the
/ et c/ modpr obe. conf file depending on your version of RedHat. These HBA options
settings are required for desired multipath failover/failback operation:
CAUTION: The new Ramdisk is always created with the name i ni t r d- <uname -
r >. i mg. Edit the boot loader to add the correct ramdisk name.
Example: For kernel <uname - r >=2. 6. 18- 53. el 5, the ramdisk created by the
script will be initrd.2. 6. 18- 53. el 5. i mg.
NOTE: There are number of Emulex driver parameters that can be changed by
modifying the / et c/ modpr obe. conf configuration file that enables these
driver parameter values when the drivers are loaded during boot up. Only the
parameters affecting the InServ Storage Server are discussed here.
# cat /etc/modprobe.conf
al i as et h0 e1000
al i as et h1 e1000
al i as scsi _host adapt er mpt base
al i as scsi _host adapt er 1 mpt scsi h
al i as usb- cont r ol l er ehci - hcd
al i as usb- cont r ol l er 1 uhci - hcd
alias scsi_hostadapter2 lpfc
alias scsi_hostadapter3 lpfc
4.5
3PAR Confidential
For RedHat 5:
opt i ons l pf c l pf c_t opol ogy=0x0 l pf c_devl oss_t mo=1
For RedHat 4:
opt i ons l pf c l pf c_t opol ogy=0x0 l pf c_nodev_t mo=1
2 To increase or modify maximum number of LUNs the OS can discover, add SCSI layer
parameters to / et c/ modpr obe. conf .
For example, for the OS to support 256 LUNs per target port:
opt i ons scsi _mod max_l uns=256
3 Change the / et c/ modpr obe. conf file after making the driver topology changes.
If a fan-out configuration is used, where an InServ Storage Server port is connected to many
hosts through a fabric, it is possible that the target port will run out of I/O buffers and will
result in the target port issuing a QUEUE FULL SCSI status message to any new incoming I/O
requests from any other hosts on that port. To prevent this event, you can throttle the host
Port Queue Depth and LUN Queue Depth. For the Emulex driver, the Port Queue depth is
defined by driver parameter l pf c_hba_queue_dept h and LUN Queue Depth by
l pf c_l un_queue_dept h. Change the default values if any throttling is required.
NOTE: The kernel loads the SCSI drivers from ramdisk in the order in which they
are defined in the modpr obe. conf file and assigns the SCSI device entries (sda,
sdb) in ascending order starting with the first entry for each entry where a SCSI
device exists. If the host has a SCSI boot disk, it must obtain device entry sda since
those entries are hard coded in the bootloaders. Therefore, the
scsi _host adapt er entry that supports the boot disk must appear first in the
/ et c/ modpr obe. conf file.
# cat / et c/ modpr obe. conf
al i as et h0 e1000
al i as et h1 e1000
al i as scsi _host adapt er mpt base
al i as scsi _host adapt er 1 mpt scsi h
al i as usb- cont r ol l er ehci - hcd
al i as usb- cont r ol l er 1 uhci - hcd
al i as scsi _host adapt er 2 l pf c
al i as scsi _host adapt er 3 l pf c
opt i ons l pf c l pf c_t opol ogy=0x02 l pf c_devl oss_t mo=1
4.6
3PAR Confidential
4 Rebuild the ramdisk image using the mki ni t r d command:
# / sbi n/ mki ni t r d - v - f / boot / <r amdi sk i mage name> <ker nel - ver si on>
The following example shows a ramdisk build:
REQUIRED: Storage Administrators should carefully consider the number of hosts
connected to an InServ Storage Server port and the number of LUN exports for
calculating the throttling configuration values. Performance degradation and
SCSI timeout issues will result if the values are set too low.
NOTE: The ramdisk image needs to be rebuilt for any changes made to / et c/
modpr obe. conf to be effective. The system will pick up the ramdisk changes on
bootup.
# /sbin/mkinitrd -v -f /boot/initrd-2.6.18-53.el5.img 2.6.18-53.el5
Cr eat i ng i ni t r amf s
Looki ng f or deps of modul e scsi _mod
Looki ng f or deps of modul e sd_mod scsi _mod
Looki ng f or deps of modul e scsi _t r anspor t _spi : scsi _mod
. . .
copy f r om`/ l i b/ modul es/ 2. 6. 18- 8. el 5/ ker nel / dr i ver s/ scsi /
scsi _t r anspor t _f c. ko'
[ el f 64- x86- 64] t o `/ t mp/ i ni t r d. l 13681/ l i b/ scsi _t r anspor t _f c. ko' [ el f 64- x86-
64]
copy f r om`/ l i b/ modul es/ 2. 6. 18- 8. el 5/ ker nel / dr i ver s/ scsi / l pf c/ l pf c. ko'
[ el f 64-
x86- 64] t o `/ t mp/ i ni t r d. l 13681/ l i b/ l pf c. ko' [ el f 64- x86- 64]
. . .
Loadi ng modul e j bd
Loadi ng modul e ext 3
Loadi ng modul e scsi _mod
Loadi ng modul e scsi _mod wi t h opt i ons max_l uns=256
Loadi ng modul e sd_mod
Loadi ng modul e mpt base
Loadi ng modul e mpt scsi h
Loadi ng modul e scsi _t r anspor t _f c
Loadi ng modul e l pf c wi t h opt i ons l pf c_t opol ogy=0x02 l pf c_devl oss_t mo=1
4.7
Installing the QLogic HBA
3PAR Confidential
5 Check the contents of the / et c/ gr ub. conf or / boot / gr ub/ gr ub. conf with grub as
the bootloader so that the i ni t r d maps to the correct ramdisk image.
4.3 Installing the QLogic HBA
Install the QLogic host bus adapter(s) in the host server based on the documentation provided
with the HBAs and host server.
4.3.1 Building the QLogic Driver
If you are building the QLogic driver, follow these steps:
1 Download the driver package (SANsurfer Linux Installer for RedHat Enterprise Linux kernel)
from www.qlogic.com and extract the driver contents. The example below demonstrates
this step:
Follow the provided README to build the driver.
# t ar xvf z <ql ogi c dr i ver package i n t gz f or mat >
# vi /etc/grub.conf
def aul t =<l abel number >
t i meout =5
hi ddenmenuf
t i t l e RedHat Ent er pr i se Li nux Ser ver ( 2. 6. 18- 8. el 5)
r oot ( hd0, 2)
ker nel / boot / vml i nuz- 2. 6. 18- 8. el 5 r o r oot =LABEL=/ r hgb qui et
i ni t r d / boot / i ni t r d- 2. 6. 18- 8. el 5. i mg
NOTE: If you are using the in-box QLogic driver by the RedHat host installation,
skip this section and go to section 4.3.2 Modifying the /etc/modprobe.conf file
and Building the Ramdisk on page 4.8
4.8
3PAR Confidential
For example:
2 Go to the driver installation directory by issuing the following command:
# cd ql af c- l i nux- <ver si on>- i nst al l
For example:
3 Run the script:
# . / ql i nst al l - i - dp
This script will perform the following:
a Modifies the contents of / et c/ modpr obe. conf .
b Copies the QLogic source to / usr / sr c/ ql ogi c, builds the QLogic driver and copies
the QLogic drivers to / l i b/ modul es/ `uname - r `/ ker nel / dr i ver s/ scsi /
ql a2xxx.
c Builds the ramdisk image using the default ramdisk image name.
d Copies all the old contents before overwriting to /usr / sr c/ ql ogi c/
<dr i ver ver si on>/ backup.
4.3.2 Modifying the /etc/modprobe.conf file and Building the Ramdisk
# tar xvfz qlafc-linux-8.01.07.15-2-install.tgz
ql af c- l i nux- 8. 01. 07. 15- 2- i nst al l /
ql af c- l i nux- 8. 01. 07. 15- 2- i nst al l / set _dr i ver _par am
. . . . . . . . . . . . . . . . .
ql af c- l i nux- 8. 01. 07. 15- 2- i nst al l / ql i nst al l
ql af c- l i nux- 8. 01. 07. 15- 2- i nst al l / ql - pci . i ds
# cd ql af c- l i nux- 8. 01. 07. 15- 2- i nst al l
NOTE: If you are using the QLogic driver that was installed during the OS
installation (inbox driver), steps 1 and 2 will not be necessary. Start at step 3.
4.9
3PAR Confidential
1 Identify the contents added by the installer to / et c/ modpr obe. conf . The following
output is the result of testing based on the QLA23XX, QLE23XX, QLE24XX, and QLE25XX
cards in the system.
2 Modify the contents of / et c/ modpr obe. conf to remove the boldface items above.
alias scsi_hostadapter qla2xxx_conf
al i as scsi _host adapt er 1 ql a2xxx
al i as scsi _host adapt er 2 ql a2300
opt i ons ql a2xxx ql 2xf ai l over =0 Conf i gRequi r ed=0
i nst al l
ql a2xxx
/ sbi n/ modpr obe qla2xxx_conf; / sbi n/ modpr obe - - i gnor e- i nst al l
ql a2xxx
r emove ql a2xxx / sbi n/ modpr obe - r - - f i r st - t i me - - i gnor e- r emove ql a2xxx && {
/
sbin/modprobe -r --ignore-remove qla2xxx_conf; }
4.10
3PAR Confidential
3 Modify the opt i ons ql a2xxx line to include items ql por t _down_r et r y=1
ql 2xf ai l over =0 as shown below.
The modified output of / et c/ modpr obe. conf should look like the following:
If a fan-out configuration is used, where an InServ port is connected to many hosts through the
fabric, it is possible that the target port will run out of I/O buffers and will result in the target
port issuing a QUEUE FULL SCSI status message to any new incoming I/O requests from any host
on that port. To prevent this event, you can throttle the host Port Queue depth and LUN
Queue depth. By default, the QLogic driver sets Port Queue depth (Execution Throttle) to FFFF
(65535) (overriding the default BIOS execution value of 32) and sets the LUN Queue Depth to
32(default). You can throttle LUN Queue depth value to a lower value using the
ql2xmaxqdepth parameter. QLogic does not offer any driver setting to change the Port Queue
depth or Execution Throttle. Change the default values if any throttling is required.
opt i ons ql a2xxx qlport_down_retry=1 ql2xfailover=0 Conf i gRequi r ed=0
i nst al l ql a2xxx / sbi n/ modpr obe - - i gnor e- i nst al l ql a2xxx
r emove ql a2xxx / sbi n/ modpr obe - r - - f i r st - t i me - - i gnor e- r emove ql a2xxx
NOTE: The kernel loads the SCSI drivers from ramdisk in the order in which they
were defined in the / et c/ modpr obe. conf file and assigns the SCSI device
entries (sda, sdb) in ascending order starting with the first entry, for each entry
where a SCSI device exists. If the host has a SCSI boot disk, it must obtain device
entry sda as those entries are hard coded in the bootloaders. Therefore, the
scsi _host adapt er entry that supports the boot disk must appear first in the /
et c/ modpr obe. conf file. The various QLogic driver entries 2300, 2322, 2400 in
the above example are based on the QLogic HBA's present on 3PAR's test server.
4.11
3PAR Confidential
In the following example, the output shows the / et c/ modpr obe. conf when the
ql2xmaxqdepth is set to 16.
4 Rebuild the ramdisk image after the /etc/modprobe.conf file entries are modified.
5 To make the changes, you can issue the mki ni t r d command or use the QLogic driver
script.
# mki ni t r d - f - v / boot / i ni t r d- <uname - r >. i mg <uname - r >.
For example:
6 Perform one of the two following actions to verify that all the required drivers are added
to the ramdisk image:
opt i ons ql a2xxx ql por t _down_r et r y=1 ql 2xf ai l over =0 ql 2xmaxqdept h=16
Conf i gRequi r ed=0
i nst al l ql a2xxx / sbi n/ modpr obe - - i gnor e- i nst al l ql a2xxx
remove qla2xxx /sbin/modprobe -r --first-time --ignore-remove qla2xxx
REQUIRED: Storage Administrators should carefully consider the number of hosts
connected to an InServ Storage Server port and the number of LUN exports for
calculating the throttling configuration values. Performance degradation and
SCSI timeout issues will result if the values are set too low.
# mki ni t r d - f - v / boot / i ni t r d- 2. 6. 18- 53. el 5. i mg 2. 6. 18- 53. el 5
4.12
Setting the SCSI Timeout
3PAR Confidential
Check the verbose output. For example:
Use the QLogic installer script to rebuild the ramdisk image. The backup ramdisk will be
stored at / usr / sr c/ ql ogi c/ <dr i ver ver si on>/ backup.
# . / ql i nst al l - br
7 Check the contents of the / et c/ gr ub. conf or / boot / gr ub/ gr ub. conf with grub as
the bootloader so that the i ni t r d maps to the correct ramdisk image.
4.4 Setting the SCSI Timeout
The SCSI timeout needs to be set at 60 seconds in order for the InServ Storage Server to
operate properly with RedHat servers. Use the following guidelines depending on your version
of RedHat:
RedHat 5the SCSI timeout value is already set to the default value of 60 seconds and does
NOT need to be changed.
RedHat 4the SCSI timeout value is 30 seconds and needs to be changed to 60 seconds.
Cr eat i ng i ni t r amf s
. . . .
. . . .
Looki ng f or deps of modul e scsi _mod
Looki ng f or deps of modul e sd_mod: scsi _mod
. . . .
. . . .
Looki ng f or deps of modul e ql a2xxx: i nt er modul e scsi _mod
Looki ng f or deps of modul e i nt er modul e
. . . .
. . . .
# vi / et c/ gr ub. conf
def aul t =<l abel number >
t i meout =5
hi ddenmenu
t i t l e RedHat Ent er pr i se Li nux Ser ver ( ker nel name)
r oot ( hd0, 0)
ker nel / <ker nel name> r o r oot =LABEL=/ r hgb qui et
i ni t r d / <RamDi sI mage>
4.13
3PAR Confidential
4.4.1 Using UDEV Rules to Set the SCSI Timeout
For RedHat 4 configurations, change the timeout from 30 seconds to 60 seconds using the
udev rules or a SCSI timeout script so that the change will be effective only for 3PAR devices.
The udev rule method is preferable since it changes the SCSI timeout value dynamically
whenever a SCSI device instance is created (for example: / dev/ sda).
If using the timeout script, then run the script manually whenever device instances are created
and the timeout value is lost on reboot or driver reload.
1 Make sure the udev package is installed on your server. If not, install it from the RedHat
CD. For example:
2 Create udev rules 56_3par_timeout.rules under / et c/ udev/ r ul es. d with the following
contents:
/ et c/ udev/ r ul es. d/ 56_3par _t i meout . r ul es
WARNING: If not set to 60 seconds, the SCSI timeout will result in host disks
being taken offline during InServ Storage Server rolling upgrades. Furthermore,
Remote Copy requires the SCSI timeout value of 60 seconds, otherwise remote
copy operations will become stale with a node reboot.
NOTE: The udev rules method has been tested on RedHat Linux Updates 5 and 6.
For RedHat 4 Update 4 and below, check and verify if the udev rule method
works. If it does not work, then use the ql _ch_scsi _t i meout script method in
4.4.2 Using QLogic Scripts to Set the SCSI Timeout on page 4.15 to change the SCSI
timeout value.
# r pm- qa | gr ep udev
udev- 039- 10. 19. el 4. x86_64. r pm
KERNEL="sd*[ ! 0- 9] ", SYSFS{vendor }="3PARdat a", PROGRAM="/ bi n/ sh - c ' echo 60 >
/ sys/ bl ock/ %k/ devi ce/ t i meout ' " NAME="%k"
4.14
3PAR Confidential
The udev rule number 56_3par _t i meout . r ul es should follow after the 50- udev. r ul es.
Change the udev rule number accordingly.
The 56_3par _t i meout . r ul es is selected based on the test system configuration. See 4.4.1
Using UDEV Rules to Set the SCSI Timeout on page 4.13 to verify that the
56_3par _t i meout . r ul es udev rule is working.
4.4.1.1 Verifying the SCSI Timeout Settings
Verify the udev rules setting after the 3PAR storage volumes have been exported to the host
For details, see Allocating Storage for Access by the RedHat Host on page 6.1.
# udevi nf o - a - p / sys/ bl ock/ sdx
For example:
You can also verify the SCSI timeout settings as following:
cat / sys/ cl ass/ scsi _host / host */ devi ce/ */ */ t i meout
For RedHat 5 using Emulex HBAs, verify using the following:
cat / sys/ cl ass/ scsi _host / host */ devi ce/ */ */ */ t i meout
If the udev rule is created after the host sees InServ Storage Server volumes, execute the
udevst ar t command, which runs the udev rules on all devices and sets the timeout to 60.
The time it takes for the udevst ar t command to complete is based on the number of devices
and I/O throughput, so the recommendation is to run the command during non-peak activity.
# udevst ar t
Reboot of host starts the udev rule by default.
REQUIRED: Make sure there is no break between the two lines in the
56_3par _t i meout . r ul es.
# l s / et c/ udev/ r ul es. d/
. . . . .
40- mul t i pat h. r ul es
50- udev. r ul es
51- by- i d. r ul es
56- 3par - t i meout . r ul es
# udevi nf o - a - p / sys/ bl ock/ sdn | gr ep t i meout
SYSFS{t i meout }="60"
4.15
3PAR Confidential
4.4.2 Using QLogic Scripts to Set the SCSI Timeout
The following script changes the SCSI timeout value to 60 seconds for LUNs discovered by each
of the QLogic HBA ports. Use this script if you are running Remote Copy.
If you have implemented the timeout value change using the udev method, then do not use
this script.
When you run the script, the SCSI timeout value for each of the current LUNs discovered will be
changed immediately. However, when rebooting the server, the timeout value will revert to
the default value of 30 seconds.
The following example shows the content for Scr i pt ql _ch_scsi _t i meout . sh:
You can have this script run during the OS boot up sequence by adding the contents of the
script into the / et c/ r c. l ocal file. Make sure / et c/ r c. l ocal file has the permissions
values set to 777.
ql ogi cname="/ sys/ cl ass/ scsi _host "
t i meout =60
l s $ql ogi cname | gr ep "[ 0- 9] [ 0- 9] *" | whi l e r ead l i ne
do
f name=${ql ogi cname}/ $l i ne
cur r =`pwd`
cd $f name
f i nd . - f ol l ow - name "t i meout " | gr ep - v "gener i c" | whi l e r ead l i ne2
do
vendor check=`cat ${l i ne2%t i meout }vendor | gr ep - c "3PARdat a"`
i f [ $vendor check - gt 0 ] ; t hen
echo "modi f yi ng f i l e: [ $f name$l i ne2] "
echo "$t i meout " > $l i ne2
f i
done
cd $cur r
done
4.16
3PAR Confidential
The following example shows the contents of / et c/ r c. l ocal :
4.4.3 Using Emulex Scripts to Set the SCSI Timeout
If the udev rule was not implemented, you can change the SCSI timeout value from 30 seconds
to 60 seconds by running the set _t ar get _t i meout . sh. You can download this shell script
from http://www.emulex.com by selecting Linux Tools on the Emulex Linux driver download
web page. The timeout value is a dynamic variable and can be changed even while I/O is being
served on the devices.
Example: Emulex changing timeout script
# cat / et c. / r c. l ocal
! / bi n/ sh
#
# Thi s scr i pt wi l l be execut ed *af t er * al l t he ot her i ni t scr i pt s.
# You can put your own i ni t i al i zat i on st uf f i n her e i f you don' t
# want t o do t he f ul l Sys V st yl e i ni t st uf f .
t ouch / var / l ock/ subsys/ l ocal
ql ogi cname="/ sys/ cl ass/ scsi _host "
t i meout =60
l s $ql ogi cname | gr ep "[ 0- 9] [ 0- 9] *" | whi l e r ead l i ne
do
f name=${ql ogi cname}/ $l i ne
cur r =`pwd`
cd $f name
f i nd . - f ol l ow - name "t i meout " | gr ep - v "gener i c" | whi l e r ead l i ne2
do
vendor check=`cat ${l i ne2%t i meout }vendor | gr ep - c "3PARdat a"`
i f [ $vendor check - gt 0 ] ; t hen
echo "modi f yi ng f i l e: [ $f name$l i ne2] "
echo "$t i meout " > $l i ne2
f i
done
cd $cur r
done
# . / set _t ar get _t i meout . sh <host _num> <t ar get _i d> <cmd_t i meout >
# l s / sys/ cl ass/ scsi _host / host 2/ devi ce
t ar get 2: 0: 0
4.17
3PAR Confidential
The SCSI instance number is 2 and target is 0 from the above output:
You can also manually change the timeout value using the following commands:
# cd / sys/ cl ass/ scsi _host / host 2/ devi ce/ t ar get 2: 0: 0/ 2: 0: 0: 0
# echo 60 > t i meout
# cat t i meout
60
The set _t ar get _t i meout script needs to be executed for all the SCSI instances of l pf cj
driver; if the operation if performed manually, the command needs to be executed for all the
devices.
# . / set _t ar get _t i meout 2 0 60
modi f yi ng devi ce / sys/ cl ass/ scsi _devi ce/ 2: 0: 0: 0/ devi ce
f ound t i meout at val ue 30
new t i meout val ue i s 60
modi f yi ng devi ce / sys/ cl ass/ scsi _devi ce/ 2: 0: 0: 1/ devi ce
f ound t i meout at val ue 30
new t i meout val ue i s 60
. . .
NOTE: If the Emulex driver is unloaded and reloaded for any reason, the timeout
setting will reset to the default setting of 30 seconds for all Emulex attached
devices. If this occurs, set the timeout value back to 60 seconds using any of the
described methods. This is not applicable if the timeout is changed using the
udev rule.
4.18
Setting Up Multipathing Software
3PAR Confidential
4.5 Setting Up Multipathing Software
3PAR supports the following multipath solutions for RedHat:
Device mapper.
VERITAS Volume Manager.
4.5.1 Setting Up Device Mapper
In RedHat 5.4, multipath tools are installed by default. Prior to RedHat 5.4, multipath tools are
not installed during the default OS installation, so you need to install device mapper from the
RedHat installation CDs, as demonstrated in the following example.
Check for installed device mapper packages by issuing r pm- qa| gr ep devi ce- mapper :
You can use the following commands to configure multipath devices:
mul t i pat h inspects Linux devices to see if there are multiple paths to the same device and
communicates to the kernel device mapper to set up a device map (dm) device for the
device and is responsible for the path coalescing and device map creation.
The mul t i pat hd daemon checks path health and will reconfigure the multipath map
whenever a path comes up or goes down so as to maintain correctly the path mapping
state.
kpar t x reads partition tables on specified devices and creates device maps over partition
segments that are detected.
Device Mapper also depends on the udev and sysf sut i l s filesystem packages. udev is a
user space process which dynamically manages the creation of devices under the / dev/
# mount - r o / dev/ cdr om/ mnt / cdr om
# cd / mnt / cdr om/ RedHat / RPMS
# r pm- i hv - - per cent devi ce- mapper - mul t i pat h- 0. 4. 5- 31. el 4. x86_64. r pm
# umount / mnt / cdr om
# rpm -qa|grep device-mapper
devi ce- mapper - 1. 02. 25- 2. el 4
devi ce- mapper - mul t i pat h- 0. 4. 5- 31. el 4
devi ce- mapper - 1. 02. 25- 2. el 4
4.19
3PAR Confidential
filesystem. The sysf sut i l s package exports the view of the system hardware configuration
to udev userspace process for device node creation. These packages must be present on the
system.
For example:
In RedHat 5.4, the following packages appear after installation:
If / usr is a separate partition and is not part of the root (/) partition in the installed RedHat
Operating System, then copy the shared library l i bsysf s. so and create the required
sysml i nks from the / usr / l i b directory to the / l i b directory.
The following examples show partitions for 32-bit and 64-bit operating systems:
On a 32-bit installed operating System:
# cp / usr / l i b/ l i bsysf s. so. 1. 0. 2 / l i b/
# l n - s / l i b/ l i bsysf s. so. 1. 0. 2 / l i b/ l i bsysf s. so. 1
# l n - s / l i b/ l i bsysf s. so. 1 / l i b/ l i bsysf s. so
On 64-bit installed operating System:
# cp / usr / l i b64/ l i bsysf s. so. 1. 0. 2 / l i b64/
# l n - s / l i b64/ l i bsysf s. so. 1. 0. 2 / l i b64/ l i bsysf s. so. 1
# l n - s / l i b64/ l i bsysf s. so. 1 / l i b64/ l i bsysf s. so
udev- 039- 10. 19. el 4
# r pm- qa | gr ep sysf s
sysf sut i l s- devel - 1. 2. 0- 1
sysf sut i l s- 1. 2. 0- 1
udev- 095- 14. 21. el 5
# r pm- qa | gr ep sysf s
l i bsysf s- 2. 0. 0- 6
sysf sut i l s- 2. 0. 0- 6
4.20
3PAR Confidential
4.5.1.1 Modifying the /etc/multipath.conf File
The / et c/ mul t i pat h. conf file is used by device mapper where the multipathing
parameters have been set. The default installed / et c/ mul t i pat h. conf file must be edited
with the following changes for a minimum configuration connecting to a 3PAR array. Entries
listed in multipath.conf override the default kernel parameters for dm-multipath. In general,
the kernel defaults are sufficient with the exception of the devices entries for 3PAR. In the
specific case of booting the host from an InServ Storage Server volume (a.k.a, SAN boot), there
are additional defaults entries required:
1 Remove or comment out all entries in the / et c/ mul t i pat h. conf file except for the
devices section of devices currently in use.
2 Edit the devices structure to add entries for 3PAR array and remove other product entries
that are not needed.
CAUTION: If / usr is a separate partition, there will be a system hang during
bootup when multipath starts and cannot find the shared library
l i bsysf s. so. 1 because / usr partition gets mounted at the later stage of the
boot process. So, copying the shared library l i bsysf s. so. 1 to / l i b directory
will resolve the issue.
NOTE: The sysf sut i l s- xx package contains the l i bsysf s. so. 1 library. If
any upgrades are made to this package, the new library file should be copied over
to the / l i b directory.
NOTE: Refer to Chapter 8, Booting the Host from the InServ Storage Server for
SAN boot requirements. Refer to RedHat documentation of DM Multipath
Configuration and Administration for additional options in mul t i pat h. conf
entries.
4.21
3PAR Confidential
After all of the edits are made, the relevant sections of / et c/ mul t i pat h. conf should
appear as follows:
4.5.1.2 Enabling Multipath
Perform the following actions to enable multipath.
1 Invoke the mul t i pat h command for any name changes to be effective.
2 Verify that the mul t i pat hd daemon is enabled by the rc script to run on every host boot
up. The following output shows that it is enabled for run-level 3, 4 and 5. Enable it
appropriately for your configuration:
3 Check that the appropriate rc scripts have been created for each run level. The start
number(s) may not match those shown here.
# cat / et c/ mul t i pat h. conf
devi ces {
devi ce {
vendor "3PARdat a"
pr oduct "VV"
pol l i ng_i nt er val 5
no_pat h_r et r y 12
f eat ur es "0"
har dwar e_handl er "0"
pat h_gr oupi ng_pol i cy mul t i bus
pat h_sel ect or "r ound- r obi n 0"
r r _wei ght uni f or m
r r _mi n_i o 100
pat h_checker t ur
f ai l back i mmedi at e
}
}
# chkconf i g - - l i st mul t i pat hd
mul t i pat hd 0: of f 1: of f 2: of f 3: on 4: on 5: on 6: of f
# l s / et c/ r c3. d/ *mul t i *
/ et c/ r c3. d/ S13mul t i pat hd
# l s / et c/ r c5. d/ *mul t i *
4.22
3PAR Confidential
You can also use the chkconf i g command to enable multipathing:
# chkconf i g mul t i pat hd on
4.5.2 Setting Up Veritas DMP Multipathing
For Active/Active multi-path load balancing and failover, install VERITAS Storage Foundation
and High Availability Software, following the instructions given in the VERITAS Volume
Manager Installation Guide and Administrator Guide available from www.VERITAS.com.
When installing the VERITAS Volume Manager, the following VERITAS driver Modules and file
system modules are included:
It is required to have the Veritas Array Support Library (ASL) for 3PAR InServ Storage Server
installed on the RedHat host if you are using a Veritas Storage Foundation version prior to
5.0mp3. To obtain the Veritas Array Support Library (ASL) for 3PAR InServ Storage Server,
complete the following tasks:
1 Download the latest Veritas Array Support Library (ASL) for 3PAR InServ Storage Server
from the http://support.veritas.com site.
2 Under the volume manager for Unix product, search for 3PAR and select the ASL (Array
Support Library) for 3PAR InServ Storage Server on Volume Manager of the version to be
used (or later) for Linux and install it on the host.
NOTE: If using the QLogic HBA, the QLogic Non-Failover Driver should have been
installed for VERITAS DMP support. Device Mapper (DM) or multipath modules
should not be configured if VERITAS DMP is used for multipathing.
/ l i b/ modul es/ <ker nel _ver si on>/ ver i t as/ vxvm/ dmpaaa. ko
/ l i b/ modul es/ <ker nel _ver si on>/ ver i t as/ vxvm/ dmpaa. ko / l i b/ modul es/
<ker nel _ver si on>/ ver i t as/ vxvm/ dmpapf . ko
/ l i b/ modul es/ <ker nel _ver si on>/ ver i t as/ vxvm/ dmpapg. ko
/ l i b/ modul es/ <ker nel _ver si on>/ ver i t as/ vxvm/ dmpj bod. ko / l i b/ modul es/
<ker nel _ver si on>/ ver i t as/ vxvm/ vxdmp. ko
/ l i b/ modul es/ <ker nel _ver si on>/ ver i t as/ vxvm/ vxi o. ko
/ l i b/ modul es/ <ker nel _ver si on>/ ver i t as/ vxvm/ vxspec. ko
/ l i b/ modul es/ <ker nel _ver si on>/ ver i t as/ vxf s/ f dd. ko
/ l i b/ modul es/ <ker nel _ver si on>/ ver i t as/ vxf s/ vxf s. ko
/ l i b/ modul es/ <ker nel _ver si on>/ ver i t as/ vxf s/ vxpor t al . ko
4.23
3PAR Confidential
3 To install the ASL, the Veritas vxconf i gd daemon must be running. Running vxi nst al l
will start the daemon. Once you install the ASL package, you must run the vxdct l enable
command to claim the disk array as a 3PAR array.
4 Configure the VERITAS vxdmp driver to manage the InServ Storage Server paths providing
path failure management and dynamic load balancing.
5 To confirm the VERITAS vxdmp driver has registered and claimed the 3PAR InServ storage
server, issue the following VERITAS command:
If you are using the Veritas Storage Foundation version 5.0mp3 or higher, then you do not
need to install the ASL (Array Support Library) for the InServ Storage Server. To verify that the
InServ Storage Server is recognized and supported by the installation, run the following
command:
However, if the output does not show the InServ Storage Server, perform the following step to
have the storage server added as a 3PARDATA device:
# vxddl admaddsuppor t al l
# vxddladm listsupport libname=libvx3par.so
ATTR_NAME ATTR_VALUE
=======================================================================
LI BNAME l i bvx3par . so
VI D 3PARdat a
PI D VV
ARRAY_TYPE A/ A
ARRAY_NAME 3PARDATA
=======================================================================
VI D 3PARdat a
PI D VV
ARRAY_TYPE A/ A
ARRAY_NAME 3PARDATA
4.24
3PAR Confidential
Then verify the InServ Storage Server is supported, as shown in the following example.
4.5.3 Installing the 3PAR Host Explorer Package
With InForm OS 2.3.1, the Host Explorer daemon running on the RedHat server to send
information about the host configuration to a 3PAR InServ over the Fibre Channel link is
available. For installation and activation of this package, please refer to 3PAR Host Explorer
Users Guide in DCS.
=======================================================================
VI D 3PARdat a
PI D VV
ARRAY_TYPE A/ A
ARRAY_NAME 3PARDATA
WARNING: If the ARRAY_NAME is not designated as 3PARDATA, the
multipathing layer may not discover devices correctly.
5.1 Connecting a Host Server with iSCSI
3PAR Confidential
5
Connecting a Host Server
with iSCSI
In this chapter
5.1 Setting Up the Ethernet Switch and Network Interface Card 5.2
5.3.5 Creating the iSCSI Connection in RedHat 5 Using the iscsiadm Command 5.11
5.2
Setting Up the Ethernet Switch and Network Interface Card
3PAR Confidential
5.4 Configuring CHAP for the iSCSI Host 5.14
5.4.1 Setting the Host CHAP Authentication on the InServ Storage Server 5.14
5.4.2 Setting Up the Bidirectional CHAP on the InServ Storage Server 5.17
5.1 Setting Up the Ethernet Switch and Network Interface Card
Connect the Linux host Ethernet cable and the InServ iSCSI target ports Ethernet cables to the
Ethernet switches.
If you are using VLANs, make sure that the switch ports which connect to the InServ iSCSI
target ports and iSCSI Initiator ports reside in the same VLANs and/or that you can route the
iSCSI traffic between the iSCSI Initiator ports and the InServ iSCSI target ports. Once the iSCSI
Initiator and InServ iSCSI target ports are configured and connected to the switch, you can use
the pi ng command on the iSCSI Initiator host to make sure it sees the InServ iSCSI target ports.
The procedures in this chapter assumes the you have completed the following tasks:
Set up and configuration of the host Network Interface Card (NIC) that will be used by the
iSCSI Initiator software to connect to the InServ iSCSI target ports.
Installation of the iSCSI Initiator software package.
5.2 Configuring RedHat 4 for iSCSI
This section discusses the necessary tasks for setting up iSCSI for RedHat 4.
5.2.1 Installing iSCSI on RedHat 4
Install the iSCSI initiator software package if it has not been installed. The software package
can be installed from the respective Service Pack distribution CDs of your RedHat 4 OS version
using the RPM tool.
NOTE: Ethernet switch VLANs and routing setup and configuration is beyond the
scope of this document. Please consult your switch manufacturer's Guide for
instructions of how to setup VLANs and routing.
5.3
Configuring RedHat 4 for iSCSI
3PAR Confidential
5.2.2 Setting Up iSCSI for RedHat 4
Complete the following steps to setup the RedHat 4 iSCSI host:
1 Check state of the iSCSI service run level information with the chkconf i g command.
# chkconf i g - - l i st | gr ep i scsi
i scsi 0: of f 1: of f 2: of f 3: of f 4: of f 5: of f 6: of f
2 Check your system run level.
# r unl evel
N 5
3 Configure the iSCSI service run level the same as your system run level and verify that the
setting for the run level has changed. Now, every time you boot up the system, the iSCSI
service will run.
# chkconf i g - - l evel 5 i scsi on
i scsi 0: of f 1: of f 2: on 3: on 4: on 5: on 6: of f
4 Edit the / et c/ i scsi . conf file and at the end of the file add the following lines to
configure the InServ iSCSI target port to connect to. In this example we are adding an iSCSI
target port with an IP address of 10.0.0.10 and 10.0.0.20
ConnFai l Ti meout =10
Di scover yAddr ess=10. 0. 0. 10
5 Reload the iscsi service:
# / et c/ i ni t . d/ i scsi r el oad
/ et c/ i ni t . d/ i scsi r el oad
6 Make sure that the mul t i pat hd daemon is not running. If it is, you can stop it by running
the script / et c/ i ni t . d/ mul t i pat hd st op.
# / et c/ i ni t . d/ mul t i pat hd st at us
mul t i pat hd i s st opped
NOTE: Dynamic Driver Reconfiguration: Configuration changes can be made to
the iSCSI driver without having to stop it or to reboot the host system. To
dynamically change the configuration of the driver, insert the /etc/init.d/iscsi
reload to the /etc/iscsi.conf file. This will cause the iSCSI daemon to re-read the
iscsi.conf file and to create any new Discovery Address connections it finds. Those
discovery sessions will then discover targets and create new target connections.
5.4
3PAR Confidential
7 Verify that the module i scsi _sf net is not loaded.
# l smod | gr ep i scsi _sf net
8 Connect to the InServ iSCSI target port by running the script / et c/ i ni t . d/ i scsi that
loads the module i scsi _sf net and starts the iSCSI daemon.
# / et c/ i ni t . d/ i scsi st ar t
Checki ng i scsi conf i g: [ OK ]
Loadi ng i scsi dr i ver : [ OK ]
St ar t i ng i scsi d: [ OK ]
9 Verify that the module i scsi _sf net has been loaded.
# l smod | gr ep i scsi _sf net
i scsi _sf net 96093 26
scsi _t r anspor t _i scsi 14017 1 i scsi _sf net
scsi _mod 145297 7
i scsi _sf net , l pf c, l i bat a, cci ss, ql a2xxx, scsi _t r anspor t _f c, sd_mod
5.2.3 Configuring RedHat 4 iSCSI Settings with Device Mapper Multipathing
listed in mul t i pat h. conf override the default kernel parameters for dm- mul t i pat h.
In general, the kernel defaults are sufficient with the exception of the devices entries for 3PAR.
NOTE: Refer to RedHat documentation of DM Multipath Configuration and
Administration for additional options in mul t i pat h. conf entries.
NOTE: Refer to 4.5.1 Setting Up Device Mapper on page 4.18 for the installation
of the device mapper rpm packages.
5.5
3PAR Confidential
3 Verify that /etc/multipath.conf file contains the following content:
4 Run the mul t i pat h command for any name changes to be effective.
up.
The following output shows that it is enabled for run-level 3, 4 and 5. Enable it appropriately
for your configuration.
numbers may not match those shown here.
Alternatively, you can use the chkconf i g command to enable multipathing if it is not
enabled:
devi ces {
devi ce {
vendor "3PARdat a"
pr oduct "VV"
f eat ur es "0"
r r _mi n_i o 100
pat h_checker t ur
}
}
# l s / et c/ r c3. d/ *mul t i *
# l s / et c/ r c5. d/ *mul t i *
5.6
3PAR Confidential
5.2.4 Starting the iSCSI Daemon for RedHat 4
Complete the following steps to start the iSCSI daemon for the host running RedHat 4.
1 Remove and reload the i scsi _sf net driver with the rc script.
# / et c/ i ni t . d/ i scsi r est ar t
St oppi ng i scsi d: [ OK ]
Removi ng i scsi dr i ver : [ OK ]
# i scsi - l s
2 Check to see if the i scsi d daemon is running by using the script / et c/ i ni t . d/ i scsi
status.
# / et c/ i ni t . d/ i scsi st at us
i scsi d ( pi d 30532 30529) i s r unni ng. . .
If the daemon is running, stop it with the script / et c/ i ni t . d/ i scsi st op.
# / et c/ i ni t . d/ i scsi st op
St oppi ng i scsi d: [ OK ]
Removi ng i scsi dr i ver : [ OK ]
If the i scsi d daemon in not running, start it with the script / et c/ i ni t . d/ i scsi to
establish an iSCSI connection to the iSCSI target ports.
NOTE: Since volumes are exported to the host for the first time after the iSCSI
driver has been loaded with the rc script, you must remove and reload the iscsi
driver to discover the volumes being exported.
5.7
3PAR Confidential
5.3 Configuring RedHat 5 for iSCSI
5.3.1 Installing iSCSI on RedHat 5
iSCSI is installed through the i scsi - i ni t i at or - ut i l s driver and rpm package by default
during the RedHat installation. There are a couple of ways to configure and start i scsi -
i ni t i at or - ut i l s on RedHat: either by using the various i scsi - i ni t i at or - ut i l s
available from the CLI or through the UI.
In this document, we will be referencing the i scsi - i ni t i at or - ut i l s from the CLI. The
i scsi admutility is a command-line tool that allows discovery and login to iSCSI targets. This
tool also provides access and management of the open- i scsi database. The following steps
are required to discover LUNs:
1 Discover targets at a given IP address.
2 Establish iSCSI login with node record id found in the discovery process.
3 Record iSCSI session statistics information.
5.3.2 Setting Up iSCSI for RedHat 5
You can adjust the iSCSI timers for better SAN performance. iSCSI timers and session
parameters are specified in / et c/ i scsi / i scsi d. conf file.
The r epl acement _t i meout iSCSI timeout parameter prevents I/O errors from propagating to
the application by controlling how long the iSCSI layer should wait for a timed-out path/
session to reestablish itself before failing any commands on it. The default
r epl acement _t i meout value is 120 seconds.
To adjust replacement_timeout, complete the following steps:
1 Open / et c/ i scsi / i scsi d. conf and edit the following line:
node. sessi on. t i meo. r epl acement _t i meout = [ r epl acement _t i meout ]
2 Set this parameter to 5 seconds for a faster failover.
node. sessi on. t i meo. r epl acement _t i meout = 5
3 Change the time interval to wait for a connection before sending a pi ng.
node.conn[0].timeo.noop_out_interval = [r epl acement _t i meout ]
This allows connections that were broken to be re-established before a pi ng is sent out (in
case of a bad cable or a broken Ethernet link).
5.8
3PAR Confidential
4 Set this parameter to 10 seconds for a ping to succeed after a connection is made.
node. conn[ 0] . t i meo. noop_out _i nt er val = 10
5 To set the host log into the iSCSI nodes every time the iSCSI daemon is started or the host is
rebooted, edit the iSCSI configuration in / et c/ i scsi / i scsi d. conf and change the
values of the following default settings:
node. st ar t up = aut omat i c
node. conn[ 0] . st ar t up = aut omat i c
6 Check the state of the iscsi service run level with the chkconf i g command:
i scsi 0: of f 1: of f 2: of f 3: of f 4: of f 5: of f 6: of f
i scsi d 0: of f 1: of f 2: of f 3: on 4: on 5: on 6: of f
7 Verify that Run l evel 5 is turned on. If not turned on, issue the following commands:
# chkconf i g i scsi on
i scsi 0: of f 1: of f 2: on 3: on 4: on 5: on 6: of f
i scsi d 0: of f 1: of f 2: of f 3: on 4: on 5: on 6: of f
8 To prevent one host from using up all the I/O buffers, you will need to throttle the host I/O
by reducing the node.session.cmds_max and node. sessi on. queue_dept h values.
The following values were determined based on the multihost-to-single InServ Storage Server
port testing completed at the 3PAR test labs and may require further tuning for your particular
configuration.
node. sessi on. cmds_max = 32
node. sessi on. queue_dept h = 8
9 As an option, you can also enable the Header and Data Digest for error handling and
recovery within the connection.
NOTE: The node. conn[ 0] .startup variable is optional and not defined in the
default i scsi d configuration file.
CAUTION: The InServ Storage Server target iSCSI port has a limited number of I/O
buffers and, with many iSCSI initiators connected to a given port, will result in
queuing that exceeds the I/O buffer capacity and could lead to network and I/O
timeout issues, as well as application failures.
5.9
3PAR Confidential
Typically, whenever a CRC error occurs, the SCSI layer tries to recover by disabling the
connection and recovering. However, by enabling the header and data digest, individual
iSCSI PDUs will be retried for recovery for those connections missing the data (CRC Error) or
missing a PDU or sequence number (Header Digest). If the recovery does not occur, then the
low level SCSI recovery will be initiated. The Header and Data Digest is optional since the
SCSI layer will still perform CRC error recovery at the session level rather than at the PDU
level.
You can enable the Header and Data Digest by adding the following lines in iSCSI
configuration file / et c/ i scsi / i scsi d. conf :
node. conn[ 0] . i scsi . Header Di gest = CRC32C
node. conn[ 0] . i scsi . Dat aDi gest = CRC32C
10 Enable any other configuration changes such as CHAP authentication. For details, see 5.4.1
Setting the Host CHAP Authentication on the InServ Storage Server on page 5.14.
5.3.3 Configuring RedHat 5 iSCSI Settings with Device Mapper Multipathing
listed in mul t i pat h. conf override the default kernel parameters for dm- mul t i pat h.
In general, the kernel defaults are sufficient with the exception of the devices entries for 3PAR.
CAUTION: Enabling Header and Data Digest will cause some I/O performance
degradation due to data checking.
NOTE: In order for the parameters changes to take effect, run the iSCSI service
after the change.
NOTE: Refer to RedHat documentation of DM Multipath Configuration and
Administration for additional options in mul t i pat h. conf entries.
5.10
3PAR Confidential
1 Run the mul t i pat h command for any name changes to be effective.
up.
The following output shows that it is enabled for run-level 3, 4 and 5. Enable it appropriately
for your configuration:
number(s) may not match those shown here.
Alternatively, you can use the chkconf i g command to enable multipathing if it is not
enabled:
devi ces {
devi ce {
vendor "3PARdat a"
pr oduct "VV"
f eat ur es "0"
r r _mi n_i o 100
pat h_checker t ur
}
}
# l s / et c/ r c3. d/ *mul t i *
# l s / et c/ r c5. d/ *mul t i *
5.11
3PAR Confidential
5.3.4 Starting the iSCSI Daemon for RedHat 5
To start the iSCSI daemon for the RedHat host, complete the following steps:
1 To start the open- i scsi module, issue the following command:
2 After open-iscsi is started, you can check the state of the open-iSCSI service run level
information with the chkconf i g command. Run level 5 should be on.
3 Verify that the i scsi module is loaded.
5.3.5 Creating the iSCSI Connection in RedHat 5 Using the iscsiadm Command
After connecting the host to the InServ iSCSI target port, use the i scsi admcommand to
create the iSCSI connection, complete following steps:
1 Discover the target node using the i scsi admcommand in discovery mode:
i scsi adm- mdi scover y - t sendt ar get s - p <t ar get i p addr ess>: <i scsi por t >.
i scsi d dead but pi d f i l e exi st s
Tur ni ng of f net wor k shut down. St ar t i ng i SCSI daemon: [ OK ]
[ OK ]
Set t i ng up i SCSI t ar get s: Logi n sessi on [ 10. 0. 0. 10: 3260 i qn. 2000-
05. com. 3par dat a: 20110002ac0001a6]
Logi n sessi on [ 10. 0. 0. 20: 3260 i qn. 2000- 05. com. 3par dat a: 21110002ac0001a6]
[ OK ]
i scsi 0: of f 1: of f 2: of f 3: of f 4: of f 5: on 6: of f
# l smod | gr ep i scsi
i scsi _t cp 56897 2
l i bi scsi 59329 2 i b_i ser , i scsi _t cp
scsi _t r anspor t _i scsi 63569 4 i b_i ser , i scsi _t cp, l i bi scsi
scsi _mod 184057 10
sg, i b_i ser , i scsi _t cp, l i bi scsi , scsi _t r anspor t _i scsi , ql a2xxx, l pf c, scsi _t r ansp
or t
_f c, cci ss, sd_mod
5.12
3PAR Confidential
For example:
2 The contents of the discovery can be viewed using the i scsi adm- mdi scover y
command.
For example:
3 Identify the iscsi node login that record has been discovered from the discovery process.
i scsi adm- mnode - T <t ar get name> - p <t ar get i p addr ess>: <i sci spor t > - l .
For example:
4 The content of the login node can be viewed using the i scsi admcommand.
For example:
5 Now examine the iscsi session and content session of the node info by issuing
i scsi adm- msessi on.
For example:
Refer to the Appendix for more RedHat Linux i scsi admcommand usage.
In RedHat 5.4, the open-iSCSI persistent configuration is implemented as a DBM database
available during the Linux iSCSI installation.
Discovery table (/ var / l i b/ i scsi / send_t ar get s)
# i scsi adm- mdi scover y - t sendt ar get s - p 10. 100. 0. 101: 3260
10. 100. 0. 101: 3260, 31 i qn. 2000- 05. com. 3par dat a: 20310002ac000079
# i scsi adm- mdi scover y
10. 100. 0. 101: 3260 vi a sendt ar get s
# i scsi adm- mnode - T i qn. 2000- 05. com. 3par dat a: 20310002ac000079 - p 10. 100. 0. 101: 3260 - l
Loggi ng i n t o [ i f ace: def aul t , t ar get : i qn. 2000- 05. com. 3par dat a: 20310002ac000079, por t al : 10. 100. 0. 101, 3260]
Logi n t o [ i f ace: def aul t , t ar get : i qn. 2000- 05. com. 3par dat a: 20310002ac000079, por t al : 10. 100. 0. 101, 3260] : successf ul
# i scsi adm- mnode - T i qn. 2000- 05. com. 3par dat a: 20310002ac000079 - p 10. 100. 0. 101: 3260
10. 100. 0. 101: 3260, 31 i qn. 2000- 05. com. 3par dat a: 20310002ac000079
# i scsi adm- msessi on
t cp: [ 1] 10. 100. 0. 101: 3260, 31 i qn. 2000- 05. com. 3par dat a: 20310002ac000079
5.13
3PAR Confidential
Node table (/ var / l i b/ i scsi / nodes)
The following example shows settings for send_targets and node tables:
send_t ar get s/
dr w- - - - - - - 2 r oot r oot 4096 Feb 26 16: 51 10. 102. 2. 131, 3260
dr w- - - - - - - 2 r oot r oot 4096 Feb 26 10: 22 10. 102. 2. 31, 3260
nodes/
dr w- - - - - - - 3 r oot r oot 4096 Feb 26 10: 22 i qn. 2000-
05. com. 3par dat a: 20310002ac0000b1
dr w- - - - - - - 3 r oot r oot 4096 Feb 26 10: 58 i qn. 2000-
05. com. 3par dat a: 21310002ac0000b1
To change or modify the send_targets or nodes, remove the above entry first to use the
i scsi admutility to add the new send_t ar get s or nodes, after which the persistent tables
will update
NOTE: RedHat5 iSCSI iface setup describes how to bind a session to a NIC port
using iSCSI software. Running i scsi adm- miface reports iface configurations
setup in /var/lib/iscsi/ifaces.
For more details please refer to RedHat5U4 open-iscsi release note.
5.14
Configuring CHAP for the iSCSI Host
3PAR Confidential
5.4 Configuring CHAP for the iSCSI Host
There are two CHAP authentication configurations available. Host CHAP authentication where
the InServ iSCSI target port authenticates the iSCSI Initiator host when it tries to connect to it
and bidirectional (mutual) CHAP authentication where both the iSCSI target and host
authenticate each other when the host it's trying to connect to the target.
You must set the Open iSCSI host definition before setting and configuring the CHAP for the
iSCSI host. See 2.2 Configuring the InServ Storage Server Running InForm OS 2.2.X on page 2.6.
5.4.1 Setting the Host CHAP Authentication on the InServ Storage Server
To set the host CHAP authentication an iSCSI host definition must have been created on the
InServ Storage Server and the CLI set host i ni t chap command must be used to set the host
CHAP secret.
For In Form OS 2.3.X, the output shows:
For In Form OS 2.2.X, the output shows:
The following example uses the host CHAP password host_secret0 for the host. Be aware that
CHAP secret must be at least 12 characters long.
Set the host CHAP secret.
# showhost
0 r edhat l i nux Gener i c i qn. 1994- 05. com. r edhat : a3df 53b0a32d - - -
# showhost
0 l i nux i qn. 1994- 05. com. r edhat : a3df 53b0a32d - -
i qn. 1994- 05. com. r edhat : a3df 53b0a32d
# set host i ni t chap - f host _secr et 0 r edhat l i nux
5.15
3PAR Confidential
Verify the host CHAP secret.
5.4.1.1 Setting the Host CHAP for RedHat 5
To set the host CHAP for RedHat 5, complete the following steps:
1 Go to the iSCSI Initiator host console, or, at a terminal, edit the / et c/ i scsi d. conf file
and enable CHAP authentication:
#To enabl e CHAP aut hent i cat i on set node. sessi on. aut h. aut hmet hod
#t o CHAP. The def aul t i s None.
node. sessi on. aut h. aut hmet hod = CHAP
2 Configure the host CHAP password for the discovery and login session by again editing the
configuration file / et c/ i scsi d. conf file.
#To set a di scover y sessi on CHAP user name and passwor d f or t he i ni t i at or
#aut hent i cat i on by t he t ar get ( s) , uncomment t he f ol l owi ng l i nes:
di scover y. sendt ar get s. aut h. user name = r edhat l i nux
di scover y. sendt ar get s. aut h. passwor d = host _secr et 0
#To set a CHAP user name and passwor d f or i ni t i at or
#aut hent i cat i on by t he t ar get ( s) , uncomment t he f ol l owi ng l i nes:
node. sessi on. aut h. user name = r edhat l i nux
node. sessi on. aut h. passwor d = host _secr et 0
3 Perform discovery and login as describe in 6.6 Discovering Devices with an iSCSI Connection
on page 6.14.
If the targets have been discovered previously, you must logout of the iSCSI sessions, delete the
node and send target records before performing discovery and logins by completing the
following steps:
a Perform an iSCSI Logout
# i scsi adm- mnode - - l ogout al l =al l
# showhost - chap
I d Name - I ni t i at or _CHAP_Name- - Tar get _CHAP_Name-
0 r edhat l i nux r edhat l i nux - -
NOTE: The Out goi ngUser name variable can be set to anything you want, but
the Out goi ngPasswor d has to be the same as the host CHAP secret configured
on the InServ Storage Server.
5.16
3PAR Confidential
b Remove the iSCSI Node
# i scsi adm- mnode - o del et e - T i qn. 2000- 05. com. 3par dat a: 20310002ac000079 - p
10. 100. 0. 101, 3260
c Remove the SendTarget iSCSI Discovery
# i scsi adm- mdi scover y - o del et e - p 10. 100. 0. 101
d Stop and start the iscsi daemon:
# / et c/ i ni t . d/ i scsi d st op
St oppi ng i SCSI daemon:
# / et c/ i ni t . d/ i scsi d st ar t
[ OK ]
e Repeat the steps as described in 5.3.5 Creating the iSCSI Connection in RedHat 5 Using
the iscsiadm Command on page 5.11 to rediscover the iSCSI target nodes and create the
iSCSI login sessions.
5.4.1.2 Setting the Host CHAP for RedHat 4
To set the host CHAP for RedHat 4, complete the following steps:
1 Go to the iSCSI Initiator host console, or, at a terminal, edit the /etc/iscsi.conf file and
configure the host CHAP password.
Out goi ngUser name=r edhat l i nux
Out goi ngPasswor d=host _secr et 0
NOTE: You must have the Out goi ngUser name and Out goi ngPasswor d
variables under the Di scover yAddr ess variable and they must be offset by a
space in order to apply only to this iSCSI target port.
NOTE: The Out goi ngUser name variable can be set to anything you want, but
the Out goi ngPasswor d has to be the same as the host CHAP secret configured
on the InServ Storage Server.
5.17
3PAR Confidential
2 Check to see if the i scsi d daemon is running by using the script /etc/init.d/iscsi status.
i scsi d ( pi d 30532 30529) i s r unni ng. . . ( RedHat 4)
Checki ng f or ser vi ce i SCSI dr i ver i s l oaded
5.4.2 Setting Up the Bidirectional CHAP on the InServ Storage Server
To set bidirectional CHAP (mutual), complete the following steps. The CLI set host i ni t chap
and set host t ar get chap commands must be used to set bidirectional CHAP on the InServ
Storage Server.
1 Verify that a host definition has been created on the InServ Storage Server. The following
example uses host _secr et 0 for the host CHAP password and t ar get _secr et 0 for the
target CHAP password.
For 2.3.X, the output shows:
For 2.2.X, the output shows:
2 Set the host CHAP secret.
# showhost
0 r edhat l i nux Gener i c i qn. 1994- 05. com. r edhat : a3df 53b0a32d - - -
# showhost
0 l i nux i qn. 1994- 05. com. r edhat : a3df 53b0a32d - -
i qn. 1994- 05. com. r edhat : a3df 53b0a32d - -
NOTE: The following example uses the host CHAP password host_secret0 for the
host. Be aware that CHAP secret must be at least 12 characters long.
# set host i ni t chap - f host _secr et 0 r edhat l i nux
5.18
3PAR Confidential
3 Set the target CHAP secret.
4 Verify the host and target CHAP secret.
5.4.2.1 Setting the Bidirectional CHAP for RedHat 5
To configure the bidirectional CHAP for RedHat 5, go to the iSCSI Initiator host console, or, at a
terminal, edit the / et c/ i scsi d. conf f i l e and configure the host and target CHAP
passwords for discovery and login sessions by completing the following steps
.
1 Perform the CHAP configuration settings for the host initiator:
# To enabl e CHAP aut hent i cat i on set node. sessi on. aut h. aut hmet hod
# t o CHAP. The def aul t i s None.
node. sessi on. aut h. aut hmet hod = CHAP
# To set a di scover y sessi on CHAP user name and passwor d f or t he i ni t i at or
# aut hent i cat i on by t he t ar get ( s) , uncomment t he f ol l owi ng l i nes:
di scover y. sendt ar get s. aut h. user name = r edhat l i nux
di scover y. sendt ar get s. aut h. passwor d = host _secr et 0
# To set a CHAP user name and passwor d f or i ni t i at or
# aut hent i cat i on by t he t ar get ( s) , uncomment t he f ol l owi ng l i nes:
node. sessi on. aut h. user name = r edhat l i nux
node. sessi on. aut h. passwor d = host _secr et 0
# set host t ar get chap - f t ar get _secr et 0 r edhat l i nux
# showhost - chap
I d Name - I ni t i at or _CHAP_Name- - Tar get _CHAP_Name-
0 r edhat l i nux r edhat l i nux S121
NOTE: Notice that two Di scover yAddr ess variables with the same IP address
for the InServ iSCSI target port are required. One for the host CHAP username and
password variables (Out goi ngUser name and Out goi ngPasswor d) and another
one for target CHAP username and password variables (I ncomi ngUser name and
I ncomi ngPasswor d).
5.19
3PAR Confidential
2 Perform the CHAP configuration setting for the target:
#To set a di scover y sessi on CHAP user name and passwor d f or t ar get ( s)
# aut hent i cat i on by t he i ni t i at or , uncomment t he f ol l owi ng l i nes:
di scover y. sendt ar get s. aut h. user name_i n = S121
di scover y. sendt ar get s. aut h. passwor d_i n = t ar get _secr et 0
# To set a CHAP user name and passwor d f or t ar get ( s)
# aut hent i cat i on by t he i ni t i at or , uncomment t he f ol l owi ng l i nes:
node. sessi on. aut h. user name_i n = S121
node. sessi on. aut h. passwor d_i n = t ar get _secr et 0
3 Perform discovery and login as describe in 6.6 Discovering Devices with an iSCSI Connection
on page 6.14.
If the targets have been discovered previously, you must logout of the iSCSI sessions, delete the
node and send target records before performing discovery and logins by completing the
following steps:
a Perform an iSCSI Logout
b Remove the iSCSI Node
# i scsi adm- mnode - o del et e - T i qn. 2000- 05. com. 3par dat a: 20310002ac000079 - p
10. 100. 0. 101, 3260
c Remove the SendTar get i SCSI Di scover y
d Stop and start the iscsi daemon:
# / et c/ i ni t . d/ i scsi d st op
e Stopping iSCSI daemon:
# / et c/ i ni t . d/ i scsi d st ar t
[ OK ]
NOTE: The Out goi ngUser name and I ncomi ngUser name variables can be set to
anything you want, but the Out goi ngPasswor d and I ncomi ngPasswor d must
match the host CHAP password and target CHAP password configured on the
InServ Storage Server.
5.20
3PAR Confidential
f Repeat the steps as described in 5.3.5 Creating the iSCSI Connection in RedHat 5 Using
the iscsiadm Command on page 5.11 to re-discover the iSCSI target nodes and create
the iSCSI login sessions.
5.4.2.2 Setting the Bidirectional CHAP for RedHat 4
To configure the bidirectional CHAP for RedHat 4, complete the following steps.
1 Go to the iSCSI Initiator host console, or, at a terminal, edit the /etc/iscsi.conf file and
configure the host and target CHAP passwords.
2 Start or restart the i scsi d daemon with the script / et c/ i ni t . d/ i scsi .
Out goi ngUser name=r edhat l i nux
Out goi ngPasswor d=host _secr et 0
I ncomi ngUser name=S4121
I ncomi ngPasswor d=t ar get _secr et 0
NOTE: Notice that two Di scover yAddr ess variables with the same IP address
for the InServ iSCSI target port are required: one for the host CHAP username and
password variables (Out goi ngUser name and Out goi ngPasswor d) and another
one for target CHAP username and password variables ( I ncomi ngUser name
and I ncomi ngPasswor d).
NOTE: You can choose the Out goi ngUser name and I ncomi ngUser name
variables, but the Out goi ngPasswor d and I ncomi ngPasswor d must match the
host CHAP password and target CHAP password configured on the InServ Storage
Server.
REQUIRED: The variables under the Di scover yAddr ess variable must be offset
with a space in order for the variables not to be global and to apply only to the
specific Di scover yAddr ess above them.
5.21
3PAR Confidential
3 Check to see if the i scsi d daemon is running by using the script / et c/ i ni t . d/ i scsi
st at us.
i scsi d ( pi d 30532 30529) i s r unni ng. . . ( RedHat 4)
Checki ng f or ser vi ce i SCSI i SCSI dr i ver i s l oaded
5.22
3PAR Confidential
6.1 Allocating Storage for Access by the RedHat Host
3PAR Confidential
6
Allocating Storage for
Access by the RedHat Host
In this chapter
6.1 Creating Storage on the InServ Storage Server 6.2
6.1.1 Creating Virtual Volumes 6.2
6.1.2 Creating Thin Provisioned Volumes 6.3
6.2 Exporting LUNs to the Host 6.3
6.3 Restrictions on Volume Size and Number 6.5
6.4 Discovering Devices with an Emulex HBA 6.6
6.4.2 Verifying Devices Found by the Host Using the Emulex HBA 6.7
6.5 Discovering Devices with a QLogic HBA 6.9
6.5.2 Verifying Devices Found by the Host Using the QLogic HBA 6.12
6.2
Creating Storage on the InServ Storage Server
3PAR Confidential
6.6 Discovering Devices with an iSCSI Connection 6.14
6.1 Creating Storage on the InServ Storage Server
This section describes the general steps and commands that are required to create the Virtual
Volumes that can then be exported for discovery by the RedHat host. For additional
information, see the CLI Administration Guide. For complete details on creating Virtual
Volumes, see the appropriate 3PAR documentation.
6.1.1 Creating Virtual Volumes
Virtual Volumes are the only data layer visible to hosts. After devising a plan for allocating
space for host servers on the InServ storage server, create the Virtual Volumes.
After devising a plan for allocating space for the RedHat host, you need to create the required
Virtual Volumes on the InServ Storage Server. You can create volumes that are provisioned
from one or more Common Provisioning Groups (CPGs). Volumes can be fully provisioned from
a CPG or can be thinly provisioned. You can optionally specify a CPG for snapshot space for
fully provisioned volumes.
Using the InForm Management Console:
1 From the Menu bar, select:
Actions > Provisioning>VV>Create Virtual Volume
2 Use the Create Virtual Volume wizard to create a base volume.
3 Select one of the following options from the Provisioning list:
Fully Provisioned from PDs
Fully Provisioned from CPG
Thinly Provisioned
Using the CLI:
To create a fully provisioned or thinly provisioned virtual volume, issue the following CLI
command:
cr eat evv [ opt i ons] <usr _CPG> <VV_name> [ . <i ndex>] <si ze>[ g| G| t | T]
6.3
Exporting LUNs to the Host
3PAR Confidential
Here is an example:
Consult the InForm Management Console help and the 3PAR Command Line Reference for
complete details on creating volumes for the InForm OS version that is being used on the
InServ Storage Server. Please note that the commands and options available for creating a
virtual volume may vary for earlier versions of the InForm OS.
6.1.2 Creating Thin Provisioned Volumes
To create Thin Provisioned volumes, see the following documents:
OS Concepts Guide
CLI Administration Guide
CLI reference Guide
6.2 Exporting LUNs to the Host
This section explains how to export LUNs to the host server as VVs, referred to as Virtual LUNs
(VLUNs).
To export VVs as Virtual LUNs, issue the following command:
cr eat evl un [ cnt ] <number of LUNs> <name_of _vi r t ual _LUNs> [ . i nt ]
<st ar t i ng_LUN_number > <host name/ host def i ni t i on>, where:
[ cnt ] Specifies the number of identical virtual volumes to create using an integer from 1
through 999. If not specified, one virtual volume is created.
<name_of _vi r t ual _LUNs>specifies name of the virtual volume being exported as a
virtual LUN.
<st ar t i ng_LUN_number > indicates that is the starting LUN number.
# cr eat evv - cnt 5 TESTLUNs 5G
NOTE: To create thinly provisioned virtual volumes, a 3PAR Thin Provisioning
license is required.
6.4
Exporting LUNs to the Host
3PAR Confidential
<host name/ host def i ni t i on>indicates that hostname is the name of the host created
in sections 2.1.1, 2.1.2, or 2.1.3.
Example:
To verify VLUNS have been created:
In the example above, there are 5 paths to the Inserv Storage Server from each HBA port.
# cr eat evl un cnt 5 TESTLUNs. 0 0 host name/ host def i ni t i on
# showvl un
Act i ve VLUNs
Lun VVName Host Name - - - - - - - - Host _WWN/ i SCSI _Name- - - - - - - - Por t
Type
0 TESTLUNS. 0 r edhat host 210100E08B31901B0: 4: 1 host
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
10 t ot al
VLUN Templ at es
Lun VVName Host Name - Host _WWN/ i SCSI _Name- Por t Type
0 TESTLUNS. 0 r edhat host - - - - - - - - - - - - - - - - - - - host
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
5 t ot al
NOTE: Exporting VLUNS to an iSCSI host is the same as exporting VLUNs to a host
that has a Fibre Channel connection to the Inserv Storage Server.
6.5
Restrictions on Volume Size and Number
3PAR Confidential
The showvl un output is different for hosts with iSCSI connection. For example:
6.3 Restrictions on Volume Size and Number
Follow the guidelines for creating virtual volumes (VVs) and Virtual LUNs (VLUNs) in the InForm
OS CLI Administrator's Manual while adhering to these cautions and guidelines:
This configuration supports sparse LUNs (meaning that LUNs may be skipped). LUNs may
also be exported in non-ascending order (e.g. 0, 5, 7, 3).
The InServ Storage Server supports the exportation of VLUNs with LUNs in the range from 0
to 65535. Testing was limited to 256 LUNs per storage server port (target) using LUN
numbers 0 to 255.
The maximum LUN size that can be exported to a RedHat host is 16TB when the installed
InForm OS version is 2.3.X. or later. A LUN size of 16TB on a RedHat host is dependent on
the installed RedHat version and update since some older versions of RedHat will not
support a volume greater than 2TB.
# showvl un
Act i ve VLUNs
Lun VVName Host Name - - - - - - - - Host _WWN/ i SCSI _Name- - - - - - - - Por t
Type
0 TESTLUNS. 0sqa- dl 145- 04 i qn. 1994- 05. com. r edhat : a3df 53b0a32d 0: 3: 1 host
2 TESTLUNS. 2 sqa- dl 145- 04 i qn. 1994- 05. com. r edhat : a3df 53b0a32d 0: 3: 1 host
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
10 t ot al
6.6
Discovering Devices with an Emulex HBA
3PAR Confidential
6.4 Discovering Devices with an Emulex HBA
There are two method of dynamically adding new LUNs. The method of echo statement and
the method of echo scsi add statement. 3PAR recommends that you use the echo
statement method where the scan is performed using the sys devi ce tree.
6.4.1 Scan Methods for LUN Discovery
You can use the following two methods to discover LUNs from the RedHat host.
Method 1Uses sysf s scan for scanning multiple devices at once
Method 2Uses addi ng si ngl e devi ces for adding single devices one at a time
6.4.1.1 Method 1 - sysfs Scan
After exporting VLUNS to the host using the cr eat evl un command in section 5.2, use the
echo statement on the sysf s file system to scan for devices:
# echo - <t ar get number > <l un number > > <devi ce scan pat h>
Example:
The device path is / sys/ cl ass/ scsi _host / host 2 and the target is 0 (t ar get 2: 0: 0) and
the exported device is LUN 1. The following is the echo command to be used.
# echo "- 0 1" > / sys/ cl ass/ scsi _host / host 2/ scan
The following message log provides and example of the resulting output:
Alternatively, you can scan for all LUNs and targets for a given l pf c instance using the
following command:
# echo "- - - " > / sys/ cl ass/ scsi _host / host 2/ scan
ker nel : Vendor : 3PARdat a Model : VV Rev: 0000
ker nel : Type: Di r ect - Access ANSI SCSI r evi si on: 03
ker nel : SCSI devi ce sdv: 524288 512- byt e hdwr sect or s ( 268 MB)
ker nel : SCSI devi ce sdv: dr i ve cache: wr i t e back
ker nel : sdv: unknown par t i t i on t abl e
ker nel : At t ached scsi di sk sdv at scsi 2, channel 0, i d 0, l un 1
ker nel : At t ached scsi gener i c sg22 at scsi 2, channel 0, i d 0, l un 1, t ype 0
scsi . agent [ 12915] : di sk at / devi ces/ pci 0000: 00/ 0000: 00: 02. 0/ 0000: 01: 00. 2/
0000: 03: 0b. 0/ 0000: 04: 04. 0/ host 2/ t ar get 2: 0: 0/ 2: 0: 0: 1
6.7
3PAR Confidential
If the device has changed its size, then issue the following command to obtain the new disk
size:
# echo 1 > / sys/ cl ass/ scsi _host / host 2/ devi ce/ r por t - 2: 0- 0/ t ar get 2: 0: 0/ 2: 0: 0: 1
Rescan
6.4.1.2 Method 2 - Adding Single Devices
To add LUNS by using the echo scsi add statement, run the following commands:
# echo "scsi add- si ngl e- devi ce 0 1 2 3" >/ pr oc/ scsi / scsi
(SCSI midlayer will re-scan)
Where 0 1 2 3 is replaced by your Host Channel ID.
# echo "scsi add- si ngl e- devi ce 2 0 0 1" > / pr oc/ scsi / scsi
You can see the new LUN presented to the OS by the SCSI Mid-Layer in the / var / l og/
messages file.
6.4.2 Verifying Devices Found by the Host Using the Emulex HBA
To verify that the RedHat host has discovered the exported devices, look at the contents of the
file / pr oc/ scsi / scsi . In this example, we have LUN 0 exported to the RedHat host through
8 paths (4 Inserv ports connecting to 2 Emulex HBA ports). This file should contain entries for
the attached devices:
ker nel : SCSI devi ce sdac: 524288 512- byt e hdwr sect or s ( 268 MB)
ker nel : SCSI devi ce sdac: dr i ve cache: wr i t e back
ker nel : sdac: unknown par t i t i on t abl e
ker nel : At t ached scsi di sk sdac at scsi 2, channel 0, i d 0, l un 14
Dec 12 14: 08: 50 sqa- del l 2850- 01 scsi . agent [ 14234] : di sk at / devi ces/
pci 0000: 00/
0000: 00: 02. 0/ 0000: 01: 00. 2/ 0000: 03: 0b. 0/ 0000: 04: 04. 0/ host 2/ t ar get 2: 0: 0/
2: 0: 0: 14
NOTE: The echo command needs to be executed on multiple host l pf c HBA SCSI
instances where LUNs have been exported.
6.8
3PAR Confidential
scsi 0 and scsi 1 refers to the HBA adapter instance (/ sys/ cl ass/ scsi _host / host 0 & /
sys/ cl ass/ scsi _host / host 1). The Id refers to the Inserv Storage Server target port (4
Inserv Storage Server target ports exist: Id 0,1, 2, 4).
# cat / pr oc/ scsi / scsi
At t ached devi ces:
Host : scsi 0 Channel : 00 I d: 00 Lun: 00
Vendor : 3PARdat a Model : VV Rev: 0000
Type: Di r ect - Access ANSI SCSI r evi si on: 05
6.9
Discovering Devices with a QLogic HBA
3PAR Confidential
6.5 Discovering Devices with a QLogic HBA
There are two methods of dynamically adding new LUNs. The method of echo statement
following the QLogic scan scsi - ql ascan and the method of echo scsi add statement.
3PAR recommends that you use the echo statement method where the scan is performed
using the sys devi ce t r ee.
6.5.1 Scan Methods for LUN Discovery
You can use the following methods to discover LUNs from the RedHat host.
Method 1Uses sysf s scan through the echo statement for adding multiple devices at
once
Method 2Uses add si ngl e devi ce for adding a single device at a time
6.5.1.1 Method 1 - sysfs Scan Using the echo Statement
After exporting VLUNS to the host using the cr eat evl un command in 6.2 Exporting LUNs to
the Host on page 6.3, use the QLogic scan scsi - ql ascan to discover devices by completing
the following steps:
1 Run the scsi - ql ascan script, by issuing the following command:
# echo "scsi - ql ascan" > / pr oc/ scsi / ql a2xxx/ <adapt er - i d>
In the following example, 0 is the HBA instance created by qla2xxx driver module.
2 Repeat for any other HBA instances created by the driver module. The QLogic scan will
allow the driver layer to discover the InServ Storage Server.
NOTE: If you are using the QLogic driver that is installed during the OS
installation, you can skip performing the scsi - ql ascan script and scan for
devices using the following echo command:
# echo " - <t ar get number > <LUN number >" > <devi ce scan pat h>
# echo "scsi - ql ascan" > / pr oc/ scsi / ql a2xxx/ 0
6.10
3PAR Confidential
Example:
3 Once the driver layer has discovered the device, run the echo statement for the RedHat OS
layer to discover the 3PAR devices:
# echo " - <t ar get number > <l un number >" > <devi ce scan pat h>
# cat / pr oc/ scsi / ql a2xxx/ 0
QLogi c PCI t o Fi br e Channel Host Adapt er f or QLA2462:
Fi r mwar e ver si on 4. 06. 03 [ I P] [ 84XX] , Dr i ver ver si on 8. 02. 23
BI OS ver si on 1. 29
FCODE ver si on 1. 27
EFI ver si on 1. 09
Fl ash FWver si on 4. 00. 30 0082
I SP: I SP2422, Ser i al # RFC0823R29292
Request Queue = 0x12a100000, Response Queue = 0x12a690000
Request Queue count = 4096, Response Queue count = 512
Tot al number of act i ve commands = 0
Tot al number of i nt er r upt s = 12368
Devi ce queue dept h = 0x20
Number of f r ee r equest ent r i es = 282
Number of mai l box t i meout s = 0
Number of I SP abor t s = 0
Number of l oop r esyncs = 0
Number of r et r i es f or empt y sl ot s = 0
Number of r eqs i n pendi ng_q= 0, r et r y_q= 0, done_q= 0, scsi _r et r y_q= 0
Host adapt er : l oop st at e = <READY>, f l ags = 0x5a43
Dpc f l ags = 0x0
MBX f l ags = 0x0
Li nk down Ti meout = 030
Por t down r et r y = 001
Logi n r et r y count = 008
Commands r et r i ed wi t h dr opped f r ame( s) = 0
Pr oduct I D = 0000 0000 0000 0000
SCSI Devi ce I nf or mat i on:
scsi - ql a0- adapt er - node=2000001b321a0c63;
scsi - ql a0- adapt er - por t =2100001b321a0c63;
scsi - ql a0- t ar get - 0=20410002ac000031;
FC Por t I nf or mat i on:
scsi - ql a0- por t - 0=2f f 70002ac000031: 20410002ac000031: 090800: 81;
scsi - ql a0- por t - 4=2f f 70002ac000031: 21510002ac000031: 6b0600: 84;
SCSI LUN I nf or mat i on:
( I d: Lun) * - i ndi cat es l un i s not r egi st er ed wi t h t he OS.
( 0: 0) : Tot al r eqs 156, Pendi ng r eqs 0, f l ags 0x0, Df l ags 0x0, 0: 0: 81 00
6.11
3PAR Confidential
If the device path is / sys/ cl ass/ scsi _host / host 2, the target is 0 (target 2:0:0), and the
exported device is LUN 1, then the echo statement would appear as the following example:
# echo "- 0 1" > / sys/ cl ass/ scsi _host / host 2/ scan
The following message log provides and example of the resulting output:
Alternatively, you can scan for all LUNs and targets for a given l pf c instance using the
following command:
# echo "- - - " > / sys/ cl ass/ scsi _host / host 2/ scan
If the device has changed its size, then issue the following command to obtain the new disk
size:
# echo 1 > / sys/ cl ass/ scsi _host / host 2/ devi ce/ r por t - 2: 0- 0/ t ar get 2: 0: 0/ 2: 0: 0: 1
Rescan
6.5.1.2 Method 2 - Scan using add si ngl e devi ce
The second method involves performing a QLogic driver scan scsi - ql ascan followed by
adding LUNs using the echo scsi add statement. To scan using add si ngl e devi ce
method, complete the following steps.
1 Issue scsi - ql ascan to discover devices:
# echo "scsi - ql ascan" > / pr oc/ scsi / ql a2xxx/ <adapt er - i d>
2 Once the new LUN is visible to the QLogic driver layer, force the SCSI mid-layer to do its
own scan and build the device table entry for the new device:
# echo "scsi add- si ngl e- devi ce 0 1 2 3" >/ pr oc/ scsi / scsi
ker nel : SCSI devi ce sdv: 524288 512- byt e hdwr sect or s ( 268 MB)
ker nel : SCSI devi ce sdv: dr i ve cache: wr i t e back
ker nel : sdv: unknown par t i t i on t abl e
ker nel : At t ached scsi di sk sdv at scsi 2, channel 0, i d 0, l un 1
scsi . agent [ 12915] : di sk at / devi ces/ pci 0000: 00/ 0000: 00: 02. 0/ 0000: 01: 00. 2/
0000: 03: 0b. 0/ 0000: 04: 04. 0/ host 2/ t ar get 2: 0: 0/ 2: 0: 0: 1
NOTE: If you are using the QLogic driver that is installed during the OS
installation, you can skip performing the scsi - ql ascan script and scan for
devices using the following echo command:
# echo " - <t ar get number > <LUN number >" > <devi ce scan pat h>
6.12
3PAR Confidential
The SCSI midlayer will re-scan, where 0 1 2 3 is replaced by your Host Channel ID.
Example:
# echo "scsi add- si ngl e- devi ce 4 0 0 1" > / pr oc/ scsi / scsi
You can see the new LUN presented to the OS by the SCSI mid-Layer in the / var / l og/
messages file.
6.5.2 Verifying Devices Found by the Host Using the QLogic HBA
To verify that the RedHat host has discovered the exported devices, look at the contents of the
file / pr oc/ scsi / scsi . In this example, we have LUN 0 exported to the RedHat host through
8 paths (4 Inserv ports connecting to 2 QLogic HBA ports). This file should contain entries for
the attached devices:
NOTE: You must run the scsi add- si ngl e- devi ce command individually for
all the newly discovered LUNs and on all host ports to which the LUNs were
exported.
ker nel : ql a2300 0000: 03: 08. 0: scsi ( 4: 0: 0: 1) : Enabl ed t agged queui ng, queue
dept h 32.
ker nel : SCSI devi ce sdh: 14680064 512- byt e hdwr sect or s ( 7516 MB)
ker nel : SCSI devi ce sdh: dr i ve cache: wr i t e back
ker nel : sdh: sdh1
ker nel : At t ached scsi di sk sdh at scsi 4, channel 0, i d 0, l un 1
ker nel : At t ached scsi gener i c sg8 at scsi 4, channel 0, i d 0, l un 2, t ype
0scsi . agent [ 1203] : di sk at / devi ces/ pci 0000: 03/ 0000: 03: 08. 0/ host 4/
t ar get 4: 0: 0/
4: 0
6.13
3PAR Confidential
scsi 0 and scsi 1 refers to the HBA adapter instance (/ pr oc/ scsi / ql a2xxx/ 0 & / pr oc/
scsi / ql a2xxx/ 1). The Id refers to the Inserv target port (there are 4 Inserv target ports: Id
0,1, 2, 4).
6.14
Discovering Devices with an iSCSI Connection
3PAR Confidential
6.6 Discovering Devices with an iSCSI Connection
The methods for discovering LUNs with an iSCSI connection differ between RedHat 4 and
RedHat 5.
6.6.1 Discovering Devices with RedHat 4
Complete the following steps to discover devices with an iSCSI connection on the RedHat 4
Host:
1 On the RedHat 4 iSCSI Initiator host, use the i scsi - r escan command to rescan for the
newly exported LUN.
# i scsi - r escan
Rescanni ng host 10
Rescanni ng host 11
2 Use the i scsi - l s command to display the scanned iSCSI devices.
3 Verify the contents of / pr oc/ scsi / scsi for the new device.
4 Check the block device files created in the system.
# l s / sys/ bl ock/ sd?
/ sys/ bl ock/ sda:
dev devi ce queue r ange r emovabl e sda1 si ze st at
5 Verify which block device files are 3PAR volumes.
# cat / sys/ bl ock/ sd*/ devi ce/ vendor
ATA
ATA
3PARdat a
6 You can verify an i SCSI device with the following command:
# i scsi - devi ce / dev/ sdc
/ dev/ sdc i s an i SCSI devi ce
6.15
3PAR Confidential
6.6.2 Discovering Devices with RedHat 5
Complete the following steps to discover devices with an iSCSI connection on the RedHat 5
Host:
1 Scan for new LUNs using the echo command - - - > / sys/ cl ass/ i scsi _host /
host <host _i nst ance>/ scan.
#echo "- - - " > / sys/ cl ass/ scsi _host / host 1/ scan
2 Verify that the iSCSI exported volumes have been discovered.
WARNING: The current RedHat 4 Update 5 iSCSI implementation does NOT
properly handle the mounting of file systems on iSCSI devices bootup time and
does NOT properly handle the unmounting of file systems on iSCSI devices while
shutting down and/or rebooting the host. A workaround is to have scripts to
mount the file systems after the host has booted and the proper devices have
been created and have scripts to un-mount them before shutting down or
rebooting the host.
NOTE: When VLUNs are exported, they will not appear on the host automatically.
After a new VLUN is exported from an InServ Storage Server iSCSI port, reload the
iSCSI daemon to rescan for new LUNs.
6.16
3PAR Confidential
3 Check the block device file created in the system.
# l s / sys/ bl ock/ sd?
/ sys/ bl ock/ sda:
dev devi ce hol der s queue r ange r emovabl e sda1 sda2 si ze sl aves st at
subsyst em uevent
# cat / sys/ bl ock/ sd*/ devi ce/ vendor
3PARdat a
7.1 Modifying 3PAR Devices on the Host Server
3PAR Confidential
7
Modifying 3PAR Devices on
the Host Server
In this chapter
7.1 Creating Device Mapper Devices 7.1
7.2 Displaying Detailed Device Mapper Node Information 7.5
7.3 Partitioning Device Mapper Nodes 7.6
7.4 Creating Veritas Volume Manager Devices 7.8
7.5 Removing a Storage Volume from the Host 7.9
7.1 Creating Device Mapper Devices
Complete the following steps to create device mapper devices.
1 Run the mul t i pat h command to create new Device Mapper nodes under the / dev/
mapper directory.
# mul t i pat h
2 Verify the Device Mapper devices have been created by issuing mul t i pat h - 11.
7.2
Creating Device Mapper Devices
3PAR Confidential
For example, in the following RedHat 4 host output, the / dev/ mapper /
350002ac0000c003e Device Mapper node is seen from device sda on driver instance
(0:0:0:0) and from sdb on driver instance (1:0:0:0).
3 After creating the devices, use the mul t i pat h - v 3 command to retrieve more detailed
information about Device Mapper nodes and their associated paths. Make sure of the
following settings:
The path checker is set to t ur for each of the devices.
The no_pat h_r et r y is set to a value of 12.
Output from the mul t i pat h - v 3 command differs in RedHat 4 and RedHat 5; however, the
information displayed on the Device Mapper remains the same.
# mul t i pat h - l l
350002ac001b40031
[ si ze=5 GB] [ f eat ur es="1 queue_i f _no_pat h"] [ hwhandl er ="0"]
\ _ r ound- r obi n 0 [ act i ve]
\ _ 0: 0: 0: 0 sda 8: 0 [ act i ve] [ r eady]
\ _ 1: 0: 0: 0 sdb 8: 16 [ act i ve] [ r eady]
7.3
3PAR Confidential
Example using RedHat 4:
# mul t i pat h - v 3
l oad pat h i dent i f i er s cache
#
# al l pat hs i n cache :
#
350002ac001b40031 0: 0: 0: 0 sda 8: 0 [ act i ve] 3PARdat a/ VV / 0000
350002ac001b40031 1: 0: 0: 0 sdb 8: 16 [ act i ve] 3PARdat a/ VV / 0000
===== pat h i nf o sda ( mask 0x1f ) =====
bus = 1
dev_t = 8: 0
si ze = 10485760
vendor = 3PARdat a
pr oduct = VV
r ev = 0000
h: b: t : l = 0: 0: 0: 0
t gt _node_name =
ser i al = 01B40031
pat h checker = t ur ( cont r ol er set t i ng)
st at e = 2
get pr i o = / bi n/ t r ue ( i nt er nal def aul t )
pr i o = 0
ui d = 350002ac001b40031 ( cache)
===== pat h i nf o sdb ( mask 0x1f ) =====
bus = 1
dev_t = 8: 16
si ze = 10485760
vendor = 3PARdat a
pr oduct = VV
r ev = 0000
h: b: t : l = 1: 0: 0: 0
t gt _node_name =
ser i al = 01B40031
path checker = tur (controler setting)
st at e = 2
get pr i o = / bi n/ t r ue ( i nt er nal def aul t )
pr i o = 0
ui d = 350002ac001b40031 ( cache)
#
# al l pat hs :
#
350002ac001b40031 0: 0: 0: 0 sda 8: 0 [ act i ve] [ r eady] 3PARdat a/ VV /
350002ac001b40031 1: 0: 0: 0 sdb 8: 16 [ act i ve] [ r eady] 3PARdat a/ VV
par ams = 1 queue_i f _no_pat h 0 1 1 r ound- r obi n 0 2 1 8: 0 100 8: 16 100
st at us = 2 0 0 0 1 1 A 0 2 0 8: 0 A 0 8: 16 A 0
===== pat h i nf o sda ( mask 0x8) =====
pr i o = 0
ui d = 350002ac001b40031 ( cache)
===== pat h i nf o sdb ( mask 0x8) =====
pr i o = 0
ui d = 350002ac001b40031 ( cache)
pgpol i cy = mul t i bus ( cont r ol er set t i ng)
sel ect or = r ound- r obi n 0 ( i nt er nal def aul t )
f eat ur es = 0 ( i nt er nal def aul t )
hwhandl er = 0 ( i nt er nal def aul t )
r r _wei ght = 2 ( conf i g f i l e def aul t )
r r _mi n_i o = 100 ( conf i g f i l e def aul t )
no_path_retry = 12 (controler setting)
pg_t i meout = NONE ( i nt er nal def aul t )
0 10485760 mul t i pat h 0 0 1 1 r ound- r obi n 0 2 1 8: 0 100 8: 16 100
set ACT_NOTHI NG: map unchanged
7.4
3PAR Confidential
Example using RedHat 5:
# mul t i pat h - v 3
sdc: not f ound i n pat hvec
sdc: mask = 0x1f
sdc: bus = 1
sdc: dev_t = 8: 32
sdc: si ze = 10485760
sdc: vendor = 3PARdat a
sdc: pr oduct = VV
sdc: r ev = 0000
sdc: h: b: t : l = 2: 0: 0: 0
sdc: ser i al = 004B0079
sdc: path checker = tur (controller setting)
sdc: st at e = 2
sdc: get pr i o = / bi n/ t r ue ( conf i g f i l e def aul t )
sdc: pr i o = 0
sdc: get ui d = / sbi n/ scsi _i d - g - u - s / bl ock/ %n ( conf i g f i l e def aul t )
sdc: ui d = 350002ac0004b0079 ( cal l out )
sdd: not f ound i n pat hvec
sdd: mask = 0x1f
sdd: bus = 1
sdd: dev_t = 8: 48
sdd: si ze = 10485760
sdd: vendor = 3PARdat a
sdd: pr oduct = VV
sdd: r ev = 0000
sdd: h: b: t : l = 3: 0: 0: 0
sdd: ser i al = 004B0079
sdd: path checker = tur (controller setting)
sdd: st at e = 2
sdd: get pr i o = / bi n/ t r ue ( conf i g f i l e def aul t )
sdd: pr i o = 0
sdd: get ui d = / sbi n/ scsi _i d - g - u - s / bl ock/ %n ( conf i g f i l e def aul t )
sdd: ui d = 350002ac0004b0079 ( cal l out )
===== pat hs l i st =====
uui d hci l dev dev_t pr i dm_st chk_st vend/ pr od/ r
350002ac0004b0079 2: 0: 0: 0 sdc 8: 32 0 [ undef ] [ r eady] 3PARdat a, VV
350002ac0004b0079 3: 0: 0: 0 sdd 8: 48 0 [ undef ] [ r eady] 3PARdat a, VV
par ams = 1 queue_i f _no_pat h 0 1 1 r ound- r obi n 0 2 1 8: 32 100 8: 48 100
st at us = 2 0 0 0 1 1 E 0 2 0 8: 32 A 0 8: 48 A 0
sdc: mask = 0x8
sdc: pr i o = 0
sdd: mask = 0x8
sdd: pr i o = 0
sdc: owner shi p set t o 350002ac0004b0079
sdc: not f ound i n pat hvec
sdc: mask = 0xc
sdc: st at e = 2
sdc: pr i o = 0
sdd: owner shi p set t o 350002ac0004b0079
sdd: not f ound i n pat hvec
sdd: mask = 0xc
sdd: st at e = 2
sdd: pr i o = 0
350002ac0004b0079: pgf ai l back = - 2 ( conf i g f i l e def aul t )
350002ac0004b0079: pgpol i cy = mul t i bus ( cont r ol l er set t i ng)
350002ac0004b0079: sel ect or = r ound- r obi n 0 ( i nt er nal def aul t )
350002ac0004b0079: f eat ur es = 0 ( i nt er nal def aul t )
350002ac0004b0079: hwhandl er = 0 ( i nt er nal def aul t )
350002ac0004b0079: r r _wei ght = 2 ( conf i g f i l e def aul t )
350002ac0004b0079: mi ni o = 100 ( conf i g f i l e def aul t )
350002ac0004b0079: no_path_retry = 60 (controller setting)
pg_t i meout = NONE ( i nt er nal def aul t )
350002ac0004b0079: set ACT_NOTHI NG ( map unchanged)
7.5
Displaying Detailed Device Mapper Node Information
3PAR Confidential
7.2 Displaying Detailed Device Mapper Node Information
Use mul t i pat h - l for listing of devices and dmset up to get detailed dm node information.
Example:
The dmset up command can be used with various options to get more information on device
mapper mappings.
NOTE: With no_pat h_r et r y set to a value other than 0 in the / et c/
mul t i pat h. conf file, I/O will be queued for the period of the retries and
f eat ur es=1 queue_i f _no_pat h will be shown in mul t i pat h - l command
output.
# mul t i pat h - l
350002ac001b40031
\ _ 0: 0: 0: 0 sda 8: 0 [ act i ve]
\ _ 1: 0: 0: 0 sdb 8: 16 [ act i ve]
NOTE: If you see the device status as [ undef ] in the output, this is a RedHat
defect that has been raised with RedHat to be fixed.
Instead, use mul t i pat h - l l command, which shows the correct device status as
ready.
7.6
Partitioning Device Mapper Nodes
3PAR Confidential
Example:
7.3 Partitioning Device Mapper Nodes
The following section provides guidelines for partitioning Device Mapper nodes.
When partitioning a Device Mapper node, do not use f di sk on the / dev/ mapper / XXX
nodes. The following error output may be seen as a result of using f di sk.
# dmset up t abl e
350002ac001b40031: 0 10485760 mul t i pat h 1 queue_i f _no_pat h 0 1 1 r ound- r obi n
0 2 1 8: 0 100 8: 16 100
# dmset up l s - - t ar get mul t i pat h
350002ac0004b0079 ( 253, 7)
# dmset up i nf o 350002ac0004b0079
Name: 350002ac0004b0079
St at e: ACTI VE
Read Ahead: 256
Tabl es pr esent : LI VE
Open count : 1
Event number : 0
Maj or , mi nor : 253, 7
Number of t ar get s: 1
UUI D: mpat h- 350002ac0004b0079
# dmset up t abl e - - t ar get mul t i pat h
350002ac0004b0079: 0 10485760 mul t i pat h 1 queue_i f _no_pat h 0 1 1 r ound- r obi n
0 2 1 8: 32 100 8: 48 100
WARNING: Rereading the partition table failed with error 22: Invalid
argument.The kernel still uses the old table. The new table will be used at the
next reboot. Syncing disks
NOTE: Do not use / dev/ mapper / XXX path on f di sk command to create
partitions.
7.7
Partitioning Device Mapper Nodes
3PAR Confidential
Use f di sk on the underlying disks / dev/ sdXX and execute the following command when
Device Mapper multipath maps the device to create a / dev/ mapper / <devi ce node>
partition.
Device Mapper node 350002ac001b40031 is formed from underlying devices sda and sdb
representing two paths from the same storage volume.
# f di sk / dev/ sda - - cr eat e a par t i t i on.
After the f di sk command completes, use the kpar t x command to list and create DM devices
for the partitions on the device.
# kpar t x / dev/ mapper / 350002ac001b40031
350002ac001b40031p1 : 0 10477194 / dev/ mapper / 350002ac001b40031 62
# kpar t x - a / dev/ mapper / 350002ac001b40031 - - wi l l add a par t i t i on mappi ng
# l s / dev/ mapper
350002ac001b40031 350002ac001b40031p1
where 350002ac001b40031p1 is a partition device of whole disk 350002ac001b40031.
# mul t i pat h - l
50002ac001b40031
\ _ 0: 0: 0: 0 sda 8: 0 [ act i ve]
\ _ 1: 0: 0: 0 sdb 8: 16 [ act i ve]
WARNING: While using f di sk, make sure the correct underlying device is used.
Use the mul t i pat h command to identify the underlying device.
WARNING: All I/O creating the file system and mount points needs to be done
using the Device Mapper device nodes / dev/ mapper / XXX.
Data corruption will occur if any I/O is attempted on / dev/ sdX device nodes.
WARNING: Issuing the mul t i pat h - F command will flush out all the Device
Mapper mapping and can be very destructive if I/O is being served to any of the
existing devices. Avoid using the - F option.
7.8
Creating Veritas Volume Manager Devices
3PAR Confidential
Use kpar t x to delete a device mapper instance and then use f di sk to delete the partition.
The Device Mapper node name represents the storage volume ID (excluding the first digit 3).
Use the InForm CLI showvv or showvl un commands to get the volume name it represents.
Example:
1 On your FC connected host run l s / dev/ mapper.
2 Run the showvl un command on the InServ Storage Server using the output above (minus
first digit).
3 On the iSCSI host, run l s / dev/ mapper.
4 On the iSCSI host, run showvl un - l vw - a | gr ep - i <LUN>.
7.4 Creating Veritas Volume Manager Devices
If the VERITAS Volume Manager is being used for multipathing, and new VLUNs are exported
from the storage server, complete the following steps to add new VLUNs without disrupting
the I/O on the existing VLUNs:
1 Add the new exported VLUN using any of the Discovery methods.
2 After verifying that the new VLUN is detected and the device instance is created, force the
VERITAS layer to scan for new devices.
# ls /dev/mapper
350002ac001b40031
# showvlun -lvw -a |grep -i 50002ac001b40031
0 t est vl un 50002AC001B40031 r edhat host 2100001B321A0C63 0: 4: 1 host
0 t est vl un 50002AC001B40031 r edhat host 2101001B323A0C63 1: 5: 1 host
# ls /dev/mapper
350002AC0004B0079
# showvlun -lvw -a |grep -i 50002ac0004b0079
0 t est vl un 50002AC0004B0079 r edhat host i qn. 1994- 05. com. r edhat : a3df 53b0a32d 1: 3: 1 host
0 t est vl un 50002AC0004B0079 r edhat host i qn. 1994- 05. com. r edhat : a3df 53b0a32d 0: 3: 1 host
7.9
Removing a Storage Volume from the Host
3PAR Confidential
# vxdct l enabl e
3 Check that the new devices are seen by the VERITAS Volume Manager.
The VLUNs discovered on the Linux host should be labeled using the Linux f di sk command
before they can be used by the Volume Manager.
If disks are admitted to the Volume Manager, never use the raw device paths / dev/ sdX for
performing I/O, and instead use VERITAS volume device paths / dev/ vx/ .
7.5 Removing a Storage Volume from the Host
Use one of the two following methods to remove a storage volume from the host if using
device mapper.
Method 1
Issue the following commands:
# kpar t x - d / dev/ mapper / <devi ce node>
# dmset up r emove <devi ce node>
#echo "1" >/ sys/ cl ass/ scsi _host / <host i nst ance>/ devi ce/ <r por t - host i nst ance>/
<t ar get i nst ance>/ <devi ce i nst ance>/ del et e
Example:
# vxdisk list
DEVI CE TYPE DI SK GROUP STATUS
3PARDATA1_0 aut o: cdsdi sk t est dg t est dg onl i ne
3PARDATA1_1 aut o - - er r or
# kpar t x - d / dev/ mapper / 350002ac001b40031
# dmset up r emove 350002ac001b40031
# echo "1" > / sys/ cl ass/ scsi _host / host 0/ devi ce/ t ar get 0\ : 0\ : 0/ 0\ : 0\ : 0\ : 0/ /
del et e
<t ar get i nst ance>/ <devi ce i nst ance>/ del et e
NOTE: When using the echo command, make sure the devices are removed from
each of the host HBA instances.
7.10
3PAR Confidential
Method 2
Issue the following commands:
#kpar t x - d / dev/ mapper / <devi ce node>
#dmset up r emove <devi ce node>
#echo "scsi r emove- si ngl e- devi ce X X X X " > / pr oc/ scsi / scsi
Example:
After removing the storage volume from the host using one of the two methods, remove the
VLUN from the Inserv Storage Server by issuing r emovevl un <VVname> <LUN> <host >.
# kpar t x - d / dev/ mapper / 350002ac001b40031
# dmset up r emove 350002ac001b40031
# echo "scsi r emove- si ngl e- devi ce 0 0 0 0" > / pr oc/ scsi / scsi
NOTE: When using the echo command, make sure the devices are removed from
each of the host HBA instances.
# r emovevl un t est vl un 0 r edhat host
WARNING: While removing the device, make sure the correct underlying device
is used. Use the mul t i pat h command to identify the underlying device.
NOTE: The removed SCSI device is updated in / pr oc/ scsi / scsi , / pr oc/
par t i t i ons and in / sys/ devi ce pat h.
CAUTION: For iSCSI devices, do not remove the last iSCSI device in /pr oc/ scsi /
scsi without first stopping the iSCSI daemon / et c/ i ni t . d/ i scsi stop,
otherwise data corruption can occur and the host will hang.
7.11
3PAR Confidential
Any change to the / et c/ mul t i pat h configuration file requires running the mul t i pat h
command to be effective. If the change is not reflected, try stopping and starting the
mul t i pat hd script.
# / et c/ i ni t . d/ mul t i pat hd st op
# / et c/ i ni t . d/ mul t i pat hs st ar t
7.12
3PAR Confidential
8.1 Booting the Host from the InServ Storage Server
3PAR Confidential
8
Booting the Host from the
InServ Storage Server
In this chapter
8.1 Inserv Storage Server Setup Requirements 8.1
8.2 RedHat Host HBA BIOS Setup Considerations 8.2
8.2.1 Booting from the InServ Storage Server Using QLogic HBAs 8.2
8.2.2 Booting from the InServ Storage Server Using Emulex HBAs 8.3
8.3 Installation from RedHat 5 Linux CDs 8.4
8.4 Modifying the /etc/multipath.conf file 8.5
8.5 Changing the Emulex HBA Inbox Driver Parameters 8.6
8.6 Installing the New QLogic Driver 8.6
8.1 Inserv Storage Server Setup Requirements
Booting from the InServ Storage Server is supported in fabric and direct connect modes.
You can connect each of the HBA ports to an Inserv Storage Server port. During the RedHat
installation process, you will specify the correct argument that will take into account
multipathing during the installation process.
Make sure you have allocated enough space when creating your virtual volumes to be able to
install your RedHat 5 OS on it.
8.2
RedHat Host HBA BIOS Setup Considerations
3PAR Confidential
After creating your first virtual volume, you must export it to your RedHat host as VLUN 0,
since RedHat requires the root/boot volume LUN to be 0 when booting from the SAN.
8.2 RedHat Host HBA BIOS Setup Considerations
The HBA BIOS needs to be set up properly to handle booting from the Inserv Storage Server.
8.2.1 Booting from the InServ Storage Server Using QLogic HBAs
When booting from the InServ Storage Server using a QLogic HBA, complete the following
steps:
1 During the host boot, press CTL-Q or ALT-Q when prompted for the QLogic Fast!UTIL HBA
utility.
2 From the QLogic Fast!UTIL screen, choose Select Host Adapter menu and select the host
adapter from which you want to boot.
3 When the Fast!UTIL Options menu appears, select Configuration Settings.
4 Select Adapter Settings.
5 Select Host Adapter BIOS>Enabled and press ESC.
6 From the Configuration Settings menu, select Selectable Boot Settings.
7 From the Selectable Boot Settings menu, select Selectable Boot>Enabled.
8 Arrow down to the next field, (Primary) Boot Port Name, LUN, and press ENTER.
9 From the Select Fibre Channel Device menu, you should see the 3PAR device under
ID0 with its Rev, Port Name, and Port ID shown. Press ENTER.
10 From the Select LUN menu, select the first line LUN 0 with a status of Supported, and
press ENTER.
11 Press ESC twice to return to the Configuration Settings Modified dialogue box.
12 Select Save changes.
13 Return to the Fast!UTIL Options menu and select Select host Adapter.
14 Select your next HBA port to boot from and repeat these steps.
15 When done, from the Fast!UTIL Options menu:
8.3
RedHat Host HBA BIOS Setup Considerations
3PAR Confidential
a Select Exit Fast!UTIL
b Select Reboot system
The settings will be saved and the host is rebooted.
8.2.2 Booting from the InServ Storage Server Using Emulex HBAs
When booting from the InServ Storage Server using an Emulex HBA, complete the following
steps:
1 During the host boot, press ALT-E or CTL-E when prompted by the Emulex HBA Utility, a
screen appears that will show the Emulex adapters in the system. Select <Adapter #> and
press ENTER.
a After that, if a screen is being displayed that says: The BIOS of the Adapter is
Disabled. If the screen says, The BIOS of the Adapter is Enabled, then skip to
step 2.
b Select option 2 Configure This Adapter's Parameters and press ENTER.
c From the next screen Select option 1 Enable or Disable BIOS and press ENTER. The
following message appears:
The BI OS i s Di sabl ed! !
d Enable Press 1, Disable Press 2:
e Select 1 and press ENTER. The following message appears:
The BI OS i s Enabl ed! !
f Press ESC twice.
2 Select option 1 Configure Boot Devices press ENTER. The following list appears:
List of Saved Boot Devices
3 Select option 1. Unused DID:<all zeros> WWPN: <all zeros> LUN:00 Primary Boot and
press ENTER. The following dialog box appears:
01. DID:<did_value> WWPN:<3PAR Port WWPN) Lun:00 3PARdataVV 0000
4 Select the two digit number of the Desired Boot Device 01 and press ENTER. The following
dialogue box appears:
Enter two digits of starting LUN (Hex):
5 Type 00 and press ENTER.
8.4
Installation from RedHat 5 Linux CDs
3PAR Confidential
The following dialog box appears:
DID: XXXXXX WWPN: <3PAR port WWPN>
01. Lun:00 3PARdataVV 0000
a Select 01 and press ENTER.
Another Dialogue box will appear:
1. Boot this device via WWPN
2. Boot this device via DID
b Select 1 and press ENTER. The following screen appears:
List of saved boot devices
1 Used DID:000000 WWPN:<3PAR Port WWPN> Lun:00 Primary Boot
6 Press ESC twice to return to the Emulex Adapters in the System menu.
7 Select the next HBA port to boot from and repeat these steps.
8 When done, press x to exit.
9 You will be prompted to reboot the system. Select Y.
After the system comes up, make sure the RedHat installation CD is in the drive tray to
continue with the next steps.
8.3 Installation from RedHat 5 Linux CDs
Use the following procedure to install from the RedHat 5 Linux CDs.

1 When prompted by the install CDs after the host comes up, at the boot prompt, type the
following command:
boot : l i nux mpat h
This command communicates that multiple paths are connected from the storage to the host.
2 Respond to all the prompts during the install process by selecting the default settings.
REQUIRED: To ensure the root or boot disk is protected by multipath, the
multipath option must be enabled at the beginning of the RedHat 5 installation.
8.5
Modifying the /etc/multipath.conf file
3PAR Confidential
When the installation completes, the host is rebooted.
8.4 Modifying the /etc/multipath.conf file
During a RedHat SAN boot install using the mpat h option, the / et c/ mul t i pat h. conf file is
automatically edited by the install processes. As part of the / et c/ mul t i pat h. conf edits
performed during install, the global multipath option user _f r i endl y_names enabled.
After initial boot up, the / et c/ mul t i pat h. conf file will look similar to this
(350002ac001b90031 is the scsi _i d of the boot device in this example):
Note that using the user _f r i endl y_names option can be problematic in the following
situations:
If the system root device is using multipath and you use the user _f r i endl y_names option,
the user-friendly settings in the / var / l i b/ mul t i pat h/ bi ndi ngs file are included in the
i ni t r d. If you later change the storage setup, such as by adding or removing devices, there is
a mismatch between the bindings setting inside the i ni t r d and the bindings settings in
/ var / l i b/ mul t i pat h/ bi ndi ngs.
def aul t s {
user _f r i endl y_names yes
}
bl ackl i st {
devnode "^( r am| r aw| l oop| f d| md| dm- | sr | scd| st ) [ 0- 9] *"
devnode "^( hd| xvd| vd) [ a- z] *"
wwi d "*"
}
# Make sur e our mul t i pat h devi ces ar e enabl ed.
bl ackl i st _except i ons {
wwi d "350002ac001b90031"
}
CAUTION: A bindings mismatch between i ni t r d and / var / l i b/ mul t i pat h/
bi ndi ngs can lead to a wrong assignment of mount points to devices, which can
result in file system corruption and data loss.
8.6
3PAR Confidential
Use the alias option to override the user _f r i endl y_names option for the system root device
in the / et c/ mul t i pat h. conf file.
Verify that the SAN boot disk created is / dev/ sda.
Identify the scsi _i d of the boot disk via:
This identified that 350002ac001b90031 is the WWID of the boot disk in the above example.
Establish an alias name of mpat h0 for the WWID of the boot disk using multipaths entries in /
et c/ mul t i pat h. conf .
# f di sk - l - u / dev/ sda
Di sk / dev/ sda: 32. 2 GB, 32212254720 byt es
255 heads, 63 sect or s/ t r ack, 3916 cyl i nder s, t ot al 62914560 sect or s
Uni t s = sect or s of 1 * 512 = 512 byt es
Devi ce Boot St ar t End Bl ocks I d Syst em
/ dev/ sda1 * 63 208844 104391 83 Li nux
/ dev/ sda2 208845 62910539 31350847+ 8e Li nux LVM
# scsi _i d - g - u - s / bl ock/ sda
350002ac001b90031
8.7
3PAR Confidential
The contents of / et c/ mul t i pat h. conf file should be edited to look like this:
After the modifications to the / et c/ mul t i pat h. conf , restart the multipath daemon or
reboot the host so the changes take effect.
# chkconf i g mul t i pat hd of f
You should find the SAN boot LUN mapped as mpat h0. In the following example, with a SAN
boot LUN ID of 350002ac001b90031:
def aul t s {
user _f r i endl y_names yes
}
mul t i pat hs {
mul t i pat h {
wwi d 350002ac001b90031
al i as mpat h0
}
}
devi ces {
devi ce {
vendor "3PARdat a"
pr oduct "VV"
f eat ur es "0"
r r _mi n_i o 100
pat h_checker t ur
}
}
8.8
Changing the Emulex HBA Inbox Driver Parameters
3PAR Confidential
8.5 Changing the Emulex HBA Inbox Driver Parameters
Refer to 4.2.2 Modifying the etc/modprobe.conf file and Building the Ramdisk on page 4.4 of
this document for changing the Emulex HBA parameters and rebuilding the ramdisk.
8.6 Installing the New QLogic Driver
To install a new QLogic driver to replace the Linux in-box driver, complete the following steps:
1 Go to http://www.qlogic.com and download the driver package ql af c- l i nux-
<ver si on>- i nst al l . t gz to the RedHat host.
2 To extract the files, run the following command:
# t ar xvzf ql af c- l i nux- <ver si on>- i nst al l . t gz
3 From the directory where the file was extracted, change to the driver directory:
# cd ql af c- l i nux- <ver si on>- i nst al l
# l s / dev/ mapper
cont r ol mpath0 mpath0p1 mpath0p2 Vol Gr oup00- LogVol 00 Vol Gr oup00-
LogVol 01
# mul t i pat h - l l
mpat h0 ( 350002ac001b90031) dm- 0 3PARdat a, VV
[ si ze=20G] [ f eat ur es=1 queue_i f _no_pat h] [ hwhandl er =0] [ r w]
\ _ r ound- r obi n 0 [ pr i o=2] [ act i ve]
\ _ 0: 0: 0: 0 sda 8: 0 [ act i ve] [ r eady]
\ _ 1: 0: 0: 0 sdb 8: 16 [ act i ve] [ r eady]
# df
Fi l esyst em 1K- bl ocks Used Avai l abl e Use%Mount ed on
/ dev/ mapper / Vol Gr oup00- LogVol 00
14283576 3632732 9913564 27%/
/ dev/ mapper / mpat h0p1 101086 16002 79865 17%/ boot
t mpf s 2023232 0 2023232 0%/ dev/ shm
8.9
Installing the New QLogic Driver
3PAR Confidential
4 Run ql i nst al l - - upgr ade to install the new driver:
# . / ql i nst al l - - upgr ade
5 After the installation completes, reboot the RedHat host. When the host comes back up,
check the driver version:
# modi nf o ql a2xxx | gr ep ver si on
ver si on: xx. yy. zz
To change the ql por t _down_r et r y parameter, issue the following command:
# scl i
After the main menu comes up, select 3: HBA Parameters.
6 Then from the HBA Parameters Menu, select the HBA that you want to change.
7 Then from the HBA Parameters Menu, select 2: Configure HBA Parameters.
8 From the Configure Parameters Menu, select 13: Port Down Retry Count.
9 Then enter the value 1 for the Down Retry Count:
10 Enter Port Down Retry Count [0-255] [30]: 1
11 From the Configure Parameters Menu, select 19: Commit Changes.
12 From the HBA Parameters Menu, select 4: Return to Previous Menu.
REQUIRED: You must use the - - upgr ade requirement for a successful driver
installation.
NOTE: You can also use the -up option with the ql i nst al l command. Make sure
the - up or the - upgr ade option is used when installing the new driver. Upgrade
builds and installs the QLOgic HBA driver, installs the SNIA HBA API library, creates
ramdisk to load driver at boot time, does not load and unload current drivers and
does not do persistent binding.
NOTE: Modify the HBA parameter to set ql por t _down_r et r y to 1. However, do
NOT use the procedure to rebuild the ramdisk as described in 4.3.1 Building the
QLogic Driver on page 4.7. Instead, use the scl i utility which was installed
during the driver install process to change this HBA parameter value.
8.10
Installing the New QLogic Driver
3PAR Confidential
13 From the HBA Parameters Menu you can select the next HBA port to repeat the steps to
modify its parameter.
14 After finishing, exit the scl i utility.
15 To ensure that change has taken effect, run the following command:
# cat / pr oc/ scsi / ql a2xxx/ <hba i nst ance> | gr ep "Por t down r et r y"
Por t down r et r y = 001
WARNING: Modifying the / et c/ modpr obe. conf file to make the QLogic HBA
ql por t _down_r et r y parameter change and using the mki ni t r d command to
rebuild the ramdisk as described in section 3.3.4 will cause the driver not to load
and boot properly after rebooting the host.
9.1 Using Veritas Cluster Servers
3PAR Confidential
9
Using Veritas Cluster Servers
3PAR supports use with Veritas Cluster Server.
There are no special setup considerations for the InServ Storage Server.
Please refer to Veritas Cluster Server Installation Guide and Veritas Cluster Server User's Guide
at http://www.symantec.com on installation and setup instructions.
CAUTION: Make sure device mapper is disabled if using the Veritas DMP for
multipathing software.
9.2
3PAR Confidential
10.1 Using RedHat Xen Virtualization
3PAR Confidential
10
Using RedHat Xen
Virtualization
3PAR supports the use of RedHat Enterprise Linux 5 Xen Virtualization.
There are no special setup considerations for the InServ Storage Server.
Please refer to RedHat Enterprise Linux 5 Xen Virtualization Guide at http://www.redhat.com
for installation and setup instructions.
10.2
3PAR Confidential
11.1 Using RedHat Cluster Services
3PAR Confidential
11
Using RedHat Cluster
Services
3PAR supports RedHat Cluster services for RedHat 4 and RedHat 5.
For installation and administration RedHat Cluster services, refer to the RedHat Linux
Installation Guide and Configuring and Managing a RedHat Cluster at http://www.redhat.com.
There are no special considerations for the InServ Storage Server outside of the standard setup
procedures described in this Implementation Guide.
11.2
3PAR Confidential
12.1 Using RedHat KVM Hypervisor
3PAR Confidential
12
Using RedHat KVM
Hypervisor
RedHat Enterprise Linux 5 update 4 includes full support for the Kernel-based Virtual Machine
(KVM) hypervisor. The KVM hypervisor requires a processor with the Intel-VT or AMD-V
virtualization extensions. The Red Hat Enterprise Linux KVM package is limited to 64-processor
cores. A guest OS can only be used on the hypervisor type that they were created on.
For installation, administration, and OS support by the RedHat KVM, refer to Red Hat
Enterprise Linux 5 virtualization Guide at https://www.redhat.com/virtualization/rhev/server/.
The following fully-virtualized guests OS versions are tested:
RHEL5U4
SuSE11
Windows 2003 R2 SP2
Windows 2008 SP2
Windows 2008 R2
Currently, a code-signing driver does not exist for Windows 2008 R2.
In order to running test on Windows 2008, disable the driver signature enforcement first to
update the RedHat virtual IO SCSI controller.
There are no special considerations for the InServ storage Server outside of the standard setup
procedures described in this implementation guide.
12.2
3PAR Confidential
A.1 RedHat iscsiadm Utility Usage
3PAR Confidential
A
RedHat iscsiadm Utility
Usage
This appendix provides examples of a few commands using the i scsi admutility to set up the
iSCSI sessions:
Discover targets using SendTar get s i SCSI Di scover y
# i scsi adm- mdi scover y - t sendt ar get s - p 10. 0. 0. 10: 3260
iSCSI login
# i scsi adm- mnode - T i qn. 2000- 05. com. 3par dat a: 20110002ac0001a6 - p
10. 0. 0. 10: 3260 - l
iSCSI Logout
10. 0. 0. 10: 3260 - u
Change iSCSI parameter
10. 0. 0. 10: 3260 - o
updat e - n node. cnx[ 0] . i scsi . MaxRecvDat aSegment Lengt h - v 65536
Adding custom iSCSI node
# i scsi adm- mnode - o new - p 10. 0. 0. 30: 3260
Removing iSCSI Node
# i scsi adm- mnode - o del et e - T i qn. 2000- 05. com. 3par dat a: 20110002ac0001a6 - p
10. 0. 0. 10: 3260
Removing SendTar get i SCSI Di scover y
A.2
3PAR Confidential
Display iSCSI node configuration
10. 0. 0. 20: 3260
Show all records in discovery database
# i scsi adm- mdi scover y
Show discovery record setting
# i scsi adm- mdi scover y - p 10. 0. 0. 10: 3260
Show all node records
Display session statistics
# i scsi adm- msessi on - r 1 - - st at s
Display session and device information
# i scsi adm- msessi on - i
B.1 Setting up the NVRAM and BIOS with QLogic HBA
3PAR Confidential
B
Setting up the NVRAM and
BIOS with QLogic HBA
This appendix describes how to set up the NVRAM and BIOS with the QLogic HBA.
Configure the following NVRAM settings for QLogic 23xx, 24xx and 25xx cards using the
QLogic Fast!UTIL. Access the Fast!UTIL utility by hard booting the server and, when prompted
and follow these steps:
1 Enter Fast!Util by pressing CTRL+Q when prompted.
2 Select a host adapter from the main menu.
3 Restore the default settings of the HBA as follows: Configuration Settings > Restore
Default Settings.
4 Make the following setting changes:
NOTE: The NVRAM settings on QLogic HBAs can be changed by any server in
which they are installed. These settings will persist for an HBA even after it is
removed form a server. To obtain the correct settings for this configuration, you
will be instructed to return all NVRAM settings to their default settings.
NOTE: Each HBA port is reported as a host bus adapter and the following settings
need to be made for each of them.
B.2
3PAR Confidential
Configuration Settings > Advanced Adapter Settings > Execution Throttle: 256
Configuration Settings > Advanced Adapter Settings > LUNs per Target: 256
Configuration Settings > Extended Firmware Settings > Data Rate: 2
(AutoNegotiate)
5 Specify the connection option.
Specify Loop topology for Direct Connect configurations: Configuration Settings >
Connection Options: 0 (Loop Only)
Specify Point to Point topology for Fabric configurations: Configuration Settings >
Connection Options: 1 (Point to Point Only)
6 Repeat for each port listed as a separate HBA port.
NOTE: The parameters provided through these menu options can vary between
different QLogic HBA models.
C.1 Configuring the QLogic HBA Using the SCLI Utility
3PAR Confidential
C
Configuring the QLogic
HBA Using the SCLI Utility
This appendix describes how to configure QLogic HBA settings using the SCLI utility.
In order to make topology changes to the QLogic cards in the Intel Itanium server which have
the Extensible Firmware Interface EFI as the system firmware (BIOS), use the QLogic SANsurfer
FC CLI utility. You can download the latest version of the SCLI utility from the QLogic web site
or use the version that is installed as part of the driver package installation.
Once you install the QLogic SANsurfer FC CLI utility for each of the HBA ports, set the correct
port connection type (direct --> loop, fabric --> point) by running the following commands:
For fabric connection
# / opt / QLogi c_Cor por at i on/ SANsur f er CLI / scl i - n X CO 1
For direct connection
# / opt / QLogi c_Cor por at i on/ SANsur f er CLI / scl i - n X CO 0
Where X is equal to the HBA FC port #, the HBA port numbers start with number 0.
NOTE: For Itanium Servers, this is the only method available. For other Intel
Platform Servers, either use the SCLI utility or the Fast!UTIl HBA BIOS method.
C.2
3PAR Confidential
For example to set the HBA ports 1 and 3 to Point to Point/Fabric topology, run the following
commands:
# / opt / Ql ogi c_Cor por at i on/ SANsur f er CLI / scl i - n 1 CO 1
To set the same HBA ports 1 and 3 to Direct topology run the following commands:
You can verify the setting, by running the following command:
# / opt / Ql ogi c_Cor por at i on/ SANsur f er CLI / scl i - I 1
# / opt / Ql ogi c_Cor por at i on/ SANsur f er CLI / scl i - I 3
Refer to the SANsurfer FC CLI utility program release notice for other command line options to
change the following settings:
LUNs per Target: 256
Data Rate: 4 (Auto Negotiate)
RH.1 Revision History

3PAR Confidential
Revision History
Release level Revision summary
320-200268 Rev A
December 2010
First release of this Implementation Guide.
RH.2
3PAR Confidential

3par RedHat 45 Implementation Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

3par RedHat 45 Implementation Guide

Uploaded by

Copyright:

Available Formats

3PAR Confidential

3PAR RedHat 4 & 5 and OEL

You might also like