Ds8000 Architecture Configuration

IBM Systems & Technology Group
IBM TotalStorage DS8000 Architecture & Configuration
Charlie Burger
Storage Systems Advanced Technical Support
2005 IBM Corporation
IBM Systems and Technology Group
Topics
DS8000 Physical Overview
DS8000 Subsystem Storage Hierarchy
Logical Configuration of CKD Volume Using DS CLI
Addendum
DS8000
Volumes with z/OS
HyperPAV
High
Performance FICON (zHPF)
DS8000
Performance Monitoring
TPC
RMF
Tivoli
Productivity Center 4.1
References
2

146
GB, 300 GB, 450 GB switched

bunch of disk (SBOD) disk drives (up to
1024) 15,000 RPM
1
TB SATA 7,200 RPM
73
& 146 GB Solid State Drive (SSD)
Encryption
available on 146 GB, 300 GB

and 450 GB 15 RPM HDDs
pSeries
Power5+
4-port
FC/FICON Host Adapter cards

(up to 128 ports)
Device
Adapters scale with capacity
64K
devices; 255 Logical Control Units

per logical DS8000 (FICON)
Two
logical DS8000s in one physical

DS8000 (9B2 LPAR)
Metro
and Global Mirror compatibility

with ESS and DS6000
Base Frame
Expansion Frame
DS8000 Physical Overview D8S100 & DS8300

Models
DS8100 (931)
DS8300 (932, 9B2)
Shared SMP processor configuration
POWER5+ dual 2-way
POWER5+ dual 4-way
Other major processors
PowerPC, Asics
PowerPC, Asics
IBM Virtualization Engine (LPAR) capability
Not available
Optional
Processor memory for cache and NVS (min/max)
16 GB/128 GB
32 GB/256 GB
Host adapter interfaces
4-port 4 Gbps or 2 Gbps Fibre

Channel/FICON, 2-port ESCON
4-port 4 Gbps or 2 Gbps Fibre

Channel/FICON, 2-port ESCON
Host adapters (min/max)
2/16
2/32
Host ports (min/max)
4/64
4/128
Drive interface
FC-AL
FC-AL
Number of disk drives (min/max)
16/384
16/1024
Device adapters
Up to eight 4-port FC-AL
Up to 16 4-port FC-AL
Maximum physical storage capacity**
384 TB
1024 TB
Disk sizes
73 GB solid-state drives
146 GB (15,000 rpm)
300 GB (15,000 rpm)
450 GB (15,000 rpm)
1 TB SATA (7,200 rpm)
146 GB (15,000 rpm)
300 GB (15,000 rpm)
450 GB (15,000 rpm)
1 TB SATA (7,200 rpm)
RAID levels
5, 6, 10
5, 6, 10
Number of expansion frames
Up to 4
http://www-03.ibm.com/systems/storage/disk/ds8000/specifications.html
5
DS8100 931 Maximum Configuration
DS8100 (931) Disks and Device Adapters (DAs)

8 arrays (64 disks) on DA0
DA Pair 0
2
2
0
3
3
1
Up to 4 DA pairs
HMC
2
2
0

DA pairs 0 to 3
2 DA pairs (DA0 and DA2)
used for disks in full base frame
4 DA pairs used for full base frame
and full expansion frame
Up to 16 Host Adapters
Up to 64 host ports
Server0 and Server1

7
S0
S1
0
b
0/1 1/0
2/3 3/2
DS8300 932/9B2 with 2 Expansion Frames
DS8300 Disks and Device Adapters (DAs)

Order of installation of disks on DA pairs
Top
to bottom
64-disk
increments(2 disk enclosure pairs) per DA
Maximum of 8 DA pairs
Base
frame
DA2, DA0
1st
expansion frame
DA6, DA4, DA7, DA5

2nd
expansion frame
DA3, DA1, DA2, DA0

3rd
expansion frame
DA6, DA4, DA7, DA5

4th
expansion frame
DA3, DA1
Up to 32 Host Adapters
Up
to 128 FICON/FCP host ports
64
ports in base frame
64
ports in expansion frame
I/O Ports
FCP/FICON capable ports
4
ports per adapter *
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Storage
Enclos u re
Port ID
I0xyz
CEC 0
x indicates DS8000 I/O enclosure (0-7)

y indicates slot/card (0,1,3,4)
z indicates port (0-3)
* There are 2 ESCON ports per adapter
10
CEC 1
I/O Drawer
RIO 1
I/O Drawer
RIO 1
I/O Drawer
RIO 0
I/O Drawer
RIO 0
DS8000 Disk Drives

Three different Fibre Channel DDM types (available in both non-encrypted and
encrypted versions):
146 GB, 15K RPM drive
One Serial Advanced Technology Attachment (SATA) DDM drive type:

1 TB, 7.2K RPM drive
Two different Solid State Drive (SSD) types:

73 GB
146 GB
IBM has withdrawn from marketing the following DS8000 disks:
11
146 GB 10,000 RPM Fibre Channel Disk Drives
500 GB 7,200 RPM FC Advanced Technology Attachment (FATA) Disk Drives
DS8000 Disk Drive Characteristics

Fibre Channel is:
Intended for heavy workloads in multi-user environments
Highest performance, availability, reliability, and functionality
Good capacity: 146 GB450 GB
Very high activity
Greater than 80% duty cycle
SATA-2 characteristics are:

Intended for lower workloads in multi-user environments
High performance, availability, and functionality
High reliability
More robust technology:
Extensive Command Queuing
High capacity: 5001,000 GB disk drives
Moderate activity
20-30% duty cycle
SSD characteristics are

Best suited for cache-unfriendly data

Random read access, high performance data sets
Both FB and CKD environments
12
DS8000 Architecture
8-32 host adapters

4 Gb FC/FICON 4-port *
p-Series Power5+ RISC

processors
2-way, 4-way
1 to 8 pairs 4Gb device

adapters w/ redundant
FC-AL loops
* ESCON 2-port
13
FC-AL Disk Layout
Single
Pack
Pack
Interconnect
Note: Diagrams only show the connections for one of

the two loops in a pack.
Problems with standard FC-AL JBOD

Full loop required to participate in xfer
Difficult to identify loop breakage
Performance drop off with loop increase
14
Switched FC-AL Logical Layout

C1
C3
C2
C4
FC Interface Card 1
FC-AL Switch
SES
Upper
connection (to Backplane
D0
device adapters
or prior set of 16
DDMs
SES
D1
D2
D3
D4
D5
Port Bypass Circuits
D6
D7
D8
D9
D10
D11
D12
D13
D14
D15
FC-AL Switch
FC Interface Card 2
C1
C2
Notes:
Cx = (x=1-4) connections to external FC-AL loops
Dy = y=0-16) DDMs
Switch provides 2 element FC-AL loop for all transfers
No performance drop off
15
C3
C4
Lower
connection (to
next set of 16
DDMs)
DS8100 (931) Components
16
DS8300 (932/9B2) Components
17
DS8300 (932/9B2) Architecture - LPAR
2 pSeries Systems
per Storage Facility
2 LPARs per
Storage Facility Image
Persistent Memory
By Dumping Memory
To System Disk while
On Battery
18
DS8300 2107-9B2 LPAR Dual Storage Images
Storage
Image 2
Storage
Image 2
19
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Empty Storage
Enclosure Image2
Storage
Enclosure
Empty Storage
Enclosure Image2
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
CEC 0
Storage
Image 1
CEC 1
Storage
Image 1
I/O Drawer
RIO 1
I/O Drawer
RIO 1
I/O Drawer
RIO 1
I/O Drawer
RIO 1
I/O Drawer
RIO 0
I/O Drawer
RIO 0
I/O Drawer
RIO 0
I/O Drawer
RIO 0
Image 1 Resource
Image 2 Resource
DS8300 LPAR Resource Allocation

Current implementation is 2 Storage Images (logical DS8000s)
50 percent of the processors is the default.
With variable LPAR implementation on DS8300 Model 9B2 (R4.0), the

split can be 25/75 percent or 75/25 percent.
50 percent of the processor memory is the default.

With variable LPAR implementation on DS8300 Model 9B2 (R4.0), the

split can be 25/75 percent or 75/25 percent.
Each storage image has access to:

Up to 16 host adapters (4 I/O drawers with up to 4 host adapters)
20
Up to 512 disk drives (up to 512 TB of capacity)
DS8000 Enhanced Addressing & Connectivity
21
ESS 800
DS8000
DS8000 w/LPAR
Max Logical Subsystems
32
255
510
Max Logical Devices
8K
64K
128K
Max Logical CKD Devices
4K
64K
128K
Max Logical FB Devices
4K
64K
128K
Max N-Port Logins/Port
128
509
509
Max N-Port Logins
512
8K
16K
Max Logical Paths/FICON Port
256
2K
2K
Max Logical Paths/CU Image
256
512
512
Max Path Groups/CU Image
128
256
256
DS8000 Caching Management

2004 - ARC (Adaptive Record Cache)
Dynamically
partitions the read cache between random and

sequential portions
2007 AMP (Adaptive Multi-Stream Pre-Fetch R3.0)

Manages
the sequential read cache and decides what, when

and how much to prefetch
2009 IWC (Intelligent Write Cache R4.2)

Manages
destage
22
the write cache and decides what order and rate to
Intelligent Write Caching (IWC)

What does it do?
Improves performance through better write cache management and better destage order of writes
Where can it be applied?

Write caches for storage controllers and even operating systems (i.e. linux)
DS8000 is the first product implementation
Benefits on DS8000
Up to 2X throughput for random write workloads
Typical database workload throughput may improve 15 - 20%
DS8000 SMP cycle usage reduced up to 12% via better data structures
Example of IBMs continuous innovation in our products

4 IBM Patents
Intelligent Write Caching is a hybrid of two other well known cache management
algorithms
CLOCK which exploits temporal ordering (Reduce the total number of destages to disk)
CSCAN which exploits spatial ordering (Average cost of each destage due to arm movement)
Both use a similar data structure so they are compatible
Key Attributes
23
Exploits and creates spatial locality
Also, exploits temporal locality
Works well even with concurrent reads
Handles a wide variety of workloads and cache sizes
Deployed at the storage controller level
And yet, it is simple
How Intelligent Write Caching Works

Organize write groups in a sorted
order forming a clock (temporal)
Clock hand moves clockwise
destaging write groups in order
(spatial)
Write groups are created with bit
initialized to 0
Clock hand can only destage
groups with bit 0
When clock hand encounters a
write group with bit 1, it resets bit
to zero, and skips it
On a write to an existing group
(hit), set the bit to 1
24
DS8000 Implementation
An IWC list for each Rank
NVS shared across all ranks in a cluster
Dynamically adapted size of each IWC list based on
workload intensity on each rank
Linear thresholding: Rate of destage is proportional to the
portion of NVS occupied by a IWC list
Destages are smoothed out (thus write bursts are not
translated into destage bursts)
25
Copy Services
Point in time copy
FlashCopy
Instant, T0 copy of what a set of volumes looked like at that one particular instant in time
Any changes after the point in time are not replicated unless another point in time
copy is requested
Not continuous - may be repeated
Logical copy - physical copy may be done later
The target volumes are immediately accessible for read and write
Target volumes are on the same storage subsystem as source volumes
May be used to protect against application-caused corruption
Remote Mirroring

Ongoing, continuous replication of a set of volumes

Target volumes are not accessible while replication is in progress
Target volumes are commonly on a different storage subsystem than source volumes
Disaster protection
Not designed to protect against application-caused corruption
Replicates what application allows

26
Data migration
Asynchronous
Global Copy
Global Mirror
z/OS Global Mirror
Synchronous
Metro Mirror
DS8000 Subsystem Storage Hierarchy
27
Storage Subsystem Hierarchy
Storage Complex
One
or multiple physical storage subsystems
Central
management point with Network Server
DS8000 Hardware Management Console (HMC)
28
S to ra g e
E n c lo su re
S to ra g e
E n c lo su re
S to ra g e
E n c lo su re
S to ra g e
E n c lo su re
S to ra g e
E n c lo su re
S to ra g e
E n c lo su re
S to ra g e
E n c lo s u re
S to ra g e
E n c lo s u re
CE C 0
CE C 0
CE C 1
CE C 1
I/O D r a w e r
R IO 1
I/O D r a w e r
R IO 1
I/O D r a w e r
R IO 1
I/O D r a w e r
R IO 1
I/O D r a w e r
R IO 0
I/O D r a w e r
R IO 0
I/O D r a w e r
R IO 0
I/O D r a w e r
R IO 0
Storage Subsystem Hierarchy (continued)

S to ra g e
E n c lo s u re
S to ra g e
E n c lo s u re
Storage Unit (Storage facility)

Single
S to ra g e
E n c lo s u re
S to ra g e
E n c lo s u re
CEC 0
physical storage subsystem
CEC 1
I/O D r a w e r
R IO 1
I/O D ra w e r
R IO 1
I/O D r a w e r
R IO 0
I/O D ra w e r
R IO 0
Storage Image
Single
logical storage subsystem
Same as physical subsystem for 2107

921/931, 2107 922/932 and 1750
Single LPAR for 2107 9A2/9B2
S to ra g e
E n c lo s u re
S to ra g e
E n c lo s u re
S to ra g e
E n c lo s u re
S to ra g e
E n c lo s u re
S to ra g e
E n c lo s u re
S to ra g e
E n c lo s u re
S to ra g e
E n c lo s u re
CEC 0
CEC 1
or
S to ra g e
E n c lo s u re
S h a rk
Im a ge 0
(L P A R 0 )
CEC 0
S h a rk
Im a g e 1
(L P A R 1 )
S h a rk
Im a ge 0
(L P A R 0 )
CEC 1
S h a rk
Im a g e 1
(L P A R 1 )
L P A R S y s te m
D is k s
I/O D r a w e r
R IO 1
I/O D ra w e r
R IO 1
I/O D r a w e r
R IO 1
I/O D r a w e r
R IO 1
I/O D r a w e r
R IO 0
I/O D ra w e r
R IO 0
I/O D r a w e r
R IO 0
I/O D r a w e r
R IO 0
Server
Server0
and Server1
Manage
extent pools and volumes
S to ra g e
E n c lo s u re
S to ra g e
E n c lo s u re
S to ra g e
E n c lo s u re
Even numbers managed by Server0

Odd numbers managed by Server1
29
S to ra g e
E n c lo s u re
S0
S1
CEC 0
CEC 1
I/O D r a w e r
R IO 1
I/O D ra w e r
R IO 1
I/O D r a w e r
R IO 0
I/O D ra w e r
R IO 0
S0
S1
DS8000 Array Site

Logical grouping of disks
Same
capacity and speed
73 GB solid state drives

146 GB solid state drives
146 GB (15,000 rpm)
300 GB (15,000 rpm)
450 GB (15,000 rpm)
1 TB (7,200 rpm)
Created and assigned to DA pair by software

during installation
DS8000 array site
8
30
disks (DDMs)
DS8000 array site
Array
RAID array
RAID5, RAID6 or RAID10
Created from 1 array site on DS8000
RAID type (RAID5, RAID6 or RAID10) and storage type (Fixed

Block/FB or Count Key Data/CKD) now defined in 2 separate
steps
31
Array defines RAID type
RANK defines storage type
Array
D
D
1 array site
RAID5
6+P
7+P
Parity is striped across all disks in
array but consumes capacity
equivalent to one disk
RAID 6
6+P+Q
5+P+Q+Spare
RAID10
3+3
4+4
D
P
D
S
RAID5 6+P+S
DS8000 8 DDM arrays

D
D
D
D
D
D
D
D
D
P
RAID5 7+P
D
D
D
D
D
D
P
Q
RAID6 6+P+Q
D
D
D
D
D
P
S
Q
RAID6 5+P+Q+S
D
D
D
D
D
D
S
S
RAID10 3+3+S+S
D
D
D
D
D
D
D
D
RAID10 4+4
32
Rank
DS8000 CKD rank
RAID array with storage type defined

CKD or FB
One-to-one relationship between an

array and a rank
33
One RAID array becomes one rank
DS8000 8 DDMs
RAID5 7+P
Rank (continued)
Ranks have no pre-determined or
fixed relation to:
Server0 or Server1
Logical Subsystems (LSSs)
CKD Rank
Ranks are divided into extents

Units of space for volume creation
CKD rank
Extents equivalent to a 3390M1
1113 cylinders or .94GB
FB rank
1GB extents
34
FB Rank
Extent Pool
Logical grouping of extents from one or more ranks from
which volumes will be created
Ranks are assigned to extent pools
Pool contains one or more ranks
LUN size is not limited to rank size

If more than one rank assigned to pool
Rotate Volumes method of using extents still the default for

volume creation
Rotate Extents or Storage Pool Striping introduced with R3
Extent pool is one storage type
CKD or FB
35
Extent Pool
Rank
Extent Pool (continued)

User-assigned to Server0 or Server1
Activity should be balanced across
both servers
36
Minimum of 2 extent pools required

to utilize system resources
Maximum number of ranks in a

single pool should be the total
number of ranks to balance system
resources
Server0 extent pools will support

even-numbered LSSs
Server1 extent pools will support

odd-numbered LSSs
Ext. Pool 0
Ext. Pool 1
Rank 0
Rank 2
Rank 4
Rank 6
Rank 1
Rank 3
Rank 5
Rank 7
Extent Pool ID
System-generated
Px
If
x is an even number:
Pool has been assigned to Server0
Even-numbered LSSs are available for
volume creation
If
x is an odd number:
Pool has been assigned to Server1
Odd-numbered LSSs are available for
volume creation
Extent pools may be given user

specified nicknames
37
Extent Pool P0
Rank
DS8000 Logical Control Units and Address Groups

Similar to ESS:
LCU has a maximum of 256 addresses
Aliases (Parallel Access Volumes/PAVs) are shared within an LCU
LCU is the basis for Copy Services paths and consistency groups
Even-numbered LCUs are associated with Server0 and odd-numbered LCUs are
associated with Server1
New:
LCU does not have a pre-determined relationship to rank/DA pair
Up to 255 LCUs are available
FB LSSs are automatically created during LUN creation
e.g. Creation of volume 1000 results in creation of LSS 10
Set of 16 Logical Subsystems (LSSs) is called an Address Group

LSS 00-0F, 10-1F, 20-2F, 30-3F, etc.
Storage type for entire Address Group (16 LSSs) is set to CKD or Fixed Block by the
first LSS defined *
CKD LSSs (LCUs) are explicitly defined

Allows specification of LCU type and SSID
* ESCON devices must be in Address Group 0 (LCU 00-0F)
38
DS8000 Flexible LCU Assignment

No predetermined or fixed relationship to rank
3 options:
1.
One LCU per rank

Simplifies management
2.
3.
39
Multiple LCUs on one rank

Provides additional addresses for
rank

Allows utilization of large drives
with small device types
One LCU across multiple ranks

Enables cross-rank Copy Services
consistency group

This will be the default for multiplerank extent pools
Server1
Server0
Ranks/Pools
Ranks/Pools
LCU 00
LCU 01
12
14
16
18
13
2A
2B
2A
2B
15
17
19
Address Group
Logical grouping of 16 LSSs (4096 volumes)
00-0F,
10-1F, 20-2F,
All LSSs in an address group are the same storage

type
All CKD or all FB
User determines whether an address group will be CKD or FB

by creation of 1st LSS in that address group
CKD LCU creation or FB LUN creation
If first LSS is CKD, all 16 LSSs must be CKD
If first LSS is FB, all 16 LSSs must be FB
DS8000 ESCON devices must be in address group 0 LSS 00-0F

40
Logical Subsystem ID
xy
Now designated by 2 digits to allow increased numbers of LSSs
x
indicates the address group

An address group is a pre-determined set of 16 LCUs/LSSs (x0-xf)
of the same storage type (all CKD or all FB)
y
indicates server assignment

If y is even, LSS is available with Server0 extent pools
If y is odd, LSS is available with Server1 extent pools
Example:
LCU 2A is created (storage type CKD)
2 indicates Address Group 2 all 16 LSSs in address group 2

(20-2f) will be CKD
A (even) indicates Server0 LSS 2A can only be used with extent
pools assigned to Server0
41
Volume
Extent Pool
Rank
Created from extents in one extent pool

Volumes can be larger than the size of a rank (if
multiple ranks are in one extent pool)
DS8000 introduced with CKD max size 64K
cylinders or 56GB (with appropriate software
support)
DS8000 with R3.1 has CKD max size 262,668
cylinders or 223 GB (with appropriate software
support)
2
A
1
0
FB max size 2TB
Extent Pool
Rank
Rank
Volumes can be presented to host server in

cylinder, 100MB or block granularity
42
Space is allocated in 1GB extents (FB) or 1113

cylinder extents (CKD)
2A11
Volumes (continued)
Volumes can be created or displayed in
Binary GB
Decimal GB
Blocks (512 Bytes)
Volumes can be dynamically deleted

FB volume creation results in FB LSS creation and
assignment of FB storage type to address group
CKD Logical Control Unit (LCU) creation is pre-req for
CKD volume creation
43
DS8000 Volume ID
User specifies 4-digit hex volume ID which includes address
group, LCU and device ID:
xyzz
x=Address Group
xy=LSS
Even LCUs are available for Server0 extent pools

Odd LCUs are available for Server1 extent pools
zz=device ID
Example: Volume ID x2A10

2=Address Group 2 (group of 16 LCUs 20-2F)
2A=LCU 2A
2A is even- may only be used with Server0 pools
10=device ID
Maximum number of volumes is 64 K

Nicknames can be created for volumes (and PAVs)
User may choose volume IDs to match z/OS device IDs
44
Extent Pool P0
Assigned to Server0
Even LCUs available
Rank R0
2A10
DS8000 Rotate Volumes
Extent Pool P0
Assigned to Server0
Even LCUs available
Rank R0
Single volume is created from extents on one

rank if possible
2A10
Single volume may spill across ranks in pool

or may be larger than the size of a single rank
Rank R0
Volumes may be dynamically deleted and

extents reused
2A10
2A11
Rank R2
45
DS8000 Rotate Volumes

Rank R0
In single-rank extent pool, multiple

volumes will be created
sequentially on the rank
In a multiple-rank extent pool,

current implementation places
multiple volumes on rank with
most free extents
2A10
2A11
Rank R0
2A10
2A11
Volumes may be dynamically

deleted and extents reused
46
Rank R2
DS8000 Storage Pool Striping

New algorithm choice for volume creation
Rotate Volumes method still the default
4
7
5
1
Rank 9
Naming
Marketing Storage Pool Striping
DS CLI & DS Storage Manager Rotate Extents
Volumes are created by allocating one Extent from

available Ranks in an Extent Pool, in a round-robin
fashion
At right - 7 GB Volume showing the order of Extent
allocation
6
Rank 10
2
Rank 11
3
CKD and Fixed Block
Extent Pool with 3 Ranks
47
Storage Pool Striping - Advantages

Technical Advantages
Method
to distribute I/O load across multiple Ranks
DS8000
Far
optimized for performance
less special tuning required for high performance data placement.
Means less work for the storage administrator

Reduces storage administrator work needed to optimize performance
48
Storage Pool Striping - Characteristics

2nd Volume start
The next Volume will be started from an Extent on

the next Rank in the round-robin rotation
If a Rank runs out of extents, it is skipped
Multiple Volume allocations will not start on the same
Rank
If
many Volumes are created with a single

command, the Volumes will not start on the
same Rank
Supports new Volume Expansion capabilities
49
Extent pool with 3 Ranks
Storage Pool Striping Considerations & Recommendations (2)

Deleting Volumes creates free Extent
units for future volume allocation
No reorg capability
Do not add new Ranks to existing MultiRank Extent Pools used for Storage Pool
Striping
No
DA0
DA0
DA3
DA3
reorg or reallocation capability
Mixing striped and non-striped Volumes

in same multi-Rank Extent Pool
Supported,
but strongly not recommended

because it will create Rank I/O imbalance
50
DA2
DA2
ExtPool 0
ExtPool 1
Storage Pool Striping Considerations &

Recommendations (3)
Not recommended for:
SVC
When providing LUNs to the SVC, current best practice is to not use
SPS and just give the SVC LUNs that are an entire RAID5 array
iSeries
Performance benefit varies due to OS striping

There are cases where performance will benefit from rotate extents
Supported and can be exploited for better capacity utilization
Specific
vendor Volume layout recommendations/requirements
DB2 BCU
Oracle ASM
Applications
where OS level striping would be better choice
Small, hot Volumes
51
Volume Virtualization Overview

Standard Logical Volumes
Track Space Efficient Logical Volumes
Used
with Space-Efficient Flash Copy
Extent Space Efficient Logical Volumes

Used
52
with Thin Provisioning
Standard Logical Volume (LV)

Standard LV consists of 1 to N Real
Extents
Each Extent contains Extent Data and
Extent Metadata
Each LV Extent is mapped to a Real
Extent on a Real Rank
All Extents allocated to a LV come from
one Extent Pool
FB Extent = 1024 MB Extent Data
Less 4.5MB Extent Metadata
CKD Extent = 1113 Cylinders Extent Data
Less 64 cylinders Extent
Metadata
53
Extent Pool
Standard LV
M
M
M
Real Rank
M
M
M
M
M
M
Extent Extent
Metadata
Data
Track Space Efficient (TSE) LV

Extent Pool
TSE LV consists of 1 to N Real Tracks

Each TSE LV Extent is mapped to a Virtual
Extent on a Virtual Rank
Virtual Extent provides only Extent metadata
If host writes to a TSE LV track, a Real Track in

the SE Repository is allocated (if not already
allocated) to the TSE LV Track
Mapping from LV Track to SE Repository Track is

maintained in the SE Repository Metadata
In same Extent Pool as TSE LV

Virtual extents allocated to a TSE LV
Real tracks allocated to a TSE LV come from the

one SE Repository
Capacity for virtual ranks & SE Repository

auxiliary volumes
Configured with real extents like a standard LV, but

is not host addressable
FB Repository Extent = 16K tracks = 1GB
CKD Repository Extent = 16695 tracks = 1GB
1 Metadata Extent for each 91 Repository Extents
Introduced with R3.0

54
Track SE LV
Intended for use as FlashCopy targets
SE Repository
Auxiliary Volume
M
M
M
Tracks
Real Rank
M
M
M
M
M
M
Virtual Rank
M
M
M
M
M
M
Rep. Metadata
Virtual Rank
Auxiliary Volume
MMMMMM
MMMMMM
Extent Space Efficient (ESE) LV

ESE LV consists of 1 to N Real Extents
Each ESE LV Extent is mapped to a Virtual
Extent on a Virtual Rank
Virtual Extent provides only Extent metadata
If write is first host write to an ESE LV

Extent, a Real Extent on a Real Rank is
allocated to the ESE LV Track
If not first host write to an ESE LV Extent,
write goes to already existing Real Extent
In same Extent Pool as ESE LV
Virtual extents allocated to a ESE LV
Real extents allocated to a ESE LV
Capacity for the metadata in the virtual rank

is from a Virtual Rank Auxiliary Volume (VR
AV)
Configured with real extents like a standard LV,

but is not host addressable
FB VR AV Real Extent = 4.5MB
CKD VR AV Real Extent = 64 cylinders
Introduced with R4.3
55
Only FB support with this release
No copy services support with this release
Extent Pool
Extent SE LV
M
M
M
Real Rank
M
M
M
M
M
M
Virtual Rank
M
M
M
M
M
M
Virtual Rank
Auxiliary Volume
MMMMMM
MMMMMM
Open Systems Host
pSeries1
Server connecting to the storage subsystem

Host definition includes one or more host
attachments
Each host attachment contains one or more

Host Bus Attachments (HBAs) with
Worldwide Port Names (WWPNs)
Multiple server HBAs can be grouped into a
single host attachment for convenience
A single host attachment can be in only one
volume group
Four host attachments (one port each) are

defined for pSeries1
Two host attachments with 2 ports each

are defined for pSeries2
One host attachment with 4 ports is
defined for pSeries3
56
SAN
SAN
pSeries2
SAN
SAN
pSeries3
SAN
SAN
Open Systems Host (continued)

Storage
Enclosure
A host attachment can access:

User-specified disk subsystem I/O ports
All
Storage
Enclosure
Storage
Enclosure
Storage
Enclosure
CEC 0
valid disk subsystem I/O ports
Configured as FC_AL or FCP
CEC 1
I/O Drawer
RIO 1
I/O Drawer
RIO 1
I/O Drawer
RIO 0
I/O Drawer
RIO 0
Access
may also be controlled through

SAN zoning
Host definitions may be given userspecified nicknames
SAN
SAN
pSeries2
57
Open Systems Volume Group

Used to control server access to LUNs (LUN
masking)
Contains:
Open systems server HBA WWPN(s) AND
LUNs to be accessed
AIX 1
AIX 1 Volume Group

Recommend creating one volume group for
each server unless LUN sharing is required
Many-many relationship to extent pools or
LSSs
Volume groups may be given user-specified
nicknames
No relation to AIX volume groups
58
1000
1001
Open Systems Volume Groups

(continued)
AIX 1
AIX 2
When LUN sharing is required:

Place host attachments for multiple servers

in the same volume group
AIX Volume Group
OR
Place LUNs in multiple volume groups

AIX 3
AIX 3 Volume Group
1000
59
1001
1000
1001
AIX 4
AIX 4 Volume Group
1000
1001
In either case,
server software
is responsible for
data integrity!
Storage Resource Summary

Disk
Individual DDMs
Array Sites
Pre-determined grouping of DDMs of same speed
and capacity (8 DDMs for DS8000; 4 DDMs for
DS6000)
Arrays
One 8-DDM Array Site used to construct one
RAID array (DS8000)
One or two 4-DDM Array Sites used to construct

one RAID array (DS6000)
Ranks
One Array forms one CKD or FB Rank (8 DDMs
for DS8000; 4 or 8 DDMs for DS6000)
No fixed, pre-determined relation to LSS

Extent Pools
1 or more ranks of a single storage type (CKD or
FB)
CKD or FB
60
Assigned to Server0 or Server1
Extent
Pool
Storage Resource Summary (continued)

Volumes or LUNs
Made
Min
Max
up of extents from one extent pool
allocation is one extent -- 1GB(FB) Mod1(CKD)

size is 2TB (FB); 223GB(CKD)
Can be larger than 1 rank if more than 1 rank in pool

Associated
with LSS during configuration
Available LSSs determined by Extent Pool server affinity

Can
be individually deleted
Open Systems Volume Group

Contains
LUNs and host attachments -- FB LUN
AIX host port
AIX host port
iSeries host
port group
masking
One
host attachment (one port or port group) can be

member of only one volume group
One
volume can be member of multiple volume groups
Multiple hosts can be contained in a single volume

group
FB
FB
61
FB (i)
FB
FB
FB
Recommendations
Create extent pools using storage pool striping with 4-8 ranks
in the pool
Balance extent pools and ranks across servers
Create one LSS per rank
Unless more addresses are needed
Use a limited number of device types for ease of management
Use custom volumes that are even multiple of extents
CKD 1113 cylinder extents
3390M3
3390M9
30051 cylinders
60102 cylinders
FB 1 GB extents
Use PAVs to allow concurrent access to base volumes for z/OS

Preferably HyperPAV
62
Logical Configuration of CKD Volumes Using DS CLI
63
CKD Volume Logical Configuration

DS8000 Storage Manager
DS8000 DS CLI
64
DS8000 Storage Manager
Easy-to-use, powerful and flexible User Interface

Wizards,
Filters, Sorts, Hyperlinks, Animation, Copy/Paste
Includes optional automated methods

Runs on DS8000 integrated HMC; accessed via Browser
65
DS8000 DS CLI
Powerful tool for automating configuration tasks and
collecting configuration information
Same DS CLI for DS6000 and for ESS 800 Copy Services
DS CLI commands can be saved as scripts which
significantly reduces the time to create, edit and verify
their content
Uses a consistent syntax with other IBM TotalStorage
products now and in the future
All of the function available to the GUI is also available
via the DS CLI
66
Supported DS CLI Platforms

The DS Command-Line Interface (CLI) can be installed on the following
operating systems:
AIX 5.1, 5.2, 5.3
HP-UX 11i v1, v2
HP Tru64 version 5.1, 5.1A
Linux (RedHat 3.0 Advanced Server (AS) and Enterprise Server
(ES)
SUSE Linux SLES 8, SLES 9, SUSE 8, SUSE 9)
Novell Netware 6.5
Open VMS 7.3-1, 7.3-2
Sun Solaris 7, 8, 9
Windows 2000, Windows Datacenter, and Windows 2003
67
CKD Logical Configuration Steps

Creating CKD extent pools
Creating arrays
Creating and associating ranks with extent pools
Creating logical control units
Creating CKD volumes
Creating CKD volume groups (system generated)
68
Creating CKD extent pools

Remember from earlier?
Minimum of 2 extent pools required
Server0 extent pools will support even-numbered LSSs
Server1 extent pools will support odd-numbered LSSs
Consider creating additional extent pools for each of the following

conditions:
Each RAID type (5, 6 or 10)
Each disk drive module (DDM) size
Each CKD volume type (3380, 3390)
Each logical control unit (LCU) address group
mkextpool -dev IBM.2107-75nnnnn rankgrp 0 -stgtype ckd P0

mkextpool -dev IBM.2107-75nnnnn rankgrp 1 -stgtype ckd P1
lsextpool dev IBM.2107-75nnnnnn -l
69
Creating Arrays
RAID array
Created from 1 array site on DS8000
Array Site
Logical grouping of disks
Same capacity and speed
Issue the lsarraysite command to find the unassigned array sites

lsarraysite -dev IBM.2107-75nnnnn -state unassigned
Issue the mkarray command to create an array
from each site with the status unassigned
mkarray -dev IBM.2107-75nnnnn -raidtype 5 -arsite A1
lsarray dev IBM.2107-75nnnnn l A1
70
Creating CKD Ranks

RAID array with storage type defined
CKD or FB
One-to-one relationship between an array and a rank
One RAID array becomes one rank
DS8000 8 DDMs
Ranks have no pre-determined or fixed relation to:
Server0, Server1 or Logical Subsystems (LSSs)
Ranks are divided into extents
Units of space for volume creation
CKD rank extents equivalent to a 3390M1

1113 cylinders or .94GB
Issue the lsarray command to find unassigned arrays

lsarray -dev IBM.2107-75nnnnn -state unassigned
Issue the mkrank command to assign a rank to rank group 0 or 1
according to the rank group number of the assigned extent pool ID.
mkrank -dev IBM.2107-75nnnnn -array a1 -stgtype ckd -extpool p1
lsrank -dev IBM.2107-75nnnnn -l
71
Creating CKD Logical Control Units (LCUs)

Up to 255 LCUs are available
LCU has a maximum of 256 addresses
Aliases (Parallel Access Volumes/PAVs) are shared within an LCU
Even-numbered LCUs are associated with Server0 and odd-numbered
LCUs are associated with Server1
LCU does not have a pre-determined relationship to rank/DA pair
Set of 16 Logical Subsystems (LSSs) is called an Address Group
LSS 00-0F, 10-1F, 20-2F, 30-3F, etc.
Storage type for entire Address Group (16 LSSs) is set to CKD or
Fixed Block by the first LSS defined
CKD LSSs (LCUs) are explicitly defined
Allows specification of LCU type and SSID
Issue lsaddressgrp to find unassigned address groups

lsaddressgrp -dev IBM.2107-75nnnnn
72
Creating CKD LCUs (cont)

Analyze the report to identify all of the address groups that are
available to be defined. Use the following criteria:

If the list is empty, all of the address groups are available to be defined.
A defined address group with the storage type fb (fixed block) is not
available to be defined.
A defined address group with the storage type ckd and with fewer than
16 LSSs is available for LCU definition.
If you are using an undefined address group to make new LCUs, select
the lowest numbered address group that is not defined.
If you are defining a new LCU in an existing CKD address group, use
the lslcu command to identify LCUs that are already defined in the
target address group.
Issue the mklcu command to create an LCU.

dscli>mklcu dev IBM.2107-75nnnnn -qty 16 -id 00 -ss 0010 -lcutype 3390-3
lslcu -dev IBM.2107-75nnnnn -l
73
Creating CKD Volumes


p1 Extent Pool
a1 Array
r1 Rank
00:0F LCUs
View your list of CKD extent pool IDs and determine which extent pool IDs that you
want to use as the source for the CKD volumes to be created. You obtained this list
when you first created your extent pools. If this list is not available, you can issue
the lsextpool command to obtain the list of extent pool IDs.
Issue the mkckdvol command to make 128 base and 128 alias volumes for each LCU.
Issue the mkckdvol command to create 128 3390 base volumes for the LCU.
mkckdvol -dev IBM.2107-75nnnnn -extpool p1 -cap 3339 -name finance#d 0000-007F
mkaliasvol -dev IBM.2107-75nnnnn base 0000-007F -order decrement -qty 2 00FF
lsrank -dev IBM.2107-75nnnnn -l
74
Initialize the CKD Volumes

Use ICKDSF to initialize the newly configured CKD Volumes
There is no VTOC, IXVTOC, VVDS or Volume Label at this time
To insure that you only initialize volumes without a label, specify
INIT UNITADDRESS(uadd) VOLID(volser) VFY(*NONE*) VTOC(n,n,nn) INDEX(n,n,nn)
75
Addendum
DS8000 Volumes with z/OS
HyperPAV
High Performance FICON
DS8000 Performance Monitoring
TPC
RMF
Tivoli Productivity Center
References
76
77

System z supports both CKD and FB volumes (FB for zSeries Linux)
FB volumes are FCP attached, CKD volumes are FICON attached
Same 4GB 4 port FC Host Attachment Feature supports either FCP or
FICON
Assigned at the Port level (FCP or FICON, single port cant be both)
FICON Fastloadnew method for Adapter Code Load
FICON FASTLOAD used for FICON attach

Compatible with FCP FastloadAllows intermix use of ports on HA
Architected event, no long busy used (yes)
Loss of Light less than 1.5 seconds for Adapter Code Load (only when Adapter
Code is upgraded)
Concurrent Code Load Support

Advise all host attachments have (at least) two ports

Preferred on two separate Host Adapters
CKD also supported by ESCON Host Attachment Feature
78
DS8000 CKD Volumes

CKD standard volumes
3380
3390M3
3390M9
CKD custom volumes
Minimum volume size specification is 1 cylinder
Minimum space allocation is 1 extent (1113 cylinders)
Maximum volume size is 65,520 cylinders/56GB (DS8000 introduced with it)

With z/OS 1.4 or higher software support
Maximum volume size is 262,668 cylinders/223GB with R3.1

With z/OS 1.9+ or higher software support
Use a multiple of 1113 cylinders if possible
Maximum number of CKD volumes

64K per logical DS8000 *
* 4K limitation for for ESCON access
79
DS8000 z/OS HCD Considerations

New device support for D/T2107
Address Group 2 LCU 20 & 21

CNTLUNIT CUNUMBR=A000,PATH=(52,53,54,55),
DS8000 supports up to 16
Address Groups
64K logical volumes
For IOCP and HCD, the CU

addresses are hex 00 FE
LCU / LSS do not have to be
contiguous
UNITADD=((00,256)),LINK=(24,34,25,35),
CUADD=20,UNIT=2107,DESC='N150 LCU20'
CNTLUNIT CUNUMBR=A100,PATH=(52,53,54,55),
UNITADD=((00,256)),LINK=(24,34,25,35),
CUADD=21,UNIT=2107,DESC='N150 LCU21
IODEVICE ADDRESS=((2000,128)),CUNUMBR=A000,
STADAT=Y,UNIT=3390B
STADAT=Y,UNIT=3390A
STADAT=Y,UNIT=3390B
STADAT=Y,UNIT=3390A
Examples provided at the DS8000 Information Center Search with IOCP

80
DS8000 z/OS HCD Considerations Subchannel Sets

Multiple Subchannel Sets

Relief for 64K Devices per LPAR
Z9 (2094) processor/ zOS 1.7 only

HCD Implementation
Initial Implementation of SS1 req POR
Channel SubSystem (CSS) definition can

contain Subchannel Sets 0 & 1
Changed QPAVS display:

DS QPAVS,E278,VOLUME
IEE459I 09.57.53 DEVSERV QPAVS 046
HOST
256 Channels per CSS

No changes to LSS definition in ESS,

DS6000, DS8000
CONFIGURATION
CONFIGURATION
--------------
--------------------
UNIT
UNIT
UA
NUM. UA
TYPE
STATUS
SSID
ADDR.
TYPE
----- --
----
------
----
----
-----------
0E278 78
BASE
3205
78
BASE
1E279 79
ALIAS-E278
3205
79
ALIAS-78
1E27A 7A
ALIAS-E278
3205
7A
ALIAS-78
Assign IODevice Base to Set 0

Assign IODevice Alias to Set 0 or 1
0E27B 7B
ALIAS-E278
3205
7B
ALIAS-78
0E27C 7C
ALIAS-E278
3205
7C
ALIAS-78
Duplicate Device Numbers Possible

Desirable
0E27D 7D
ALIAS-E278
3205
7D
ALIAS-78
1E27E 7E
ALIAS-E278
3205
7E
ALIAS-78
ALIAS-E278
3205
7F
ALIAS-78
Providing they are in separate

Subchannel Sets ...No Problem
SUBSYSTEM
1E27F 7F
****
8 DEVICE(S) MET THE SELECTION CRITERIA
Flexible LSS structure

LPAR
Information provided in: z/OS V1R7.0 HCD Planning (GA22-7525-09)

0
81
Using larger volume sizes

Benefits
Fewer objects to define and manage
Less processing for fewer I/O resources
CF CHPID, VARY PATH, VARY DEVICE
Channel path recover, link recovery, reset event processing
CC3 processing
ENF Signals
RMF, SMF
Number of physical resources: CHPIDs, Switches, CU ports, fibers
Each device consumes real storage:
768 bytes of real storage for UCB and related control blocks
256 bytes of HSA
1024 bytes/device * 64K devices = 64MB
31 bit common storage constraints
EOV processing to switch to the next volume of a sequential data set
significantly slows the access methods
Considerations

82
Data migration to larger devices may be challenging, time consuming
zSeries Parallel Access Volumes (PAVs)

Additional addresses for a single device for improved performance
PAVs are shared within an LSS
An LSS may be on multiple ranks
Multiple LSSs may be on one rank
Recommendations
Use HyperPav if possible

If not HyperPav use dynamic PAV if possible
Requires parallel sysplex and WLM
Requires WLM having dynamic PAV specified
Requires WLM specified in device definition
83
DS8000 HyperPAV
84
IO without PAV
z/OS Sysplex
Applications
do I/O to
base
volumes
Applications
do I/O to
base
volumes
Applications
do I/O to
base
volumes
Applications
do I/O to
base
volumes
85
z/OS Image
DS8000 Storage Server
UCB 0801
UCB 0802
If UCB is
BUSY then
IOS Queue
Logical Subsystem (LSS) 0800
Base UA=01
UCB 0801
If UCB is
BUSY then
IOS Queue
UCB 0802
z/OS Image
Base UA=02
Parallel Access Volumes - Today
z/OS Sysplex
Applications
do I/O to
base
volumes
Applications
do I/O to
base
volumes
Applications
do I/O to
base
volumes
Applications
do I/O to
base
volumes
86
z/OS Image
UCB 08F1
UCB 08F0
UCB 0801
UCB 08F3
UCB 08F2
UCB 0802

Alias UA=F1
Alias UA=F0
Base UA=01
UCB 08F1
UCB 08F0
UCB 0801
Alias UA=F3
Alias UA=F2
Base UA=02
UCB 08F3
UCB 08F2
UCB 0802
z/OS Image
HyperPAV
z/OS Sysplex
Applications
do I/O to
base
volumes
Applications
do I/O to
base
volumes
Applications
do I/O to
base
volumes
z/OS Image
UCB 08F3
UCB 08F2
O
O
UCB 0801
UCB 08F1
UCB 08F0
UCB 0802
Alias UA=F0
Alias UA=F1
Alias UA=F2
Alias UA=F3
z/OS Image
Base UA=01
UCB 08F0
UCB 0801
Base UA=02
Applications
do I/O to
base
volumes
87
UCB 08F1
UCB 08F3
P
O
UCB 0802
O
UCB 08F2
Benefits of HyperPAV
Reduce number of required aliases
Give back addressable device numbers
Use additional addresses to
support more base addresses
larger capacity devices.
z/OS can react more quickly to I/O loads

React instantaneously to market open conditions
Overhead of managing alias exposures reduced
WLM not involved in measuring and moving aliases
Alias moves not coordinated throughout sysplex
Initialization doesnt require static bindings
Static bindings not required after swaps
IO reduction, no longer need to BIND/UNBIND to manage
HyperPAV aliases
Increases I/O Parallelism
88
HyperPAV System Requirements

Hardware
Software
DS8000 Bundle Version 6.2.4
DS8000 FCP/FICON Host

Adapters
Licensed Features
FICON/ESCON Attach (Turbo)

DS8000 #0702 and
#7090
239x-LFA #7090 or 2244OEL #7090
PAV
DS8000 #0780 and #78xx
239x-LFA #78xx or 2244PAV #78xx
HyperPAV
DS8000 #0782 and
#7899
239x-LFA #7899 or 2244PAV #7899
89
z/OS 1.6 plus
RMF
IOS support
OA13915
DFSMS support (DFSMS, SMS, AOM,
DEVSERV)
OA13928, OA13929, OA14002,
OA14005, OA17605, OA17746
WLM support
OA12699
GRS support
OA14556
ASM support
OA14248
OA12865
Optionally Fixes for GDPS/Omegamon
HyperPAV - Migration
No HCD or DS8000 Logical Configuration Changes
required
On
existing LSSs, assuming PAV and FICON are used today
HyperPAV deployment can be staged

Load/Authorize
HyperPAV feature on DS8000
Can
run without exploiting this feature if necessary using z/OS

PARMLIB option
Enable
HyperPAV feature on z/OS images that want to utilize

HyperPAV via PARMLIB or SETIOS command
Eventually
enable HyperPAV feature on all z/OS images in the

sysplex and authorize licensed function on all attached DS8000s
Reduce
90
the number of aliases defined
HyperPAV z/OS Options and Commands

SYS1.PARMLIB(IECIOSxx)
HYPERPAV=YES|NO|BASEONLY
YES Attempt to initialize LSSes in HyperPAV mode
NO Do not attempt to initialize LSSes in HyperPAV mode
BASEONLY Attempt to initialize LSSes in HyperPAV
mode, but only start I/Os on base volumes
Enhanced Commands
SETIOS HYPERPAV=YES|NO|BASEONLY
91
SET IOS=xx
D M=DEV
D IOS,HYPERPAV
DEVSERV QPAV,dddd
HyperPAV D M=DEV
SY1 d m=dev(0710)
SY1 IEE174I 23.35.49 DISPLAY M 835
DEVICE 0710
STATUS=ONLINE
CHP
10
20
30
40
DEST LINK ADDRESS
10
20
30
40
PATH ONLINE
Y
Y
Y
Y
CHP PHYSICALLY ONLINE Y
Y
Y
Y
PATH OPERATIONAL
Y
Y
Y
Y
MANAGED
N
N
N
N
CU NUMBER
0700 0700 0700 0700
MAXIMUM MANAGED CHPID(S) ALLOWED: 0
DESTINATION CU LOGICAL ADDRESS = 07
SCP CU ND
= 002107.000.IBM.TC.03069A000007.00FF
SCP TOKEN NED
= 002107.900.IBM.TC.03069A000007.0700
SCP DEVICE NED
= 002107.900.IBM.TC.03069A000007.0710
HYPERPAV ALIASES IN POOL 4
D M=DEV(dddd) where dddd is a base

volume in a HyperPAV LSS
92
HyperPAV D M=DEV
SY1
D M=DEV(0718)
SY1
IEE174I 23.39.07 DISPLAY M 838
DEVICE 0718
STATUS=POOLED HYPERPAV ALIAS
D M=DEV(dddd) where dddd is an alias

device in a HyperPAV LSS
93
HyperPAV D M=DEV

94
SY1 d m=dev
SY1 IEE174I 23.42.09 DISPLAY M 844
DEVICE STATUS: NUMBER OF ONLINE CHANNEL PATHS
0 1 2 3 4 5 6 7 8 9 A B C D E F
000 DN 4 DN DN DN DN DN DN DN . DN DN 1 1 1 1
018 DN DN DN DN 4 DN DN DN DN DN DN DN DN DN DN DN
02E 4 DN 4 DN 4 8 4 4 4 4 4 4 4 DN 4 DN
02F DN 4 4 4 4 4 4 DN 4 4 4 4 4 DN DN 4
030 8 . . . . . . . . . . . . . . .
033 4 . . . . . . . . . . . . . . .
034 4 4 4 4 DN DN DN DN DN DN DN DN DN DN DN DN
03E 1 DN DN DN DN DN DN DN DN DN DN DN DN DN DN DN
041 4 4 4 4 4 4 4 4 AL AL AL AL AL AL AL AL
048 4 4 DN DN DN DN DN DN DN DN DN DN DN DN DN 4
051 4 4 4 4 4 4 4 4 UL UL UL UL UL UL UL UL
061 4 4 4 4 4 4 4 4 AL AL AL AL AL AL AL AL
071 4 4 4 4 DN DN DN DN HA HA DN DN . . . .
073 DN DN DN . DN . DN . DN . DN . HA . HA .
098 4 4 4 4 DN 8 4 4 4 4 4 DN 4 4 4 4
0E0 DN DN 1 DN DN DN DN DN DN DN DN DN DN DN DN DN
0F1 1 DN DN DN DN DN DN DN DN DN DN DN DN DN DN DN
FFF . . . . . . . . . . . . HA HA HA HA
************************ SYMBOL EXPLANATIONS ************************
@ ONLINE, PHYSICALLY ONLINE, AND OPERATIONAL INDICATORS ARE NOT EQUAL
+ ONLINE
# DEVICE OFFLINE
. DOES NOT EXIST
BX DEVICE IS BOXED
SN SUBCHANNEL NOT AVAILABLE
DN DEVICE NOT AVAILABLE
PE SUBCHANNEL IN PERMANENT ERROR
AL DEVICE IS AN ALIAS
UL DEVICE IS AN UNBOUND ALIAS
HA DEVICE IS A HYPERPAV ALIAS
D M=DEV shows HA for HyperPAV aliases
WLM and HyperPAV

WLM Dynamic Alias Tuning
Ignores
HyperPAV control units
Avoids
sysplex communications for devices on

HyperPAV control units
Eliminates need for multi-system interlock (DST)
Mixed
environment tolerated (within a system and

within a sysplex)
Manages non-HyperPAV aliases for all systems in
the sysplex
Control units in mixed mode (some systems in
HyperPAV, some in Base-PAV mode) only
manages aliases in Base-PAV mode
95
HyperPAV Summary
Equal or better performance than Original PAV feature
Requires
z/OS
1.6+ (with PTFs installed and set-up completed)
DS8000
PAV License activated
DS8000
HyperPAV license activated
FICON
attachment (any supported speed)
No changes to an existing DS8000 configuration required

Full co-existence with Original DS8000 PAV (static or dynamic)
and sharing z/OS without HyperPAV enabled.
Allows
for Migration to HyperPAV to flexibly proceed
"z/OS MVS Setting Up a Sysplex (SA22-7625)" publication updated

with migration and usage information.
96
DS8000 High Performance FICON (zHPF)
97
High Performance FICON (zHPF)

Improve FICON Scale, Efficiency and RAS
As the data density behind a CU and device increase, scale I/O rates
and bandwidth to grow with the data
Significant improvements in I/O rates for OLTP (small block transfer)
Improved I/O bandwidth
New ECKD commands for improved efficiency
Improved first failure data capture
Additional channel and CU diagnostics for MIH conditions
Value
Reduce the number of channels, switch ports, control unit ports and
optical cables required to balance CPU MIPS with I/O capacity
Reduce elapsed times (DB2, VSAM) 2X
Requirements

98
z10 microcode update for FICON Express 4 channel

DS8000 R4.0
99
For zOS, there is RMF

For Open Systems, there is TPC for Disk.
100
Collecting the Data with TPC
SC33-7990 Resource Measurement Facility (RMF) Report Analysis
101

DS8000 Performance Reports
102
103

TPC Key Performance Metrics

104
IO Rates and Response Times by Volume

Cache Behavior: Read Hit Percentages
Write Cache Behavior: Write Delays
Disk/Array Performance: Rates, Resp Times, Utilization
Port Behaviors: Data rates, Port Response Times
105
106
DS8000 RMF Performance Monitoring

APAR OA06476 provides support for 2107 RMF Reporting
Monitor I
Cache
Subsystem Activity Data
Type 74 Subtype 5
Device
Activity Data
Type 74 Subtype 1
ESS
Data
Type 74 Subtype 8
I/O
Queuing Activity Data
Type 78 Subtype 3
107
Collecting the Data

Monitor I Session Options in ERBRMF00
CACHE
DEVICE
DEVICE(DASD) is the default
ESS
NOESS is the default!
Options
ESS(LINK | NOLINK | RANK | NORANK)
LINK and RANK are the defaults
IOQ
IOQ(DASD) is the default
108
RMF Reporting
REPORTS(CACHE(options))
CACHE(SUMMARY)
CACHE(SUBSYS)
CACHE(DEVICE)
REPORTS(DEVICE(DASD))
REPORTS(ESS)
Link statistics
Extent Pool statistics
Rank statistics
REPORTS(IOQ)
109
RMF Cache Subsystem Summary Report
110
RMF Cache Subsystem Report
111
RMF Cache Device Report
112
RMF ESS Link Statistics
113
RMF ESS Extent Pool Statistics
114
DS8000 ESS Rank Statistics also available for FB Ranks
115
RMF Device Activity Report
If the PAV being used is a HyperPAV,

the number will have an H after it
116
RMF I/O Queuing Activity

LCU
CU
0 0003
0070
0 000D
1000
0 000E
DCM GROUP
MIN MAX DEF
1100
1
0 000F
117
1200
CHAN
PATHS
68
*
90
91
92
94
*
10
11
3
*
90
91
92
94
*
CHPID
TAKEN
0.482
0.482
5.015
5.101
5.162
% DP
BUSY
0.00
0.00
0.00
0.00
0.00
% CU
BUSY
0.00
0.00
0.00
0.00
0.00
AVG
CUB
DLY
0.0
0.0
0.0
0.0
0.0
AVG
CMR
DLY
0.0
0.0
0.0
0.0
0.0
15.278
9.423
8.755
9.399
27.578
3.667
3.526
3.659
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
10.852
0.00
0.00
0.0
0.0
DELAY
Q
LNGTH
AVG
CSS
DLY
0.000
0.00
0.0
OFFLINE
0.000
0.00
0.2
0.000
0.00
0.4
OFFLINE
0.000
0.00
0.2
CONTENTION
RATE
HPAV
WAIT MAX
0.000
0.000
RMF Monitor II & III

Monitor II (Updated for HyperPAV)
SMF Type 79 Subtype 9

Device Report
IOQ Report
Monitor III (Updated for HyperPAV)
118
Job Delays
Device Resource Delays
Data Set Delays
Storage Resource Delays
Tivoli Productivity Center 4.1
119
Tivoli Productivity Center 4.1

Tivoli Integrated Portal
SSPC for DS8000
Custom Reporting
Storage Resource Agents
Disk Performance Optimization
Storage Resource Groups
TPC-R embedded in the TPC
installation process
120
SSPC
1. Storage portal for configuration & management
2. Centralized server reduces the need to install,

manage & administer multiple servers
3. Ease of deployment by shipping pre-loaded
Administrator points browser at SSPC

for enterprise storage view of multiple
devices
Pre-loaded Software:
IBM TPC Basic Edition
SVC Admin Console
DS8K Storage Manager linkage
DS3K, DS4K, DS5K Storage Manager
TS3500 Tape Specialist linkage
IBM SSPC
4. TPC Basic Edition provides

Server, disk & tape asset & capacity reporting
Contextual, topology viewer
5. Easy Migration path to storage mgmt. suite

Disk performance reporting & trend analysis
SAN management
Storage resource management (SRM)
Replication management
121
DS8000
SVC
DS3000
DS4000
DS5000
TS3500
TS3310
Tivoli Productivity Center

Simplified storage management
Cross-platform
DS8000 automated management
Increase efficiency
Lower costs
Integrated management options
Storage Area Network Fabric
Data
Manage data replication
122
TPC Disk Manager

Manages Storage Subsystems
Connected via SMI-S Provider (CIM Agent)
Volume Management
List, create, & remove volumes
SAN Planner
Provides policy and performance based guidance in
configuring subsystem volumes and assigning the
volumes to hosts.
Monitoring
Create groups of storage devices
View job status
Create Performance Monitor tasks
Alerts
Create storage subsystem alerts
Subsystem, Disk, Volume, Port, etc
Policy Management Workload Profiles
Used by SAN Planner to define disk I/O samples for
performance.
Reports
Subsystem, Volume, Disk, Association
123
Performance Reports
TPC Automatic charts
124
Extract TPC metrics into CSV file
125
Example of TPC bulk metrics output
126
SGA07 - Storage Subsystem Performance, Monitoring

and Capacity Planning for Open Systems
Learn both theoretical foundations in storage
performance as well as specific monitoring techniques
using IBM TotalStorage Performance Center (TPC). The
course discusses essential performance characteristics
of cached disk subsystems, the essential performance
metrics, and enough theory to help understand why
storage products perform as they do. Moreover, the
course covers the practical use of TPC to monitor
performance, to spot performance issues, and to
investigate the causes. Specific TPC reports and
interpretation of the reports are covered, as well as
application of the data to long term capacity planning.
The students will have the opportunity to run through a
variety of hands on exercises with TPC as well.
http://www304.ibm.com/jct03001c/services/learning/ites.wss/us/en?pageType=course_search&sortBy=5&searchType=1&sortDirection=9&in
cludeNotScheduled=15&rowStart=0&rowsToReturn=20&maxSearchResults=200&language=en&country=us&searchString=sga07
127
DS8000 References
DS8000 Information Center
http://publib.boulder.ibm.com/infocenter/dsichelp/ds8000ic/index.jsp

Tutorials
Overviews
Publications and much more!
GC35-0515
GC26-7914
SC26-7917
SG24-6786
SC26-7917
DS8000 Introduction & Planning

DS8000 Messages Reference
DS8000 Host Systems Attachment Guide
DS8000 Architecture & Implementation
DS8000 Command-Line Interface Users Guide
The above publications can be found on the DS8000 Information Center web site!
DS8000 Code Bundle Information (Code Bundle, DS CLI, Storage Manager xref
http://www-01.ibm.com/support/docview.wss?uid=ssg1S1002949&rs=555
DS8000 Turbo Information (specs, white papers, etc.)
http://www-03.ibm.com/systems/storage/disk/ds8000/index.html
128
DS8000 References
Techdocs
http://w3-03.ibm.com/support/techdocs/atsmastr.nsf/Web/TechDocs
PRS3574

WP101528
PRS3565
TD104162
TD103689
IBM DS8000 + System z Synergy - March 2009

IBM System z & DS8000 Technology Synergy
ATU - Storage Perf Mgmt with TPC
Open System Storage Performance Evaluation
Pulling TPC Performance Metrics for Archive and Analysis
Many more white papers, presentations and trifolds can be found on Techdocs!
129
Trademarks
The following terms are trademarks of International Business Machines Corporation in the United
States, other countries or both.
AS/400, DS6000, DS8000, DS Storage Manager, Enterprise Storage Server, FICON, FlashCopy,
GDPS, IBM, iSeries, pSeries, RS/6000, RMF, IBM TotalStorage, VM/ESA, VSE/ESA, xSeries, z/OS,
zSeries, z/VM, On Demand Business
Intel and Pentium are trademarks of Intel Corporation in the United States, other countries, or both.
Microsoft, Windows and Windows NT are trademarks of Microsoft Corporation in the United States,
other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.
130
Disclaimer
Copyright 2004 by International Business Machines Corporation.
No part of this document may be reproduced or transmitted in any form without written permission from IBM Corporation.
Product data has been reviewed for accuracy as of the date of initial publication. Product data is subject to change without notice. This
information could include technical inaccuracies or typographical errors. IBM may make improvements and/or changes in the product(s)
and/or programs(s) at any time without notice. Any statements regarding IBM's future direction and intent are subject to change or
withdrawal without notice, and represent goals and objectives only.
References in this document to IBM products, programs, or services does not imply that IBM intends to make such such products,
programs or services available in all countries in which IBM operates or does business. Any reference to an IBM Program Product in this
document is not intended to state or imply that only that program product may be used. Any functionally equivalent program, that does
not infringe IBM's intellectually property rights, may be used instead. It is the user's responsibility to evaluate and verify the operation of
any on-IBM product, program or service.
THE INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED.
IBM EXPRESSLY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR
NONINFRINGEMENT. IBM shall have no responsibility to update this information. IBM products are warranted according to the terms and
conditions of the agreements (e.g., IBM Customer Agreement, Statement of Limited Warranty, International Program License Agreement,
etc.) under which they are provided. IBM is not responsible for the performance or interoperability of any non-IBM products discussed
herein.
The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents or
copyrights. Inquiries regarding patent or copyright licenses should be made, in writing, to:
IBM Director of Licensing
IBM Corporation
North Castle Drive
Armonk, NY 10504-1785
U.S.A.
131

Ds8000 Architecture Configuration

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ds8000 Architecture Configuration

Uploaded by

Copyright:

Available Formats

IBM Systems & Technology Group

IBM TotalStorage DS8000 Architecture & Configuration

2005 IBM Corporation

IBM Systems and Technology Group

Volumes with z/OS

Performance FICON (zHPF)

Productivity Center 4.1

IBM Systems and Technology Group

DS8000 Physical Overview

IBM Systems and Technology Group

DS8000 Physical Overview

GB, 300 GB, 450 GB switched

TB SATA 7,200 RPM

& 146 GB Solid State Drive (SSD)

available on 146 GB, 300 GB

FC/FICON Host Adapter cards

Adapters scale with capacity

devices; 255 Logical Control Units

logical DS8000s in one physical

and Global Mirror compatibility

IBM Systems and Technology Group

DS8000 Physical Overview D8S100 & DS8300

DS8300 (932, 9B2)

Shared SMP processor configuration

POWER5+ dual 2-way

POWER5+ dual 4-way

Other major processors

IBM Virtualization Engine (LPAR) capability

Processor memory for cache and NVS (min/max)

Host adapter interfaces

4-port 4 Gbps or 2 Gbps Fibre

4-port 4 Gbps or 2 Gbps Fibre

Host adapters (min/max)

Host ports (min/max)

Number of disk drives (min/max)

Up to eight 4-port FC-AL

Maximum physical storage capacity**

Number of expansion frames

IBM Systems and Technology Group

DS8100 931 Maximum Configuration

IBM Systems and Technology Group

DS8100 (931) Disks and Device Adapters (DAs)

Server0 and Server1

IBM Systems and Technology Group

DS8300 932/9B2 with 2 Expansion Frames

IBM Systems and Technology Group

DS8300 Disks and Device Adapters (DAs)

increments(2 disk enclosure pairs) per DA

DA6, DA4, DA7, DA5

DA3, DA1, DA2, DA0

DA6, DA4, DA7, DA5

to 128 FICON/FCP host ports

ports in base frame

ports in expansion frame

IBM Systems and Technology Group

ports per adapter *

x indicates DS8000 I/O enclosure (0-7)

* There are 2 ESCON ports per adapter

IBM Systems and Technology Group

DS8000 Disk Drives

146 GB, 15K RPM drive

300 GB, 15K RPM drive

450 GB, 15K RPM drive

One Serial Advanced Technology Attachment (SATA) DDM drive type: