Unit 2:

IBM Advanced POWER Virtualization

(APV) and Oracle

© Copyright IBM Corporation 2013

Agenda
• Advanced POWER Virtualization (APV) overview

• Hardware partitioning support

• Virtual Ethernet support

• Virtual SCSI support

© Copyright IBM Corporation 2013

IBM Virtualization Engine components on p7

• POWER Hypervisor
• Multiple operating system
support
• Micro-partitioning
• Capacity on Demand
• Simultaneous multi-
threading
• Virtual LAN
• Virtual I/O

© Copyright IBM Corporation 2013

POWER Hypervisor
• Enforces partition security
• Provides time slicing and dispatching of logical partition
workloads across physical processors according to partition
profiles
• Provides virtual LAN channels between physical partitions
• Supports partitioning and dynamic resource movement across
multiple operating system environments.

© Copyright IBM Corporation 2013

Multiple operating system support
• The IBM Power Systems support AIX 5.3 and newer versions,
i5/OS, and Linux distributions from SuSE Linux and RedHat
Linux.
• Some Power System s machines have a minimum AIX level
of 6.1 with a particular technology level.

© Copyright IBM Corporation 2013

Micro-partitioning
• Enables a logical partition to be allocated a percentage of processor
usage rather than a full physical processor.
• Multiple partitions may share CPU resources from a processor pool
– Partitions may be defined with as little as 1/10th of a processor
entitlement
– Shared processor resources may be dynamically ceded by one
partition and used by another partition
– By default, all processors not used by active dedicated partitions
are available for use in the shared processor pool
• May be overridden by disabling the “Allow idle processor to be shared” partition setting

• May dynamically increase/decrease processor entitlement, number of

virtual processors and/or memory
• Multiple virtual shared processor pools are available with POWER6 and
POWER7 technology.
© Copyright IBM Corporation 2013
Micro-partition specifications
• Virtual processors
– Minimum, desired, and maximum number of virtual processors
• Processor entitlement
– Minimum, desired, and maximum processor units of capacity
– Entitled capacity = the activated capacity for a partition
– May range from .1 up to the number of virtual processors in increments of .01
• Processor sharing mode
– Capped: Partition usage may never exceed the partition’s entitlement
– Uncapped: Partition usage may exceed the partition’s entitlement if additional resources are
available in the shared processor pool
• Uncapped weight
– For uncapped partitions, specifies the relative weight for prioritizing excess resources in the
shared processor pool
– 0 (Zero): indicates t he partition is capped
– 1-255: indicates weighted priority (1 is lowest)
© Copyright IBM Corporation 2013
POWER7 processor architecture

© Copyright IBM Corporation 2013

Capacity on Demand
• Allows system resources, such as processors and memory, to
be activated as needed.
• Utility CoD is available on POWER6 and POWER7
processor-based systems and automates the usage of CoD
processors.
• consider the software license to install on activated Capacity
on Demand (CoD) resources

© Copyright IBM Corporation 2013

Capacity on demand options
• Capacity Upgrade on Demand(CUoD)
• Enables permanent system upgrade by activating pre-installed processors
and/or memory
• Purchase agreement is activated using activation code
• On/Off Capacity on Demand
• Enables temporary use of pre-installed processors and/or memory
• Capacity can be turned ON and OFF by the customer
• Customer is charged at activation time
• Usage information is reported to IBM
• Utility Capacity on Demand
• Used for processors only
• Reserve processor capacity added to shared processor pool if base capacity
exceeded
• Trial Capacity on Demand
• Test effects of additional processors and/or memory
• Partial or total activation of installed processors and/or memory
• Resources must be returned after trial period

© Copyright IBM Corporation 2013

Simultaneous multi-threading(SMT)
• SMT technology allows
two or four separate
instruction streams (threads)
to run concurrently on the
same physical processor
• Each hardware thread
treated as an independent
logical processor.
• Any partition can be booted
in single thread (ST) or
simultaneous multi-
threading (SMT) mode
• May dynamically switch
between ST and SMT mode
using smtctl command

© Copyright IBM Corporation 2013

SMT benefits
• Enhancements in the IBM Power processors design allow for
improved overall hardware resource utilization.
• SMT provides incremental throughput capacity for most
commercial workloads
• For Oracle DB workloads, an increase of 30% or more is
typical
• SMT generally will not improve response times unless there is
latent CPU demand due to capacity constraints

© Copyright IBM Corporation 2013

SMT and shared processors
• Simultaneous multithreading (SMT) Power 7
can be used with shared
processors. LPAR 1
• With simultaneous multithreading,
each virtual processor runs two Logical
threads (POWER6) or four threads Processor L L L L L L L L
(POWER7):
– Each thread is called a logical Virtual
Processor V V
processor.
• LPAR1 example:
– 1.6 processing units
– Two virtual processors Shared
– Simultaneous multithreading processor
pool
enabled:
• Eight logical processors Physical
Processor P P P P

© Copyright IBM Corporation 2013

Virtual Ethernet adapters
• Virtual Ethernet enables inter-partition communication
without the need for physical network adapters assigned to
each partition.
• slots, virtual Ethernet is a convenient and cost-saving option
to enable that the number of LPAR can be greater than the
number of physical I/O
• LPAR within a single system to LPAR 1 LPAR 2 LPAR 3 LPAR 4

en0 en0 en en en
0
en
0 1 1
communicate with one another (if) (if) (if) (if) (if) (if)

ent ent
through a virtual Ethernet 0 0
ent
0
(vir
ent
1
(vir
ent
0
(vir
ent
1
(vir
t) t) t)
(virt) (virt) t)

LAN(VLAN).
VLAN 2
VLAN 1
Power Hypervisor

© Copyright IBM Corporation 2013

Virtual LAN
• A function of the POWER Hypervisor.
• Virtual LANs allows secure communication between logical
partitions without the need for a physical I/O adapter.
• The ability to securely share
Ethernet bandwidth across LPAR 1 LPAR 2 LPAR 3 LPAR 4

en0 en en en0 en en

multiple partitions increases (if)

0
(if)
1
(if) (if)
0
(if)
1
(if)

hardware utilization. ent

0
ent
0
(vir
ent
1
(vir
ent
0
ent
0
(vir
ent
1
(vir
t) t) t)
(virt) t)
(virt)

VLAN 2
VLAN 1
Power Hypervisor

© Copyright IBM Corporation 2013

Virtual I/O
• Virtual I/O allows a single physical I/O adapter or endpoint
device (disk or media device) to be used by multiple logical
partitions of the same server.
• File-backed devices are available with VIOS version 1.5.
• This facilitates consolidation of I/O resources and
minimization of the number of I/O adapters required.

© Copyright IBM Corporation 2013

Oracle Licensing Cost Metrics
• The following licensing metrics are available:
• Unlimited License Agreements
• A Unlimited License Agreements (ULA) is a time-based (2-3 year) unlimited
use right license for a subset of Oracle technology or applications.
• Processor licensing
• pay per Processor you run the Oracle software on
• Standard Edition Per-socket licensing
• Standard Edition or Standard Edition One on a 2 processor system need 2
licenses.
• Enterprise Edition Per-core licensing
• Multi-core processors are priced as (number of cores)*(multi-core factor)
processors
• Named User Plus Licensing
• Pay per user, however user is defined as any end-node that receives data from
or creates data for that Oracle DB including human users

© Copyright IBM Corporation 2013

Oracle licensing considerations
• General
• Oracle processor licensing based on physical core count
• Up to 4 socket servers (for example, 16-way p560Q) are eligible for Standard Edition
• All dual-core and quad-core System p configurations qualify for .75 processor factor
• SMT (on/off) does not affect Oracle licensing

• Dedicated Processor LPAR/DLPAR

• License the maximum number of processors in the LPAR
• When multiple Oracle LPARs on a server, cannot exceed # dedicated physical processors on the server

• Shared Processor Partitions

• Capped partitions: License based on the entitled partition capacity
• Uncapped partitions: License based on the maximum number of virtual processors in the partition
• When multiple Oracle shared processor partitions on a server, cannot exceed # processors in the shared
processor pool

© Copyright IBM Corporation 2013

License detection
• To see (some of) the components that should be licensed:
• Number of users and CPU/Processors:

select * from v$license;

Note: the information provided by this command is not trustworthy on multi-threaded,

multi-core or virtual systems. It is the number of processors understood to be available by the
database - not the number of processors which must be licensed.

© Copyright IBM Corporation 2013

VIO support for Oracle environments
• Single instance (Non-RAC)
• All Virtual Ethernet and Virtual SCSI configuration options are fully
supported
• RAC
• Non-RAC related components are supported:
– VSCSI access to rootvg or non-shared ORACLE_HOME
• Virtual Ethernet for public network is supported
• Virtual Ethernet for private interconnect network is supported
• Virtual SCSI for ASM (plus Voting and OCR) disks is supported
• RAC with GPFS support is planned

© Copyright IBM Corporation 2013

Agenda
• Advanced POWER Virtualization (APV) overview

• Hardware partitioning support

• Virtual Ethernet support

• Virtual SCSI support

© Copyright IBM Corporation 2013

POWER7 VIO server switch concepts
Traditional Switches Virtual I/O Server Switch

AIX Client
AIX LPAR
en0
(if) en0
(if)

POWER7 Server
ent0 ent0
(virt)

VIOS ent1
Port
(virt)

Logic ent2
(SEA)

ent0
Port
(phy)

To other device or switch To other switch

• POWER7 VLAN support is based on IEEE 802.1Q VLAN

specification

© Copyright IBM Corporation 2013

Virtual I/O network terms
Shared
Ethernet Virtual I/O Server(VIOS) AIX Client LPAR 1 AIX Client LPAR 2
Adapter
(Acts as a
Layer 2
bridge)

ent3 ent4
Interface
en0 en1 en0 en1
(LA) (SEA)
Link
Aggregation
Adapter Virtual
ent1 ent0 ent2 ent0 ent1 ent0 ent1
(Combines Ethernet
(phy) (phy) (virt) (virt) (virt) (virt) (virt)
Physical Adapter
Adapters)

PVID=2 PVID=1 PVID=1 PVID=2

Physical
Ethernet
Adapter PVID=2
Hypervisor

IEEE 802.3ad Link Aggregation (LA)

Virtual Ethernet
Cisco EtherChannel (EC)

Path Virtual ID(PVID)

IEEE 802.1Q Virtual LAN (VLAN)

Ethernet Switch

© Copyright IBM Corporation 2013

Virtual Ethernet options: Single VIOS; single LAN

• Complexity
– Easy to set up and manage Power7 Server

– No specialized configuration on switch AIX Client

– No specialized configuration on client

Virt
• Resilience Enet

– VIOS Ethernet adapter, switch port and

switch are single points of failure VIOS 1
VIO
• Throughput / Scalability Ethernet
– Performance limited to a single Ethernet Switch
adapter
Enet
PCI

© Copyright IBM Corporation 2013

Virtual Ethernet options details: Single VIOS; single
LAN segment
AIX Client LPAR 1 AIX Client LPAR 1

en en
0 0
(if) (if)
ent ent
0 0
(vir (vir
t) t)

PVID=2 PVID=2

VLAN=2

PVID=2 Hypervisor

ent
VIOS 1 1
(vir
t)
ent
2
(SE
A)
ent
0
(ph
y)
Untagged Port

Ethernet Switch

© Copyright IBM Corporation 2013

Virtual Ethernet options: AIX network interface
backup (NIB); dual VIOS
• Complexity
– Requires specialized setup on client (NIB)
– Needs to ping outside host from the client
Power7 Server
to initiate NIB failover
• Resilience AIX Client

– Protects against single VIOS / switch port /

Virt Virt
switch / Ethernet adapter failures Enet Enet

• Throughput / Scalability NIB

VIOS 1 VIOS 2
– Allows load-sharing between VIOS’s
VIO VIO
• Notes Ethernet Ethernet
Switch Switch
– NIB does not support tagged VLANs on
physical LAN
Enet Enet
– Must use external switches not hubs PCI PCI

© Copyright IBM Corporation 2013

Virtual Ethernet options details: AIX network
interface backup (NIB); dual VIOS
AIX Client LPAR 1 AIX Client LPAR 1
en en
2 2
(if) (if)
ent ent
2 2
NI NI
(LA) ent (LA)
ent
B ent entB
1 0 0 1
(vir (vir (vir (vir
t) t) t) t)
PVID=2 PVID=3 PVID=2 PVID=3 Hypervisor
VLAN=2

PVID=2 VLAN=3 PVID=3

ent ent
VIOS 1 1 1 VIOS 2
(vir (vir
t) t)
ent ent
2 2
(SE (SE
A) A)
ent ent
0 0
(ph (ph
y) y)
Untagged Port Untagged Port

Ethernet Switch Ethernet Switch

© Copyright IBM Corporation 2013

Virtual Ethernet options: AIX network interface
backup (NIB); dual VIOS with Link Aggregation
(LA)
• Complexity
– Requires specialized setup on client (NIB)
– Requires link aggregation setup on external
switches
Power7 Server
– Needs to ping outside host from the client to
initiate NIB failover
AIX Client
• Resilience
– Protection against single VIOS/ switch port / Virt Virt
switch / Ethernet adapter failure Enet Enet
– Protection against adapter failures within VIOS NIB
• Throughput / Scalability VIOS 1 VIOS 2
– Allows each client to use a different primary VIO
VIOS sharing network load across multiple VIO
VIOS’s. Ethernet Ethernet
Switch Switch
– Potential for increased bandwidth with LA
• Notes Enet Enet Enet Enet
– NIB does not support tagged VLANs on PCI PCI PCI PCI
physical LAN
– Must use external switches not hubs

© Copyright IBM Corporation 2013

Virtual Ethernet options details: AIX network interface
backup (NIB); dual VIOS with Link Aggregation
AIX Client LPAR 1 AIX Client LPAR 1
en2 en2
(if) (if)

ent2 ent2
(LA) (LA)
NIB NIB

ent1 ent0 ent0 ent1

(virt) (virt) (virt) (virt)

PVID=2 PVID=3 PVID=2 PVID=3 Hypervisor

VLAN=2

PVID=2 VLAN=3 PVID=3

VIOS 1 ent3 ent3 VIOS 2

(virt) (virt)

ent2 ent4 ent4 ent2

(LA) (SEA) (SEA) (LA)

ent0 ent0 ent0 ent1

(phy) (phy) (phy) (phy)

Untagged Port Untagged Port

Ethernet Switch Ethernet Switch

© Copyright IBM Corporation 2013

Agenda
• Advanced POWER Virtualization (APV) overview

• Hardware partitioning support

• Virtual Ethernet support

• Virtual SCSI support

© Copyright IBM Corporation 2013

© Copyright IBM Corporation 2013
Virtual SCSI basic architecture
AIX Partition Virtual I/O Server

vSCSI Target Device

PV LV
VSCSI Optical
VSCSI VSCSI

LVM

Hdisk
DVD Multi-Path
Optical
or
Disk Drive Driver

vSCSI
Adapter / Adapter /
Client Drivers
Adapter Drivers

POWER7 Hypervisor

FC or SCSI
Device

© Copyright IBM Corporation 2013

Virtual SCSI options: Single VIOS; PV VSCSI disks

• Complexity
– Simpler to setup an d manage than dual VIOS
– No specialized setup on the client VIOS 1
• Resilience
AIX A
– VIOS, SCSI adapter, SCSI disk are potential single
points of failure
– The loss of a single physical client disk will affect AIX B
only that client
• Throughput / Scalability vSCSI
– Performance limited by single SCSI adapter and
internal SCSI disks.

A B

© Copyright IBM Corporation 2013

Virtual SCSI options details: Single VIOS; PV VSCSI disks

AIX Client LPAR A AIX Client LPAR B

hdisk0 hdisk0

vscsi0 vscsi0

Hypervisor

vhost0 vhost1

vtscsi0 vtscsi0

VIOS 1 scsi0

A disk B disk

© Copyright IBM Corporation 2013

Virtual SCSI options: Single VIOS; LV VSCSI disks

• Complexity
– Simpler to setup an d manage than dual VIOS
– No specialized setup on the client
VIOS 1
• Resilience
– VIOS, SCSI adapter, SCSI disk are potential single AIX A
points of failure
– The loss of a physical disk may impact more than one
client AIX B
• Throughput / Scalability vSCSI
– Performance limited by single SCSI adapter and internal
SCSI disks.
• Notes
– Low cost disk alternative

© Copyright IBM Corporation 2013

Virtual SCSI options details: Single VIOS; LV VSCSI disks

AIX Client LPAR A AIX Client LPAR B

hdisk0 hdisk0

vscsi0 vscsi0

Hypervisor

vhost0 vhost1

vtscsi0 vtscsi0

LV LV

VIOS 1 scsi0

A disk

B disk

© Copyright IBM Corporation 2013

Virtual SCSI options: Single VIOS with multi-
path I/O
• Complexity
VIOS 1
– Simpler to setup an d manage than dual VIO
servers AIX A
– Requires multi-path I/O setup on the VIOS
– No specialized setup on the client
AIX B
• Resilience
vSCSI
– VIOS is a single point of failure
• Throughput / Scalability MPATH*
– Potential for increased bandwidth due to multi-
path I/O.
– Could divide clients across independent VIOS
allowing more VIOS adapter bandwidth. FC SAN

A
B
PV LUNs

© Copyright IBM Corporation 2013

Virtual SCSI options details: Single VIOS with
multi-path I/O
AIX Client LPAR A AIX Client LPAR B

hdisk0 hdisk0

vscsi0 vscsi0

Hypervisor

vhost0 vhost1

vtscsi0 vtscsi1

Multi-Path Driver

VIOS 1 fcs0 fcs1

A disk

B disk
PV LUNs

© Copyright IBM Corporation 2013

Virtual SCSI options: AIX MPIO default PCM
driver in client; single path in VIOS
• Complexity
– Simplest dual VIOS FC option VIOS 1 VIOS 1
AIX A
– Requires MPIO to be setup on the client (MPIO Default
PCM)
• Resilience
– Protection against failure of a single VIOS / FC AIX B
adapter (or path) (MPIO Default
PCM)
• Throughput / Scalability vSCSI vSCSI
– Primary LUNs can be split across multiple
VIOS to help balance the I/O load.
• Notes
– Lowest port costs of dual VIOS FC options
– Must be PV VSCSI disks.
FC SAN

A
B
PV LUNs
© Copyright IBM Corporation 2013
Virtual SCSI options details: AIX MPIO default
PCM driver in client; single path in VIOS
AIX Client LPAR A AIX Client LPAR B

hdisk0 hdisk0

MPIO Default PCM MPIO Default PCM

vscsi0 vscsi1 vscsi0 vscsi1

Hypervisor

vhost0 vhost1 vhost0 vhost1

vtscsi0 vtscsi1 vtscsi0 vtscsi1

VIOS 1 fcs0 VIOS 2 fcs0

A disk

B disk
PV LUNs

© Copyright IBM Corporation 2013