Oracle VM Server for SPARC (Logical Domains) Best Practices

Jeff Savit, Principal Technical Product Manager, Oracle

Honglin Su, Director of Product Management, Oracle

The following is intended to outline our general

product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any
features or functionality described for Oracles
products remain at the sole discretion of Oracle.

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Hardware and Software

Engineered to Work Together

Optimized and integrated for better

performance, reliability, security, management
Reduced change management risk
Lower cost of ownership

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Virtualization Product
Strategy
Making Software Easier to Deploy,
Access, Manage, & Support

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Virtualization Platform: Oracle VM

Server Virtualization and Management
For both Oracle and non-Oracle
Applications

Oracle VM Server for x86

Oracle VM Server for SPARC
Oracle VM Manager
Oracle Enterprise Manager

High performance
Integrated full-stack management
Faster application deployment
Enterprise-quality support
Free license

The Most Highly Integrated, Fully

Certified, and Fully Supported Server
Virtualization Platform for Oracle
Products

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Full Stack Management

With Oracle Enterprise Manager
Complete Cloud Lifecycle Management

Creating virtual system slices

Self-service assembly deployment
Automatic scale-up and scale-down
Metering and chargeback

Every layer
Deep, Integrated Hardware, Software
and Applications Management

Proactive Support
Phone home
Patch recommendations
Health checks

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

ORACLE VM MANAGER
SPARC

X86

Oracle VM Server for SPARC

Overview and Architecture

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Solaris and SPARC Virtualization

Better Resource Utilization for a More Efficient Datacenter
Dynamic Domains
Domain A
OLTP DB

Domain A
App

App

Domain B

Domain C

App

Web

DW DB

Web
Web

M-Series
8

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

T-Series

Oracle Zones

Web
Solaris Zone

App

DB

Solaris Zone

Domain B

Web

Solaris 9 Container

App

Oracle Solaris Zones

Solaris 8 Container

OLTP DB

Oracle VM Server
for SPARC

Oracle VM Server for SPARC

The Virtualization Platform combining the best of Oracle Solaris and SPARC for
Your Enterprise Server Workloads
Isolated OS and applications in each
logical (or virtual) domain
SPARC Solaris Logical Domains (VMs)

Firmware-based hypervisor

Each logical domain runs in

dedicated CPU thread(s)
SPARC Hypervisor

Free to download

Enterprise-quality support

Real-world deployment testing

Risk-free virtualization

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

T-Series Server

Optimized for SPARC / Oracle Solaris

Oracle VM Server for SPARC

Advanced Virtualization For SPARC T-Series Servers

Leading Price / Performance

Meet the most aggressive business requirements

Secure Live Migration

Increase application service level

SR-IOV and PCIe Direct I/O

Native I/O throughput!

Dynamic Reconfiguration

Change resources on the fly!

Advanced RAS

Higher availability across all levels

Physical-to-virtual (P2V) Conversion

Easily move legacy Solaris to virtualized environment

10

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

DR
Feature

Supported

CPU DR

Memory
DR
Virtual
I/O DR
Crypto
Unit DR

Oracle VM Server for SPARC

Advanced Virtualization For SPARC T-Series Servers (Contd)

CPU Whole Core Allocation and Core Affinity

Higher application performance!

CPU Dynamic Resource Management (DRM)

Better alignment of IT and business priorities

CPU Power Management

Greater power efficiency and lower cost!

Advanced Network Configuration

Flexibility to meet networking requirements

Enhanced SNMP MIB

Interoperate with 3rd party management software

Official Certification and Full Stack Support

Cover hardware, firmware, virtualization, OS and the software stack

11

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Oracle VM Server for SPARC

Best Practices

12

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Typical Simple Configuration

One domain with multiple roles - control, service and I/O as
the primary domain
Services in the primary domain:
A virtual switch (vsw) associated with the primary NIC
A virtual disk service (vds) exporting vdisk for all guests
A virtual console concentrator (vcc)

Other domains are guests with vnets and vdisks

serviced by the primary domain
Guests consoles are available through the primary domain
Primary console is available through the SP
Life cycle of a domain: define it, bind resources to it, start it
2012 Oracle

13

Simple Configuration
physical
network

vdisk backends
(file, disk, volume)

PCI buses
primary domain
vsw0

control, service, I/O domain

e1000g0

vsw

vds

ldm

vntsd

ldmd

vcc

Hypervisor

vnet

vdc

guest domain 1

2012 Oracle

14

vnet

vdc

guest domain 2

vnet

vdc

guest domain 3

vnet

vdc

guest domain 4

Domain Configuration Best Practices

Separation of function:
Applications normally run in guest domains, using virtual I/O devices
Applications can also run in I/O domains but must choose:
Benefit: native I/O performance of physical devices
Restriction: limited number of them; cannot live migrate
Applications should not run in service domains: instability or resource
demands from apps could affect client guest domains
Applications should generally not run in control domain, for security reasons
and the reasons for service domains.

Availability: use multiple service domains for redundant

access to multi-path I/O and avoid single point of failure.
Except if there are loose availability requirements, if just getting started.
Consider as adjunct for applications that provide their own HA framework (eg:
Oracle RAC, Weblogic)
2012 Oracle

15

Domain Configuration Best Practices

Separation of function:
Applications normally run in guest domains, using virtual I/O devices
Applications can also run in I/O domains but must choose:
Benefit: native I/O performance of physical devices
Restriction: limited number of them; cannot live migrate
Applications should not run in service domains: instability or resource
demands from apps could affect client guest domains
Applications should generally not run in control domain, for security reasons
and the reasons for service domains.

Availability: use multiple service domains for redundant

access to multi-path I/O and avoid single point of failure.
Except if there are loose availability requirements, if just getting started.
Consider as adjunct for applications that provide their own HA framework (eg:
Oracle RAC, Weblogic)
2012 Oracle

16

Virtualized devices
Virtual devices abstract underlying physical devices or can
exist without a matching physical device
Virtual devices are :

CPUs
Memory
Modular Arithmetic Units (crypto accelerators)
Network switches and NICs
Disk servers and disks
Consoles

Domain configuration can be changed while domain is

running without requiring a domain or application reboot
17

Virtual I/O (VIO)

Logical Domains abstracts underlying IO resources
Not always possible to provide domains direct access to the IOMMU
or the device and its registers
Exceptions provided by SR-IOV (single root I/O virtualization) and
STDIO (static direct I/O) which permits direct attachment of a device
to a guest with native performance for qualified devices

LDoms VIO infrastructure

provides indirect access to domains via virtualized devices that
communicate with the 'service' domain
Service domain owns a device and functions as a proxy

Implemented as a client-server model between domains via

Logical Domain Channels (LDCs)
Provides safety, security, and permits redundancy
18

Virtualized I/O

2012 Oracle

19

Virtual Disk Server device (vds)

Virtual disk server backends to virtual disk clients:

A physical block device (disk/LUN)

A slice of a physical device or LUN
A disk image file residing in UFS, ZFS or NFS
A ZFS vol, or volume created by SVM or VxVM
CD ROM/DVD (can use for boot)

Notes:
Can mark disk exclusive (give to one domain only. Not available for
files simulate via ZFS clones), or mark it read-only
There is no advantage to using additional virtual disk servers (vds)
rather than one. Each vdisk is managed separately either way
Use mpgroup=groupname for redundancy provided by multiple
service domains having access to the backend device
20

Examples of virtual disk backends

Physical disk backends best performance:
A physical disk: /dev/dsk/c1t48d0s2
A slice of a disk: /dev/dsk/c1t48d0s0
Logical disk backends enhanced flexibility
A file in UFS, ZFS, or NFS: /path/to/file/file.name
A volume from ZFS or volume manager (SVM, VxVM):
Ex: zfs create -V 100m ldoms/domain/test/zdisk0
Then assign /dev/zvol/dsk/ldoms/domain/test/zdisk0
Networked backends can be NFS or iSCSI
Use multi-host backends for domains you want to live
migrate: NFS, iSCSI, or SAN. ZFS is not multi-host
MPXIO can be used with physical backends.
21

Virtualized I/O Redundant domain boot disks

Service
Domain

Logical
Domain

APP

APP

APP
Driver

Service
Domain
APP

APP

APP

RAID1

RAID1

vdisk vdisk

vdisk vdisk

vds
Privileged

Logical
Domain

Driver

vds

Hyper
Privileged

Hardware

/pci@780

/pci@7c0
HBA

HBA

External Storage
2012 Oracle

22

Virtual Disk Best Practices

Choose backend that meets your performance, availability,
ease of setup, and flexibility needs.
Physical (eg: SAN LUNs) have best performance
File-based backends have less performance, but easy to set
up, clone (especially with ZFS), copy.
Configure for availability:
Use disks from multiple service domains in an mpgroup
for insulation from domain or path failure
Use mirroring or RAID just as with non-virtual systems
Configure for live migration if that's desired: not on internal
disk, not on ZFS pool

23

Virtual Network
Virtual network switch a Layer 2 switch
Configure with physical adapter for off-box network access
Configure without physical adapter for within-box inter-domain network
Inter-domain traffic on same switch is direct without overhead of going
through service domain. Can optionally set inter-vnet-link=off if running
low on logical domain channels in large configurations
Multiple VLANs per box by adding virtual switches
Supports jumbo frames, VLAN tagging

Virtual NIC vnetN seen within the guest domain

Best practice: configure for availability using multiple service
domains and either link-based or probe-based IPMP in guest
With Ldoms 2.2 and Solaris 11, can use Rx Dring mode for
better performance: # ldm set-domain extended-mapin-space=on ldm1
24

Virtual Ethernet device

Logical Domain
1

Logical Domain 2

App

App

App

V-Ether
Driver

Hypervisor

App

Logical Domain 3

App
App

App

App
App

V-Ether
Driver

App

App

App

Service Domain
V-Ether
Switch
V-Ether
Switch

V-Ether
Driver

DeviceDriver

Virtual LAN 2: 63.24/16

Virtual LAN 1: 192.168.0/24

I/O Bridge

Gb
Ether I/F

2012 Oracle

25

Virtualized I/O Redundant Network via IPMP

Service
Domain
APP

Logical
Domain
APP

Logical
Domain
APP

APP
Driver

vsw
Privileged

IPMP
vnet

vnet

Service
Domain
APP
APP

IPMP

Driver

vsw

vnet vnet

Hyper
Privileged

Hardware

/pci@780

/pci@7c0
e1000g0
e1000g2

e1000g1
e1000g3

External
Network
2012 Oracle

26

Advanced I/O Virtualization

Delivering Native I/O Throughput

Support for SR-IOV (Single Root IO

Virtualization)

Directly share I/O device resources to guest

domains
Available on SPARC T4 and SPARC T3 platforms
for selected network NICs

Also support for PCIe Direct I/O

Assign individual PCIe cards to guest domains

Provide native I/O throughput maximum IO

performance in a virtual machine for the most
demanding IO intensive workloads

At this time cannot be used with live migration,

and the bus owning the PCIe cards must belong
to the control domain

27

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Memory
Memory is not over-subscribed
You can define domains with more RAM than is present, but
can only bind those that fit
Avoids double-paging/3rd level memory problems commonly
seen with hypervisors that swap virtual machine memories

Minimum allocatable unit is 4MB

Can dynamically and non-disruptively (re)allocate
RAM in a running domain on 256MB boundaries
RAM requirements in a domain are identical to
application running on bare metal

28

Virtual CPUs == threads == strands

T4: 8 cores with 8 threads per chip up to 4 chips and
single-thread performance
T3: 16 cores with 8 threads: 128 vCPUs/chip
T3-1: 128 vCPUs, T3-2: 256 vCPUs, T3-4: 512 vCPUs

T2, T2+: up to 8 cores with 8 threads each: 64 vCPUs/chip

T5120,T5220: T2 chip, 8 cores: 64 vCPUs
T5140,T5240: 2 T2+ chips,16 cores: 128 vCPUs
T5440: 4 T2+ chips, 32 cores: 256 vCPUs

Granularity is as low as 1 vCPU per domain

vCPUs are allocated to one domain at a time.
Up to 128 domains
Can be dynamically allocated with the domain running,
adding or removing a vcpu to or from a running domain

29

Virtual CPU best practices

Allocate CPUs on core boundaries for optimal performance
Either allocate in multiples of 8, or
Allocate with whole-core constraint ldm set-core N ldg1
Don't bother for tiny domains with little CPU need
All machines, virtual or not, are idle at the same speed
Especially do this for critical domains: control, I/O, service
domains, and for important apps (eg: Oracle RAC)
You can assign specific CPU and memory resources only
use when you have specific knowledge. For black belts!
On T4, you can set for single-CPU performance: not
usually needed as the system will do the right thing in
most cases anyway
30

Hardware Crypto Acceleration

Dramatic performance improvements for crypto code
T4: No need to assign a crypto unit it 'just happens'
Many operations 'in-pipe' on CPU, others run on RSA or FP units
Added: Camelia, CRC32c, more SHA-x.
Existing ciphers are further accelerated compared to T3 and earlier

T2, T2+, T3: each core has a crypto accelerator unit

Crypto unit is allocated to domains with a thread on the same core
MAU - Modular Arithmetic Unit accelerates public key
cryptography (RSA, DSA, Diffie-Hellman)
Cipher/Hash Unit (T2,T2+,T3) accelerates bulk encryption (RC4,
DES, 3DES, AES), secure hash (MD5, SHA-1, SHA-256), and
other public key algorithms (ECC - elliptical curve crypto)
T3 adds Kasumi algorithm, SHA512 and partial hash support
31

Crypto Acceleration Best Practices

T4: You don't have to do anything it 'just happens'
T2, T2+, T3: add a crypto per core for domains that do
significant amounts of crypto
Yet another reason to align domains on core boundaries

Important to add to control domains if you plan to do live

migration

32

Getting started [1]

Read docs, wikis.
Get a T-series system and apply current firmware
Install the current version of Oracle Solaris 10 or 11
some features, servers, require recent Solaris updates
or install necessary patches (if not on latest Solaris) in OVMS README

Download and install Oracle VM Server for SPARC

It is pre-installed on Solaris 11
For Solaris 10: download is a .zip file, includes Install directory with
install-ldm script, or you can do manual package install

Define initial configuration, and reboot to activate it

Create and boot your first logical domain (simple example shown)

2012 Oracle

33

Getting started [2] initial config

You now have a single domain 'primary' owning all hardware
Resize the domain, define necessary services and reboot:
# ldm add-vdiskserver primary-vds0 primary
# ldm add-vconscon port-range=5000-5100 primary-vcc0 primary
# ldm add-vswitch net-dev=nxge0 primary-vsw0 primary
# ldm set-mau 1 primary
# ldm set-vcpu 8 primary
# ldm start-reconf primary
# ldm set-memory 8g primary
# ldm add-config initial
# shutdown -y -g0 -i6
1. If Solaris 11 control domain, use vanity name for

physical NIC, eg: net0.

2. The ldm set-mau command not used on T4 and later. If used, the value should be
the number of cores allocated to the control domain.

2012 Oracle

34

Getting started [3] first domain

simple domain with a virtual NIC, disk, DVD image, 16 CPUs
# ldm add-domain ldom1
# ldm set-vcpu 16 ldom1
# ldm set-mau 2 ldom1
# ldm set-mem 16g ldom1
# ldm add-vnet vnet1 primary-vsw0 ldom1
# zfs create -o mountpoint=/ldoms rpool/ldoms
# zfs create rpool/ldoms/ldom1
# mkfile -n 10g /ldoms/ldom1/disk0.img
# ldm add-vdiskserverdevice /ldoms/ldom1/disk0.img vol10@primary-vds0
# ldm add-vdisk vdisk10 vol10@primary-vds0 ldom1
# ldm bind ldom1
# ldm list
NAME

STATE

FLAGS

CONS

VCPU

MEMORY

UTIL

UPTIME

primary

active

-n-cv-

SP

8G

2.5%

7h 48m

ldom1

bound

5000

16

16G

2012 Oracle

35

Guest Domain Installation

Jumpstart/Network Installation
Configure the jumpstart/boot server
The jumpstart/boot/AI server can be the primary domain
Boot the guest on its virtual network interface

CDROM/DVD Installation
Export a Solaris DVD or ISO image as a virtual disk
The DVD/ISO image appears as a DVD in the guest
Boot the guest on that virtual disk business as usual

Use the disk image from an already installed guest

Install a first guest on a vdisk backed by a file or volume
Copy or clone the virtual disk for other guests

Better yet: Use Ops Center to provision and

manage the whole thing
2012 Oracle

36

Advanced configuration: Using

Multiple I/O Domains
Multiple domains with direct access to hardware
Bare metal I/O performances in multiple domains
Partition a system without using virtual I/O

Benefits for guest domains:

2012 Oracle

37

Virtual devices high-availability

Virtual devices accessible through different domains
Virtual network: multipathing with IPMP
Virtual disk: mirroring with SVM or ZFS

Split PCI
PCI-E buses on T4, T3 and T5x40 can be assigned to
separate domains
provides direct I/O access to as many domains as there are buses
in the system

I/O Domains can function as service domains and export

virtual disks or provide network service to other domains.
Ensure that each bus has appropriate disk and network
devices
This is not available on T5120/T5220, which has a single
bus (hence, one control domain which is also I/O domain)

2012 Oracle

38

Typical Configuration with 2 I/O

Domains
Primary domain: Service, I/O, control domain
Alternate domain: Service, I/O domain
Both domains have:
A virtual switch
A virtual disk service

Guest domains use both service domains

A network device from each service domain with redundancy via
IPMP
A disk device from each service domain, with redundancy
provided by an mpgroup

2012 Oracle

39

Typical Configuration with 2 I/O Domains

physical vdisk backends
network (file, disk, volume)

PCI A

PCI B

primary domain
vsw0

physical vdisk backends

network (file, disk, volume)

alternate domain
vsw0

e1000g0

vsw

e1000g2

vsw

vds

vds

Hypervisor

vnet

vdc

guest domain 1

2012 Oracle

40

vnet

vdc

guest domain 2

vnet

vdc

guest domain 3

vnet

vdc

guest domain 4

VDisk Mirroring with 2 I/O Domains

vdisk backend
(file, disk, volume)

vdisk backend
(file, disk, volume)

alternate

primary
vds

vds

Hypervisor

guest
vdc

vd
c

c0d2

c0d3

Disk Mirror
(SVM / ZFS)

2012 Oracle

41

VNet Multipathing with 2 I/O Domains

physical network

alternate

primary
vsw

vsw

Hypervisor

guest
vnet

vnet

IPMP

2012 Oracle

42

Split-PCI Best Practices

Do it when:

You have availability requirements within a single platform, and

You want resiliency against a path or device failure
You want resiliency against a service domain failure
You want to be able to do a rolling upgrade of your service
domains

Consider it even when you have a clustered app, like

Oracle RAC or Weblogic
Don't do it when:
You are first starting out, as it's more complex than a simple
install with a single control domain == service == I/O domain
You have minimal availability requirements or no SLA at all
2012 Oracle

43

Secure Live Migration

Eliminates Application Downtime

Live migration now available on

SPARC T-Series systems
SPARC T3 and T4
UltraSPARC T2 Plus
UltraSPARC T2

On-chip crypto accelerators

deliver secure, wire speed
encryption for live migration
No additional hardware required
Eliminates need for dedicated network

More secure, more flexible

See http://www.oracle.com/technetwork/serverstorage/vm/ovm-sparc-livemigration-1522412.pdf

44

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

VM
VM

Secure Live Migration (SSL)

VM

Oracle VM Server Pool

SPARC T-Series servers

External Shared Storage

VM

Domain Mobility
AM/Day/Week

PM/Night/Weekend

Migrate workloads between logical domains without shutting down the

application
Keep services running in preparation for planned maintenance
Save and reduce energy costs doing mini-consolidations in quiet
periods
Be proactive and be able to react to any service degradation
45

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Cold and Warm Domain Mobility/Migration

# ldm migrate-domain ldg1 root@targethost

Cold migration planned migration while domain down

migrates a domain that is not running
can move between different T-series system and CPU types

Warm and Live Migration move a running domain

Warm (pre 2.1): suspend domain and resume on destination system
Live: transmit domain while it continues to run, then briefly pause
domain to transmit changes
RAM always compressed and encrypted before transmission
Pre-2.2: requires same T-series CPU chip, clock rate

Prerequisites:
shared storage (NFS is fine), network connectivity
Same-named virtual network and disk services
2012 Oracle

46

Live Migration Requirements

Source System
Guest with virtual I/O devices only, Solaris 10 9/10 (update 9) or later
LDoms Manager 2.1 or later, current firmware (i.e. 7.4 or 8.1)
Power Management in performance mode (the default)

Target System

Must have enough resources (CPU, memory)

Must have appropriate VIO services (vds, vsw, vcc)
Must be able to provide required VIO devices (vdisk, vnet)
Pre-2.2: requires same T-series CPU chip, clock rate)

# psrinfo -pv
The physical processor has 8 virtual processors (0-7)
SPARC-T3 (chipid 0, clock 1649 MHz)
# prtconf -pv | grep stick-frequency
stick-frequency: 05f4bc08

47

Cold and Warm Domain Mobility/Migration

Live or warm migration is not instantaneous
Warm (pre-2.1): guest is unresponsive during move, then resumes
Live: guest runs during state copy, then briefly paused at end
Total time depends on network speed and guest memory size

Best practices
Assign crypto accelerators to control domains (if pre-T4)
Assign sufficient CPUs (1 or 2 cores) to control domain (check CPU
load during migration with ldm list or even vmstat/mpstat)

Fewer use cases requiring migration than x86

Can upgrade, or tolerate loss of control domain without migration by
using redundant domains and I/O (unlike many hypervisors)
Can manage CPU and RAM resources without guest boot, so you can
make room to enlarge or start a domain by shrinking other domains,
rather than having to migrate them to make room
2012 Oracle

48

Cross-CPU live migration

Oracle VM Server for SPARC 2.2 new feature

Can live-migrate guests between different T-series types
Guest must be running Solaris 11
Domain must have cpu-arch=generic, not default native
# ldm set-domain cpu-arch=generic mydomain
Generic mode domains lose processor-specific enhancements
such as new instructions and 2GB pagesize available on T4
See
https://blogs.oracle.com/jsavit/entry/cross_cpu_live_migration_in

See video at http://www.youtube.com/watch?v=MuGGZ1uLeN8

2012 Oracle

49

Example of live migration

50

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Live Migration Best Practices

There is no specific requirement on the number of CPUs in the
control domains. However our experience shows that >8 vCPUs
is best. We recommend 16 or more vCPUs in order to minimize
suspend and overall migration time.
You can add or remove CPUs from control domain as needed, so
you don't have to dedicate extra CPUs when not doing migration.
The source control domain is the one using CPU resources. The
target control domain uses far fewer CPU cycles.
Workloads that heavily modify memory will have longer migration
times
Add crypto units to control domain for best migration
performance, except on T4, where crypto is built into core
Review the documentation, especially the Admin Guide for
planning live migration
51

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Dynamic Reconfiguration and Virtual Resources

AM/Day/Week

PM/Night/Weekend
Adjust your systems to your business requirements on demand
> Dynamically change : CPU, Virtual IO, Memory without interruption
> Automated CPU resource management
Improve utilization by balancing resources between domains

52

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Dynamic Resource Management

Policy-driven automatic allocation and deallocation of
resources based upon thresholds of utilization
Prioritization and time-based enable/disable of RM policies
Can take CPUs from a low-priority domain if a high priority domain
needs them. Only applies to domains under DRM control, and only if
there are no unallocated CPUs. Note lower numeric priority values are
higher (better)

Resource min, max, and overall system share limits

Ramp-up (attack) and ramp-down (decay) controls to adjust
number of CPUs are shifted when workloads change
Proactive (continuous) and reactive (on-demand) resources
Example:
# ldm add-policy tod-begin=09:00 tod-end=18:00 util-lower=25 \
util-upper=75 vcpu-min=2 vcpu-max=16 attack=1 decay=1 priority=1 \
name=high-usage ldom1
2012 Oracle

53

CPU Power Management

Save power by automatically powering off unused
cores. Two modes:
Performance: All cores turned on at all time
Elastic: Cores are dynamically powered on/off as required

Behavior is set via the ILOM but can be monitored

from control domain
T3, T4 enhancements
CPU speed adjustment: Increase, decrease clock speed
based on CPU load

Memory power management: put unused RAM in deep idle

Can set power limit for entire server: drop power state
of managed items when reach power budget

54

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Best Practices

Keep firmware up to date contains the hypervisor

Allocate on core boundaries except for tiny domains
Allocate 1 or 2 cores for control domain
For high speed inter-domain communications use in-memory
vsw LDC channels - no physical device needed
For best disk performance, allocate a whole real device via a
dedicated, properly sized service domain
Exploit ZFS and other Solaris features
ZFS clones permit fast provisioning and save disk space, but
precludes live migration, since zpool on only one host at a time

Look at the server physical device architecture to ensure you

get the bandwidth you expect
Secure and minimize control and service domains
55

Best Practices
Plan for availability to meet application needs:
Redundancy for I/O path by multiple paths
Redundancy for service domain failure (paths served by different
domains)
Redundancy at physical server level (clustering)

Use multipath disk, Oracle Sun Cluster for resiliency

Use IPMP for resilient network connections
For critical applications consider hot/warm standby domains
across multiple physical servers
Don't over-engineer provide redundancy consistent with the
business need.
May not need as much if application provides redundancy (Oracle
RAC, Weblogic)
56

Management with Ops Center

Oracle VM Server
for SPARC

Lifecycle Mgmt
Create, Delete,
Configure, Boot,
Shutdown, etc.
57

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Monitoring

Resource Pools

CPU, Memory, File Policies, Auto

Systems, Aggregate
Restart
Utilization

Migration
Domain
Mobility

Accelerate Mission Critical Cloud Deployment

Faster Deployment. Higher Scalability

Coverage of both OVM x86 and SPARC

Self service portal

Automation via dynamic policies

Built for mission critical cloud

Automated fault management

Oracle Storage and networking

intelligence

Application-to-Disk performance
analytics

58

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Oracle Enterprise Manager Cloud Control12c

Self Service

Charge Back
Resource Scheduling

Capacity Planning

Oracle Enterprise Manager Ops Center12c

Dynamic
Domains

Oracle VM Containers
For SPARC
HYPERVISOR

M-Series

T-Series

Oracle VM
For x86
HYPERVISOR

All SPARC & x86

Virtualized Storage and Networking

All x86

Policy Based Automation

Ops Center Server Pools Offer Full V12N Coverage
Solaris Zones, OVM x86, and OVM SPARC

Autobalance workloads at scheduled points in time

Automatically move workloads at specific utilization level violations
Automatic Failover (HA)
Placement Policy
Based on Load, # of VMs, and Energy Usage

Live Migration of Virtual Machines **

** Solaris Zones (Containers) are cold migration
Oracle Database Cloud Zone

59

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Unleash the Power of Solaris 11

Faster System Updates
Built for the first cloud OS
IPS, Dtrace, ZFS, IB, FMA, Zones
Discovery, template driven provisioning
Intelligent patching
Advanced OS snapshotting
Network virtualization & flow control

60

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

SPARCI IaaS
For Datacenter Operations
create, provision, manage
and monitor tenants within their
virtual data centers

For Tenants
provision Solaris Zones, OVM
SPARC, OVM x86, virtual
networks and storage to
applications

61

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Self-Service Operations for SPARC

Focused User Interface Experience
Console for Self Service
Restricted privileges to perform limited
operations
Create/Destroy VMs
Startup and shutdown of VMs
Basic resource monitoring (CPU,
memory, Storage, Top Sessions)
Review Quota information
EC2 Modeled API experience

62

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Summary
Oracle is Investing More in Solaris, SPARC, and
Virtualization than Ever Before!
Oracle is the Only Vendor with a Unified Strategy for
Virtualization and Cloud Computing
Increasing business agility with Oracle VM Server for
SPARC, enabling you to Consolidate and Build Private
Clouds

63

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

For More Information

https://wikis.oracle.com/display/oraclevm
http://oracle.com/virtualization
http://blogs.oracle.com/virtualization

64

Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Solaris 10 System Virtualization

Covers all types of

Solaris virtualization:
Zones
Oracle VM for SPARC
Oracle VM for x86
Oracle VM Virtualbox
Dynamic System Domains
Jeff Victor, Jeff Savit,
Gary Combs, Simon
Hayler, Bob Netherton,
plus a host of helpers

65

Copyright 2012, Oracle and/or its affiliates. All rights reserved.