Virtual Machines:

Supporting Changing Technology

and New Applications

1
Introduction
Why virtual machines?

They allow transcending of standardized interfaces

(which sometimes are an obstacle to innovation)

They enable innovation in flexible, adaptive software & hardware,

security, network computing (and others)

They involve computer architecture in a pure sense

Virtualization technologies will be a key part of most future computer

systems

2
Outline
 Virtualization
 Virtual Machine Architecture
 Virtual Machine Implementation
 Computer Architecture Applications
• Co-Designed VMs
• Private Virtual Machines

3
Abstraction
Software
 Computer systems are built Application
Programs
on levels of abstraction fileLibraries
file

 Higher level of abstraction abstraction

Operating System

hide details at lower levels Drivers

Memory
Manager
Scheduler

 Example: files are an Execution Hardware

abstraction of a disk Memory

Translation
System Interconnect
(bus)

Controllers Controllers

I/O devices
Main
and
Memory
Networking

Hardware

4
Virtualization
 Similar to abstraction
Except
virtualization
• Same level of detail
file
file
 Construct Virtual Disks
• As files on a larger disk
• Map state
• Map operations
 VMs: do the same thing with the whole
“machine”
 Key concepts: Map state; Map Operations

5
The Family of Virtual Machines
There are things
Including lots of not
“virtual
called
machines”
“virtual machines”
IBM VM/370
IA-32 EL
Java
HP Dynamo
VMware products
Transmeta Crusoe

“The subjects of virtual machines and emulators

have been treated as entirely separate. … they
have much in common. Not only do the usual
implementations have many shared
characteristics, but this commonality extends
to the theoretical concepts on which they are
based”
-- Efrem G. Wallach, 1973

6
“Machines”
 Different perspectives on Application
what the Machine is: Programs

 OS developer Libraries

 Compiler developer Operating System

 Application programmer
Execution Hardware

Instruction Set Architecture Memory

Application Binary Interface
Program Interface Translation
• ISA System Interconnect
(bus)
• API
ABI
• Major division between hardware
User
User
• and ISA
ISA ++ library
software OS calls
calls I/O devices
Main
and
Memory
Networking

7
System Virtual Machines

 ISA level
guest guest guest guest guest guest
 Provide a system process process process process process process
environment Guest OS2
Guest OS
 VMM manages
guest OS + apps VMM VMM

 Persistent
 Examples: IBM HOST PLATFORM
VM/360, VMware,
Transmeta Crusoe
virtual
network communication

8
Process Virtual Machines
 ABI level
 Runtime manages guest
process
host guest
 Guest processes may process process
intermingle with host
processes runtime

 Not persistent guest guest

host
process process
 Guest and host OSes are process

often the same runtime

runtime
create
 Dynamic optimizers are
a special case HOST OS
 Examples: IA-32 EL, FX!
32, Dynamo file sharing
Disk

network communication

9
High Level Language Virtual Machines
 Constructed at API level
• User higher level virtual ISA
• OS abstracted as standard libraries
 A form of process VM

HLL Program HLL

Program
Compiler front-end Compiler

Intermediate Code Portable Code

(Virtual ISA )
Compiler back-end

Object Code VM loader

(ISA) Virt. Mem. Image

Loader VM Interpreter/Translator
Memory Image Host Instructions

Traditional HLL VM

10
Virtual Machine Architectures

Process VMs System VMs

different different
same ISA same ISA
ISA ISA

Multi Dynamic Classic Whole

programmed Translators OS VMs System VMs
Systems

Dynamic Hosted Co-Designed

HLL VMs VMs
Binary VMs
Optimizers
11
VM Technology – State Mapping
Host
 VM SW re-maps virtual Host Registers Register
state to real state Space
Guest Registers
• Recall virtual disk
• Registers to registers Runtime
Data
• Registers to memory Runtime
• Memory to memory Code

• Memory to disk
Host ABI
Guest Data Address
Space

Guest Code

12
VM Technology – Operation Mapping

 VM SW re-maps operations on state

 Instruction-level state changes
• Emulation
 Protected state changes
• OS operations
• Done under VMM control
 Key concepts: Emulation and Control

13
VM Technology – Emulation

 Interpretation interpreter
source code routines
• Software loop
decodes and "data"
accesses
dispatches each
instruction

dispatch
loop

14
VM Technology – Emulation
binary translated
 Binary translation and target code

code caching source code

• Translate blocks of
instructions at a time
• Hold translated blocks in
code cache binary
translator
• This was a key enabler
for VMware success

15
VM Technology – Emulation
 Staged Emulation
• Emulation techniques invoked in staged manner
• Based on performance tradeoffs

Interpreter Emulation
manager

Binary Memory
Code Cache Profile Data
Image

Translator/
Optimizer

16
Code Caches
 Contain
• Basic blocks
• Superblocks (one entrance, multiple exits)
• Optimized Superblocks
 Used in many VMs
• Dynamic binary translators: Intel IA-32 EL, Compaq FX!32
• Dynamic binary optimizers: Dynamo family
• Co-designed virtual machines: Transmeta, IBM DAISY
• High performance Java virtual machines
• System VMs with “inefficiently virtualizable” ISAs
• “Sandboxing” secure VMs (x86 DynamoRIO)

17
Code Caching with Chaining
 Chaining of blocks in code cache minimizes VM
overhead

Code Cache
Super
block

Dispatch Super
table block
lookup
code
Super Super
block block

18
VM Technology – Control
 Interpretation
• Fine grain control
• Every dynamic instruction “inspected” before execution
 Binary translation and code caching
• Coarser grain control
• Every static instruction inspected before execution
• Jumps to VM SW can be inserted anywhere
 Protection levels
• Very coarse grain control
• Every resource-related instruction trapped by protection system
Otherwise, use interpretation/translation techniques
• Used in system VMs

19
Resource Control in System VMs
Application

system call/trap
 Traps and interrupts (& sys calls)
• Transfer to VMM
• VMM determines appropriate Guest OS Guest OS
• VMM transfers to Guest OS privileged operation
next instruction
 Guest OS “return” to user app.
• Transfer to VMM virtual vector location:
• VMM bounces return back to Guest app. system return
 Resource sensitive instructions
VMM
• Trap to VMM
• VMM checks correctness check privileges
perform operation
• VMM reads/modifies guest resource return

• Returns to Guest
vector location:

20
VMs and Computer Architecture
 Use virtualization to give computer architects a layer
of software
• Beneath all conventional software
• Maintains vision of hardware as seen by conventional software
 Performance optimizations via Co-designed VMs
• VM SW can alter/enhance architecture via emulation
 Resource management – Private Virtual Machines
• VM SW can manage microarchitecture resources

21
Co-Designed Virtual Machines
 Separate the hardware/software interface from the ISA level of abstraction
 Restore the ISA to its “natural” place
⇒ as an Implementation ISA that reflects actual hardware
 Support existing ISAs
⇒ as a Virtual ISA
 Let processor designers use both
hardware and software
 A form of system VM

User Applications
User Applications

libs.
libs.
OS

V-ISA
ISA

Software
I-ISA
Hardware
Hardware

22
 VM Technology -- Concealed Memory
 VM software resides in memory concealed from
all conventional software
 This software is available to hardware designer

Code
Cache ICache
concealed memory Hierarchy
VM Code
VM Data Processor
Core
Source ISA Code
DCache
conventional Hierarchy
Source ISA Data
memory

23
Co-Designed VMs

 Of interest to both architects and micro-

architects
• Offers opportunities for performance, power saving,
fault tolerance and other implementation-dependent
features
• Allows transcending conventional ISAs
• Don’t confuse them with VLIW!
Early examples: IBM Daisy and Transmeta
Crusoe
“pioneers are the ones with arrows in their
backs”
24
Another Way of Doing Things

conventional Func.
Unit

Translation Func.
Cache Processor
Main Memory Unit Unit
Hierarchy (form uops) Pipeline
. ..
Func.
Unit

Main Memory

Software
dynamic translation
Translator Func.
Unit

Cache Processor . ..
Code Cache Translation
Hierarchy Pipeline
Unit
(form uops)

Func.
Unit

25
Fused Microarchitecture
 Fuse dependent pairs of micro-ops to macro-ops
• Current Intel approach
 Use co-designed SW to achieve wider-scale fusing
 Process & execute fused macro-ops as single
Instructions throughout the entire pipeline
 Allows pipelined wake-up/select issue logic
3-1 ALUs

cache
ports

Fuse
bit Payload
Decode
Align Wake- RAM
WB
Fetch Fuse
Rename Select RF EXE MEM
Retire
Dispatch up

26
Fusible Instruction Set
Core 32 - bit instruction formats
 RISC-ops with unique
features:
F 10 b opcode 21 bit Immediate / Displacement
• Fuse bit per instruction fuses
F 10 b opcode 16 bit immediate / Displacement 5b Rds two dependent instructions
• Dense instruction encoding,
F 10 b opcode 11b Immediate/ Disp 5 b Rsrc 5b Rds
16/32-bit ISA design
F 16 bit opcode 5b Rsrc 5b Rsrc 5b Rds  Special Features to Support
the x86 ISA
Add-on16-bit instruction formats for code density • Condition codes
F 5 b op 10b Immd/Disp • Addressing modes
F 5 b op 5b Rsrc 5 b Rds
• Aware of long immediate &
displacement values
F 5b op 5b Rsrc 5 b Rds

Fusible ISA Instruction Formats

27
Fusing Profile
 About 50% of operations are fused
 Only 5-10% of non-fused are single-cycle ALU ops
100%

90%
Percentage of Dynamic Instructions

80%

70%

60%

50%
ALU
40%
FP or NOPs
30% BR

ST
20%
LD
10% Fused

0%
ty

f
r

er

ex

ge
18 c

cf

k
pe n

2
ip

ol
vp

gc

ip
o

ga
af

rs
m
gz

rt

ra
tw
25 2.e

rlb
5.

bz
cr
6.

vo
1.

4.
pa

ve
4.

0.
17

17

6.
6.

25

25
16

5.
7.

30

A
25
18

25
19

3.

28
 Nomarlized IPC speedup (%)

0
10
20
30
40
50
60
70

-10
16
4.
gz
ip
17
5 .v
p r
17
6.
gc
Base + Code Cache

18 c
1.
m
Performance

18 cf
6.
cr
af
19 ty
+ fusing

7.
pa
rs
25 er
2.
25 eo
3. n
pe
r lb
m
25 k
+ shorter pipe

4.
ga
25 p
5.
vo
r te
25 x
6.
bz
ip
2
+ 3-1 ALU

30
0.
tw
ol
Ha f
rm
on
ic
29
Virtual Private Machines
 Multi-core systems will have many hardware-
level shared resources
• Multi-threaded processors
• Multi-level shared caches
• Shared memory ports
• Spares for fault tolerance
 And a number of important implementation
dependences
• Non-uniform memory delays
• Power optimization features
• Fault tolerance features

30
Virtual Private Machines
 Co-design a MicroVisor to provide software with
Virtual Private Machines
• Insulates conventional software from complicated
implementation-dependent features
 Provides performance virtualization
• Unlike classic VMs System System
• Quality of Service (QoS) VM VM
Conventional
• Performance Isolation Functional OS
Hypervisor (VMM)
Virtualization

VPM VPM

Performance
MicroVisor
Virtualization

Multi-Core Hardware

31
MicroVisor
 Virtualizes performance, not functionality
 Co-Designed software to support hardware resource
management
• Concealed/isolated from all conventional software
• Much larger than microcode, does not consume processor chip
real estate
• Uses conventional instructions (extended) so there is more
likelihood of some cross-system portability

μV Code ICache
concealed memory Hierarchy
μV Data Processor
Core
Code
DCache
conventional Data
Hierarchy
memory

32
“Real-izing” Processors/Memory
 Separate Real Processors
from Physical Processors Processes
• OS assigns processes to
Real Processors
OS Maps
• MicroVisor maps real
processors to physical
processors Real
Processors
 MicroVisor also maps
real memory to physical MicroVisor Maps

memory
• Extend to cache memories Physical
Processors

33
Virtual Private Machines
 MicroVisor maps high level requirements to hardware
configuration
 Requires mechanisms to provide microarchitecture level
QoS Main Memory
Main Memory

Memory Controller Mem Mem Mem Mem

Controller Controller Controller Controller

L2 Cache
L2 Cache L2 Cache L2 Cache L2 Cache

Interconnection Net
L1 Cache L1 Cache L1 Cache L1 Cache
L1 Cache L1 Cache

Proc. 0 Proc. 1 Proc. 0 Proc. 0 Proc. 0 Proc. 0

Threads Threads Thread 0 Thread 0 Thread 0 Thread 0
0&1 2&3
VPM 0 VPM 1 VPM 2 VPM 3
34
Applications
 Performance optimization
• Deal with NUMA
• Provide QoS and performance isolation in multi-threaded systems
 Power management
• Adjust resources to match power constraints
• Requires inferring demand for resources
in contrast to conventional OS
 Fault tolerance
• Detected fault triggers MicroVisor
• Diagnose, reconfigure, re-map memory/processors

35
Summary
 Many types of VMs
• But common virtualization technologies
 An important system component
• Should be studied/taught as a discipline on its own
Alongside OS, Application SW, HW
 Many avenues for computer architecture research
• Co-designed VMs
• Virtual Private Machines
• Adaptive microarchitecture
• Fault-tolerant implementations
• Primitives for supporting efficient VMs

36