Professional Documents
Culture Documents
I. I NTRODUCTION
Numerous applications and services are transitioning from
running on a single PC into running on a cloud platform.
Cloud platforms basically consist of a virtual machine (VM)
cluster and APIs to manage the cluster. VM clusters are used
to consolidate various servers into fewer real servers to benefit
from a statistical multiplexing effect of computing resources.
Because each VM tends to be supplied with enough memory
to process a peak workload, the guest operating system (OS)
always has abundant memory left unused. Unused in this
paper refers to both situations where the memory is assigned
to no VM, and where memory assigned to a certain VM is
not used by the guest OS.
In dynamic memory allocation used in a modern OS, an
internal fragmentation occurs when some part of the allocated
memory is not used. An external fragmentation occurs when
there is insufficient memory left to fulfill the allocation request.
Similarly, from the viewpoint of VM cluster management, an
internal fragmentation occurs when the memory assigned to
a VM is not fully used by the guest OS, and an external
fragmentation occurs when there is insufficient memory to
accommodate more VMs, which are therefore left unassigned.
To further complicate the matter, these fragmentations are
scattered within a VM cluster.
Even if one guest OS needs more memory than what is
assigned to it, it cannot use the memory unused by another
guest OS. In this situation, the guest OS is going short of
physical memory and makes use of a swap device. This results
978-0-7695-4302-4/10 $26.00 2010 IEEE
DOI 10.1109/CloudCom.2010.56
76
1. I need more
memory
4.
Virtual Swap
Manager
2.
Memory
Watchdog
dom-0
0. I have unused
memory
Logical
Volume
Memory
Watchdog
Requests
Extra
Memory
MemoryWatchdog
dom-U
Memory
Importer
5. Replace swap
from local to remote
Virtual
Swap
Manager
Exchanging memory
Memory
Exporter
3. Ready to export
Monitoring
Memory
Usage
Physical
Extent
on HDD
Direction to
Export
Memory
dom-0
MemoryWatchdog
ramfs
Physical
Volume
on ramfs
Physical
Extent
on iSCSI
VMM
Fig. 1.
VMM
SATA or SAS
Fig. 2.
Ethernet
Switching Hub
A. Implementation Detail
Memory Watchdogs reside in dom-0 to escape the thrashing
effect of dom-U and a guest OS. A Memory Watchdog
monitors the memory usage of each dom-U through XenStore
and calculates the amount of exportable memory by summing
up the free memory of each dom-U and dom-0, and deducting
the margin. The Memory Watchdog then reports the amount
of exportable memory to the Virtual Swap Manager at the
specified intervals. The Virtual Swap Manager keeps track of
the exportable memory reported by Memory Watchdogs under
the control of the each dom-0. When a Memory Watchdog
detects that the free memory of a dom-U is lower than m th,
and it lasts n high times, it requests extra memory from the
Virtual Swap Manager. If there are Memory Watchdogs with
larger exportable memory than the requested amount, the Virtual Swap Manager selects the one with the smallest exportable
memory. It then requests the selected Memory Watchdog to
export the specified memory amount. Upon receiving the
77
TABLE I
E XPERIMENT S ETUP
Check memory usage of each domain-U
VMM
dom-0
dom-U
iSCSI target
iSCSI initiator
LVM
Make ramfs
importing = true
Fig. 3.
Xen-3.4.1
openSUSE-11.2 64bit
openSUSE-11.2 64bit (PVM)
iSCSI Enterprise Target
Open-iSCSI
LVM2
78
50 100
200
300
50
100
150
600
800
0
50
time [sec]
(a)
400
throughput [Mbps]
0
0
time [sec]
200
12000
8000
4000
10000
0
5000
15000
10000
6000
2000
0
0
100
150
50
time [sec]
(b)
100
150
time [sec]
(c)
(d)
50
100
150
200
250
time [sec]
(a)
20
40
60
80
throughput [Mbps]
0
0
100
20
40
60
80 100
time [sec]
time [sec]
(c)
(b)
15000
0
0
0
5000
15000
5000
15000
5000
25000
Fig. 4. Time series of operations per second of sequential access workload: (a) local HDD only, (b) switch to local iSCSI, (c) switch to remote iSCSI, (d)
network throughput in the case of (c)
140
20
40
60
80 100
140
time [sec]
(d)
Fig. 5. Time series of operations per second of random access workload: (a) local HDD only, (b) switch to local iSCSI, (c) switch to remote iSCSI, (d)
network throughput in the case of (c)
79
R EFERENCES
TABLE II
BASIC P ERFORMANCE OF VSMM
device
local HDD
local SSD
GbE mtu 1,500
GbE mtu 9,000
10GbE mtu 1,500
10GbE mtu 9,000
local iSCSI
write [MB/s]
seq.
random
21.58
15.24
25.67
40.10
19.48
25.34
18.91
23.43
27.41
38.17
26.43
35.97
33.96
42.49
read [MB/s]
seq. random
13.11
4.71
29.94
12.75
18.69
14.49
25.73
20.63
29.72
30.34
35.74
32.10
41.81
34.26
VI. C ONCLUSION
We proposed a flexible memory exchange mechanism called
a VSMM, which can be used in cloud computing environments. Through the experiment, we conclude that the VSMM
can dynamically switch a virtual swap to/from remote memory
that achieves improvement in both performance and memory
usage. Moreover, this is performed transparently to a running
VM. Though current VSMM implementation is quite simple
in order to prove the feasibility and efficiency of the proposal,
a guest OS can use remote memory as a part of a virtual swap
to improve performance with no modification to the running
guest OS. It takes about 20 seconds to switch a virtual swap
from a local HDD to a remote iSCSI target. We are wondering
if this can be shortened by using off-load technology such
as the TCP and iSCSI off-load provided by certain network
interface cards.
We are currently conducting more realistic experiments
with real applications. Parts of the results show that the
configurable parameters mth and nhigh affect the applications
performance. Setting smaller nhigh can sensitively follow the
change of memory usage and use remote at the soonest
possible opportunity, but it may cause fluctuation of memory
exchange; importing and releasing repeatedly according to the
minor change of memory usage. The function for preventing
fluctuation phenomena is one planned enhancement for VSSM.
To optimize the performance and memory utilization of a
cloud environment, we will consider other decision criteria for
switching a virtual swap, such as network latency, available
bandwidth, and duration of potential usage of the remote
memory. CPU cycles of the exporting real machine involved
in exporting memory should also be considered. Our future
work includes development of a more sophisticated decision
algorithm for virtual swap switching, and an assignment
algorithm for remote memory, along with implementation of
the whole system.
80