Professional Documents
Culture Documents
OpenStack
(2015-03-13)
2014, 2015 OpenStack Some rights reserved.
OpenStack
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed
on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for
the specific language governing permissions and limitations under the License.
i
March 13, 2015
....................................................................................................... v
............................................................................................... v
Document change history ................................................................ v
1. ................................................................................................... 1
........................................................................................ 1
.......................................................................... 1
............................................................ 2
........................................................................................... 4
2. ............................................................................................... 9
...................................................................................... 10
...................................................................................... 12
...................................................................................... 23
............................................................................................. 26
............................................................................................. 34
3. .............................................................................................. 37
...................................................................................... 37
...................................................................................... 39
...................................................................................... 47
............................................................................................. 49
............................................................................................. 56
4. .............................................................................................. 61
...................................................................................... 62
...................................................................................... 63
...................................................................................... 65
............................................................................................. 70
............................................................................................. 79
5. .............................................................................................. 85
...................................................................................... 87
...................................................................................... 89
...................................................................................... 95
............................................................................................. 96
............................................................................................. 99
6. ............................................................................................ 105
.................................................................................... 105
.................................................................................... 108
.................................................................................... 112
........................................................................................... 114
........................................................................................... 117
7. ............................................................................................ 123
.................................................................................... 124
.................................................................................... 127
iii
March 13, 2015
.................................................................................... 132
........................................................................................... 134
........................................................................................... 137
8. ....................................................................... 141
.................................................................................... 142
.................................................................................... 144
.................................................................................... 146
9. ........................................................................................ 149
................................................................ 149
................................................................... 151
............................................................................. 152
................................................................................. 154
OpenStack OpenStack .......................................................... 156
................................................................................. 158
10. .............................................................................................. 161
A. ........................................................................................ 163
........................................................................................... 163
.................................................................................... 164
OpenStack .................................................................... 165
OpenStack .................................................................... 165
LaunchpadBug ...................................................................... 165
The OpenStack ................................................... 166
.................................................................................... 167
OpenStack ........................................................................ 167
................................................................................................ 169
iv
March 13, 2015
OpenStack
$ $ root
# # root
sudo
v
March 13, 2015
1
................................................................................................ 1
.................................................................................. 1
.................................................................... 2
................................................................................................... 4
OpenStack
(IaaS)>
IT
OpenStack
OpenStack
OpenStack(
)
OpenStack
This book has been written for architects and designers of OpenStack
clouds. This book is not intended for people who are deploying OpenStack.
For a guide on deploying and operating OpenStack, please refer to the
OpenStack Operations Guide (http://docs.openstack.org/openstack-ops).
linux
OpenStack
1
March 13, 2015
: 80%
: (HPC)
: (CDN)
OpenStack
OpenStack
2
March 13, 2015
OpenStack/
sprintsprint
Book Sprints
(www.booksprints.net)
520147
VMware
Twitter#OpenStackDesign
Faith Bosworth Adam HydeBook Sprint
VMwareCisco,
Cloudscaling,Comcast,EMC,Mirantis,Rackspace,Red Hat, VerizonVMware,
Anne Gentle Kenneth Hui
3
March 13, 2015
Amazon
5Amazon
Amazon
web
4
March 13, 2015
OpenStack
/
/
.OpenStack
OpenStack
5
March 13, 2015
NY-1138-Q
(SLA),
zones
6
March 13, 2015
7
March 13, 2015
.............................................................................................. 10
.............................................................................................. 12
.............................................................................................. 23
..................................................................................................... 26
..................................................................................................... 34
OpenStack
web
80%
9
March 13, 2015
Raw
IP
(VLANs)
Infrastructure-as-a-Service (IaaS)
6
2
(CSP)
10
March 13, 2015
OpenStack
OpenStack
(
)
11
March 13, 2015
6 [105]
OpenStack
hypervisor
12
March 13, 2015
hypervisor
OpenStack
CPUCPU161
1.51
m1.small1
vCPU,20GB2048MB
210CPU
CPU161
640(2x10x2x16)m1.small
1.51853GB(640x2048/1.5x1048)
(EPT)
hypervisor
13
March 13, 2015
OpenStack
OpenStack(nova-network)OpenStack
(neutron)
(nova- (nova-network)
network)
VLAN
VLANs
VLAN
VLANs
4096.
VLANs
14
March 13, 2015
OpenStack
REST
swift
API
OpenStack
/
OpenStack
OpenStack
TB
2U
15
March 13, 2015
OpenStackOpenStack
OpenStackswift
swift>
4U60
1UTB
RAID
(3)
IO
OpenStack
OpenStack
>
OpenStack
()
RAID
(IOPS)RAID
RAID
RAID
RAIDRAID5RAID6
SSD
flash
16
March 13, 2015
OpenStack
Hypervisor
(OS)
OpenStack
Ubuntu
Linux(RHEL)
CentOS
Linux(Windows),
OpenStack()
OpenStack(
OpenStack)
hypervisor
Ubuntu,RHEL,SLEShypervisor
OpenStack (nova)KVM,Xen,LXC
Hyper-V,Windows
XenServer,CenOS dom0
-hypervisor
-OpenStack
OpenStack
hypervisor
17
March 13, 2015
OpenStackHypervisor,
Hypervisor
XCP/XenServer
Hyper-V
LXC
Docker
hypervisorKVM
Xenhypervisor
hpervisorVMware
OpenStackhyperviosr
>
hypervisorHyper-V
Hyper-V
OpenStackhypervisor
hypervisor>VMware
vMotion,ESXi hypervisorVCenter/
vSphere
hypervisorhypervisor
OpenStack
OpenStackOpenStack
OpenStack
18
March 13, 2015
OpenStack (nova)
OpenStack (neutron)
OpenStack (glance)
OpenStack (keystone)
OpenStack (horizon)
OpenStack (swift).OpenStack
(cinder)
OpenStackOpenStack
OpenStack
MariaDBGalera
3Galera
APISSL
HAProxy
KeepalivedCorosync
PacemakerPacemakerCorosync
OpenStack
Pacemaker2
MemcachedRediakey-value
memcached
memcachedRedis
OpenStack
19
March 13, 2015
OpenStack
Horizon>
OpenStack(Keystone)Keystone
OpenStack
OpenStack
(GbE)
10 GbE
OpenStack
OpenStack
webOpenStack1 GbE
OpenStack10 GbE
API
SSL
SSLSSL
CPU,
OpenStack
20
March 13, 2015
OpenStack16:1CPU
1.5:1
OpenStack
OpenStack
NetAppEMC
GlusterFSCeph
OpenStack
10 GbE()
OpenStack
bonding
OpenStack
OpenStack
OpenStack
RabbitMQQPIDMySQLMariaDB
OpenStack
(nova-network)OpenStack(neutron)
OpenStack
Openstack
21
March 13, 2015
OpenStackOpenStack
3
web
OpenStack
3
OpenStack
OpenStack
22
March 13, 2015
OpenStack
OpenStack
OpenStack
OpenStack
LDAPIncluding such systems in an
OpenStack deployment may ease user management if integrating into
existing systems.
APISSL
23
March 13, 2015
(SLA)
SLA
SLA
API
OpenStack
OpenStack
OpenStack
OpenStack
API
OpenStack
()
()
24
March 13, 2015
SLA
(HA)
OpenStack
OpenStackOpenStack
CPU
CPU
CPU,CPU
CPU
OpenStack
25
March 13, 2015
OpenStack
CPU
OpenStack
OpenStack
()
(CPU)()
()
(/(IOPS)GBTB
[U]
CPU
OpenStack
26
March 13, 2015
CPU
HP BladeSystem
Dell PowerEdge M1000e10U16
1U1U
CPU
2U1U
(1U)
U4UCPU
8CPU
2U3U
1U-2U
OpenStack
OpenStack
hypervisor
(hypervisor)
()
20AMP
120 AMP
27
March 13, 2015
()
InfiniBandFDDI
OpenStack
OpenStack
50PB,
10PB,scalability
scalability
expandabilityscalability
(DAS)
OpenStack
28
March 13, 2015
OpenStack
()
hypervisor
hypervisor
OpenStack
DAS
-Hypervisor
OpenStack
RAID
RAID
RAID
TB
SSDflash
RAID
zone
29
March 13, 2015
RAIDRAIDRAID
(GbE)
(10GbE)
()
"
Cumulus Linux
Cumulus Linux
1GbE10GbE
40GbE( 100GbE)
(EoR)
OpenStack
30
March 13, 2015
OpenStack
LACP,VRRP
API
API
OpenStack
(OS)
OpenStack
(OS)hypervisor
hypervisor
Hypervisor
Hypervisor
(LACP)hypervisor
OS
hypervisor Hyper-V
hypervisor
hypervisorKVM, Kinstance or Xen
Ubuntu()
hypervisor
hpervisor
31
March 13, 2015
UbuntuKinstanceVMware vSphere
OpenStackhypervisor
Hypervisor
-
hypervisor-
-hypervisor
OpenStack
-hypervisor
hypervisor
Hypervisor
hypervisor
-hypervisor-
hypervisor
OpenStack
OpenStackOpenStack
OpenStack
Telmetry
OpenStack
OpenStack
OpenStack
OpenStack
32
March 13, 2015
OpenStack
HAProxy
OpenStack
Logstash,Splunkinstanceware Log Insight-
(CPU
)
-hypervisor
-Hypervisor
OpenStack
OpenStack
33
March 13, 2015
OpenStackSQLAlchemy python
MySQL
OpenStack
OpenStack
CPU
3 [37]
OpenStack
OpenStack
5 [85]
OpenStack
4 [61]
web
Tomcat + Nginx + MariaDB
NginxTomcat1201402 CPU
4 GB
MariaDB3Galera4 vCPU8GB
34
March 13, 2015
web
Puppet
OpenStack
OpenStack
MariaDBRabbitMQ
OpenStackKVMhypervisor
OpenStack(
)
OpenStack()
35
March 13, 2015
140webMariaDB292vCPU,584GB
1Usocket16IntelCPU
2:1CPU8OpenStack
webOpenStackweb
MariaDBNetApp
SolidfireMariaDB
Galera
webOpenStack
webOpenStack
OpenStackOpenStack
OpenStack
OpenStack10 GbE
OrchestrationTelemetryweb
Puppetweb Heat Orchestration Templates
(HOT)
OpenStack
36
March 13, 2015
.............................................................................................. 37
.............................................................................................. 39
.............................................................................................. 47
..................................................................................................... 49
..................................................................................................... 56
openstack
(HPC)
Hadoop
/(CI/CD)
(PaaS)
(NFV)
IP(VLAN)
OpenStackraw
OpenStack
CPU
37
March 13, 2015
(SDLC)
(HadoopCassandra)
(CI-CD)
Hadoop
(HPC)
38
March 13, 2015
OpenStack
(HadoopHDFS)
(s)
- (IO)
()
OpenStack
OpenStack
OpenStack
OpenStack
( x )/
39
March 13, 2015
flavor x
16002 vCPU50GB
16:1
= 50GB x 1600
20080TB
/var/lib/nova/instances/
API
web
nova-api
OpenStaclk GUI
30
8CPU8GB
(spindles/core), (RAM/core),
(Gbps/core), CPU (CPU/core).
40
March 13, 2015
CPU
(Adapted from: http://docs.openstack.org/openstack-ops/content/
compute_nodes.html#cpu_choice)
CPU,12CPU
24CPU
CPUCPU
CPU
Libvirt/KVM hypervisorCPU
CPU VT-x
AMDCPUAMD-v
OpenStackCPU
OpenStack
CPU : 16:1
RAM : 1.5:1
CPU16:1,16
12192
flavor4
48
1.5:1
1.5
41
March 13, 2015
48GB
72GB(8GB9
)
CPU
(GPU)
SSD
flavorflavor
OpenStack
vCPU,
42
March 13, 2015
CPU
43
March 13, 2015
CPU
44
March 13, 2015
1.
2.
3.
4.
OpenStack
OpenStack
OpenStack
>
OpenStack
45
March 13, 2015
OpenStack
LDAPLDAP
API>SSL
OpenStack
OpenStack
OpenStack (nova)
OpenStack (glance)
OpenStack (keystone)
Orchestration (heat)
Orchestration
Orchestration
Telemetry (ceilometer)
OrchestrationTelemetry
TelemetryOrchestration
46
March 13, 2015
OpenStack (cinder)
CPU
OpenStack (cinder)
hypervisor
OpenStackdriver,ML2driver
ML2driver
OpenStack
API
(SLA)
OpenStack
API
SLA
(HA)
OpenStack
OpenStack
Python
47
March 13, 2015
OpenStack
OpenStack
OpenStackOpenStack
API
SLA
OpenStack
OpenStack
OpenStack
()CPU
hypervisor
CPU
CPU(
)
CPU
48
March 13, 2015
OpenStackCPU
OpenStack
OpenStack
()
(U)
CPU
,CPU
CPU
CPU
49
March 13, 2015
CPU
(HP BladeSystemDell PowerEgde M1000e),
1610
50%10
8
1U()
40
32
(ToR)CPUCPU
IceHouseHPIBMDell2
CPU1U1U2
(ODM)
2UCPU
(1U)
4UCPU
48CPU
(2U3U)
1U-2U
2U4CPU,2U8CPU
OpenStack
CPU
>
1U(
2U,3U4U)
OpenStack
50
March 13, 2015
OpenStack
()
I-O
OpenStack
CPU
"
50PB10PB
OpenStack
(SSD)CPU
RAID
()
(CephGlusterFS),
(DAS)
OpenStack
51
March 13, 2015
hypervisor
DAS
-
hypervisor
1U48 10GbE
2U24 10GbE
(ToR)spine
fabric(EoR)
52
March 13, 2015
leaf-spline
bond
OpenStack
(OS)
OpenStack
(OS)hypervisor
hypervisor
Hypervisor
Hypervisor,
(LACP),hypervisor
OS
hypervisorHyper-V
hypervisorkinstanceXen
UbuntuRedHat(
)
hypervisor
hypervisor,
Hypervisor
53
March 13, 2015
UbuntuKinstanceVMware vSphere
OpenStackhypervisor
hypervisor
-
-hypervisor>
OpenStack
-hypervisor
hypervisor
()Hypervisor
hypervisor
-hypervisor
-hypervisor
"
OpenStack
OpenStack
()
OrchestrationHeat
OpenStack
OpenStack
(keystone)
(horizon)
(nova)
(glance)
(neutron)
54
March 13, 2015
(heat)
OpenStack
I-O
OpenStack
OrchestrationTelemetry
Orchestration(Orchestration
Telemetery)Orchestration
Orchestration
OpenStack
OpenStack
OpenStackOpenStack
HAProxy
OpenStack
CorosyncPacemaker(
)
OpenStack
CorosyncPacemaker
55
March 13, 2015
Logstash,Splunkinstanceware Log Insight-
(CPU
)
-hypervisor
-Hypervisor
OpenStack
OpenStack
OpenStackSQLAlchemy
MySQL
Galera,MariaDB>MySQL
(CERN)
CERN2011
3.5
91000
120 PB
100 PB
310 TB
2.5
20000
56
March 13, 2015
6 PB
(LHC),CERN
SCientific Linux RDOOpenStack
2013
CERN cells
flavor
cellcell
API
cellcell2013
cell3
3RabbitMQ
57
March 13, 2015
cell
ImagePropertiestFilter -
(LinuxWindows)
ProjectsToAggregateFilter -
default_schedule_zones -
cellMySQL/
NetApp6
CERNlegacy (nova-network)
CERNMACIP
MACIP
MACIP
OpenStackAPI1(V1)API
Ceph3PB
Scientific Linux 5 6
OrchestrationPuppet
Orchestration
Telemetry
MongoDBAPInova-apicellcell
Telemetry
keystone,glance-apiglance-registrycell
58
March 13, 2015
CERN,
OpenStack
http://openstack-in-production.blogspot.fr
CERN
59
March 13, 2015
.............................................................................................. 62
.............................................................................................. 63
.............................................................................................. 65
..................................................................................................... 70
..................................................................................................... 79
dropbox
61
March 13, 2015
A balance between cost and user requirements dictate what methods and
technologies to use in a cloud architecture.
The user pays only for the storage they actually use. This limit
typically reflects average user consumption during a month. This
does not mean that cloud storage is less expensive, only that it
incurs operating expenses rather than capital expenses.
Examples of such legal frameworks include the data protection
framework of the European Union and the requirements of the
62
March 13, 2015
OpenStack
63
March 13, 2015
OpenStack OpenStack
OpenStack
OpenStack
(
)
64
March 13, 2015
off-premiseon-premise
>
SNMP
CPU
65
March 13, 2015
()
(NOC)
Block Storage resource nodes are commonly configured with advanced RAID
controllers and high performance disks to provide fault tolerance at the
hardware level.
66
March 13, 2015
SSD
67
March 13, 2015
zone
Zone
Selecting the proper zone design is crucial for allowing the Object
Storage cluster to scale while providing an available and redundant
storage system. It may be necessary to configure storage policies that
68
March 13, 2015
You can upgrade Block Storage pools to add storage capacity without
interruption to the overall Block Storage service. Add nodes to the pool
by installing and configuring the appropriate hardware and software
and then allowing that node to report in to the proper storage pool via
the message bus. This is because Block Storage nodes report into the
scheduler service advertising their availability. Once the node is online
and available tenants can make use of those storage resources instantly.
In some cases, the demand on Block Storage from instances may exhaust the
available network bandwidth. As a result, design network infrastructure
that services Block Storage resources in such a way that you can add
capacity and bandwidth easily. This often involves the use of dynamic
routing protocols or advanced networking solutions to add capacity
to downstream devices easily. Both the front-end and back-end storage
network designs should encompass the ability to quickly and easily add
capacity and bandwidth.
partition power
For example, a system that starts with a single disk and a partition power
of 3 can have 8 (2^3) partitions. Adding a second disk means that each has
4 partitions. The one-disk-per-partition limit means that this system can
never have more than 8 disks, limiting its scalability. However, a system
that starts with a single disk and a partition power of 10 can have up to
1024 (2^10) disks.
69
March 13, 2015
As more tenants begin to access data within the cluster and their data
sets grow it is necessary to add front-end bandwidth to service data
access requests. Adding front-end bandwidth to an Object Storage cluster
requires careful planning and design of the Object Storage proxies that
tenants use to gain access to the data, along with the high availability
solutions that enable easy scaling of the proxy layer. We recommend
designing a front-end load balancing layer that tenants and consumers
use to gain access to data stored within the cluster. This load balancing
layer may be distributed across zones, regions or even across geographic
boundaries, which may also require that the design encompass geo-location
solutions.
In some cases, you must add bandwidth and capacity to the network
resources servicing requests between proxy servers and storage nodes.
For this reason, the network architecture used for access to storage
nodes and proxy servers should make use of a design which is scalable.
70
March 13, 2015
This metric is related to scalability.
71
March 13, 2015
Use of DAS impacts the server hardware choice and affects host
density, instance density, power density, OS-hypervisor, and
management tools.
()
Evaluate Compute (server) hardware four opposing dimensions:
[U]
The number of CPU cores, how much RAM, or how much storage a
given server delivers.
You must weigh the dimensions against each other to determine the best
design for the desired purpose. For example, increasing server density
can mean sacrificing resource capacity or expandability. Increasing
resource capacity and expandability can increase cost but decrease
server density. Decreasing cost often means decreasing supportability,
server density, resource capacity, and expandability.
72
March 13, 2015
This decreases density by 50% (only 8 servers in 10 U) if a
full width or full height option is used.
As of the Icehouse release, neither HP, IBM, nor Dell offered
1U rack servers with more than 2 CPU sockets.
This may cause issues for organizations that have
preferred vendor policies or concerns with support and
hardware warranties of non-tier 1 vendors.
2UCPU
(1U)
73
March 13, 2015
1U(
2U,3U4U)
1GbE10GbE
40GbE( 100GbE)
74
March 13, 2015
If this is a requirement, the hardware must support
this configuration. User requirements determine if
a completely redundant network infrastructure is
required.
(OS)
OpenStack
OpenStack
75
March 13, 2015
76
March 13, 2015
OpenStack
Which OpenStack components you choose can have a significant impact on
the overall design. While there are certain components that are always
present, Compute and Image Service, for example, there are other services
that may not need to be present. As an example, a certain design may
not require the Orchestration module. Omitting Orchestration would not
typically have a significant impact on the overall design, however, if the
architecture uses a replacement for OpenStack Object Storage for its
storage component, this could potentially have significant impacts on the
rest of the design.
Orchestration
OpenStack (keystone)
OpenStack (swift) ()
OpenStack (cinder)
OpenStack (glance)
77
March 13, 2015
Clustering
Alerting
The factors for determining which software packages in this
category to select is outside the scope of this design guide.
78
March 13, 2015
If you require any of these software packages, the design must account
for the additional resource consumption (CPU, RAM, storage, and network
bandwidth for a log aggregation solution, for example). Some other
potential design impacts include:
-Hypervisor
Telemetry uses MongoDB.
RESTful
79
March 13, 2015
REST
Swift OpenStack
10 GbE
80
March 13, 2015
10 storage servers each with 12x4 TB disks equaling 480 TB total space
with approximately 160 Tb of usable space after replicas.
3x
2x10 GbE
2x10 GbE
60 Gb
Hadoop (HDFS)
81
March 13, 2015
Ceph Swift REST
82
March 13, 2015
There are special considerations around erasure coded pools.
For example, higher computational requirements and limitations
on the operations allowed on an object; erasure coded pools do
not support partial writes.
83
March 13, 2015
Ceph
5 24x1 TB SSD
10 12x4 TB 480 TB
3 160 TB
REST
3x
2x10 GbE
2x10 GbE
60 Gb
Using an SSD cache layer, you can present block devices directly to
Hypervisors or instances. The REST interface can also use the SSD cache
systems as an inline cache.
84
March 13, 2015
.............................................................................................. 87
.............................................................................................. 89
.............................................................................................. 95
..................................................................................................... 96
..................................................................................................... 99
OpenStack
web
WAN
DNSNTP SNMP
VPNMPLS GRE
85
March 13, 2015
()
HadoopCassandraNuoDBRIAK No-SQL
(VDI)
VDI
IP (VoIP)
VoIP
(QoS)
web
MPLS
(QoS)
VoIP
(HPC)
86
March 13, 2015
VoIP
OpenStack
87
March 13, 2015
(QoS)
VoIP
SLA SLA
IP VLAN
(SPOF)
(MLAG)MLAG
MLAG
88
March 13, 2015
OpenStack
IP OpenStack
(CoS) IP
VLAN
89
March 13, 2015
( VoIP)
( web )
IP IP
IP
IP
30 40
IP
LAN
IP
IP
IP
IP MAC
VLAN 4096
MAC
MLAG
IP ICMP
ARP
MAC MAC
MAC
90
March 13, 2015
ARP MAC()
IP
ARP MAC
Cisco MAC MAC
MAC
MAC
(ToR)
BGP
MAC IP
(ToR)
ICMP
(QoS)
VLAN
IP
91
March 13, 2015
IP
IP
MAC
OpenStack
OpenStack
()
()
OpenStack
(SLA)
92
March 13, 2015
eBGP
iBGP
OpenStack
OpenStack (nova-network)
OpenStack
(neutron)(nova-network)
93
March 13, 2015
IPv6
One of the most important networking topics today is the impending
exhaustion of IPv4 addresses. In early 2014, ICANN announced that they
started allocating the final IPv4 address blocks to the Regional Internet
Registries (http://www.internetsociety.org/deploy360/blog/2014/05/
goodbye-ipv4-iana-starts-allocating-final-address-blocks/). This means
the IPv4 address space is close to being fully allocated. As a result, it will
soon become difficult to allocate more IPv4 addresses to an application
that has experienced growth, or is expected to scale out, due to the lack
of unallocated IPv4 address blocks.
IPv6 IPv6
IPv4
OpenStack IPv6
IPv6
IPv6
VoIP
QoS
OpenStack
(nova-network)
DHCP
94
March 13, 2015
OpenStack (neutron)
OpenFlow
OpenStack
OpenStack
(SLA)
IP
IP
VLAN IDGRE ID MPLS
IP
IPv4 IPv6
IPv4
95
March 13, 2015
IPv6 IP
IPv6 IPv4
OpenStack OpenStack
OpenStack
OpenStack
OpenStack
OpenStack OpenStack
IP
OpenStack
Icehouse OpenStack
96
March 13, 2015
OpenStack
OpenStack
OpenStack OpenStack
OpenStack
OpenStack
OpenStack
OpenStack
IP
OpenStack
(NAT)
IP
NAT IP
DHCP DHCP IP
IP
OpenStack NAT
NAT IPv4
OpenStack
OpenStack
HAproxy
OpenStack (LBaas)
HAproxy
IP(VIP)
IP
NAT NAT
IPv6 NAT
97
March 13, 2015
(LAG)
(HSRP) OpenStack (nova-
network) OpenStack
HSRP
web
OpenStack IPv4 OpenStack
IPv6 Icehouse
DHCPv6 NAT IPv6
IPv6 NAT64DNS64 6to4 IPv6 IPv4
IPv6
IPv4
Icehouse OpenStack
OpenStack
(IGP)
98
March 13, 2015
looking glass
(MTU)MTU
MTU
MTU MTU
MTU QoS
OpenStack
MTU
OpenStack
MTU
(QoS)
IP (VoIP)
QoS
WAN
web
SSL
SSL
IP
GRE ID
99
March 13, 2015
IP
IP HTTP
MLAG
VRRP
GRE
GRE Open
vSwitch
web
OpenStack
web
OpenStack ( MariaDB
RabbitMQ )
100
March 13, 2015
KVM OpenStack
OpenStack
web
IP (VIP)
TCP
101
March 13, 2015
DNS SNMP
10 TCP 512
Kbps 1
4.88 Gbps
QoS DNS
DNS
OpenStack
OpenStack
OpenStack
OpenStack
102
March 13, 2015
OpenStack
103
March 13, 2015
............................................................................................ 105
............................................................................................ 108
............................................................................................ 112
................................................................................................... 114
................................................................................................... 117
openstack
.,
.openstack,
.
,,openstack
105
March 13, 2015
region
OpenStack
OpenStack
OpenStack
106
March 13, 2015
VLANQoS
meshhub spoke,spine leaf,3d Torus
OpenStack
OpenStack
region
107
March 13, 2015
OpenStack
OpenStack
108
March 13, 2015
RPC
OpenStack
zone
zone
IDIcehouseOpenStack
IDID
region
regioncell
hypervisor
multi-region-capable OpenStack
region
swift
swift
109
March 13, 2015
proxy
HTTPswiftHTTP
swift
OpenStack
OpenStackOpenStack
URL
regionregion
APIGUI
region
OpenStack
HAproxyregion
anycast-advertised DNS
region
regionregion
swift
region
OpenStackregion
(RPC)
region
region
region
110
March 13, 2015
OpenStack
VPN
region
OpenStack
OpenStack
OpenStack
OpenStack (keystone)OpenStack(nova)
OpenStack(glance)OpenStack (neutron)
OpenStack(swift)
region
GUIAPI/CLI
swift
Designate
regionDNSzone
DNSregion
Orchestration(heat)Orchestration
region
heat
region
111
March 13, 2015
regionOpenStack
per-regionOpenStack
region
OpenStackOpenStack
OpenStack()
()""()
112
March 13, 2015
1. OpenStack (keystone).
2. OpenStack (glance)
3. OpenStack (nova),
4. OpenStack (cinder)
1. OpenStack (keystone)
2. OpenStack (glance)
3. OpenStack(nova),
4. OpenStack (cinder)
5. OpenStack GUI(horizon),
OpenStack Icehouse
(API,Conductor)
OpenStack
()
region
region
113
March 13, 2015
region
region
region
region25region
50region
OpenStack(RBAC)
policy.json
RBAC
policy.json
rsyncOpenStack
OpenStackregion
regionGUI
regionAPICLIregion
regionregion
regionOpenStack
region
OpenStack
OpenStack
OpenStack
114
March 13, 2015
OpenStack
OpenStackOpenStack
regionregionOpenStack
region
OpenStackregion
region
OpenStackregionOpenStack
region
OpenStack
OpenStackregion
OpenStack
OpenStack
OpenStack GUIOpenStackOpenStack
OpenStack regionOpenStack
region
115
March 13, 2015
regionregion
region
region
HTTP
HTTP
Memcache
region
region(
)
regionregion
region
region
region
region
OpenStack
(RPO)(RTO),
116
March 13, 2015
OpenStack
region
4region
regionSwift
Glanceregion
regionKeystone
API
DNSDesignateDNS
region
Desginate
region
Telemetryregionregion
Ceilometerregion
OpenStack
region
region
OpenStack
Keystone
OpenStackregion
region
region
regionregion
117
March 13, 2015
OpenStack
web
7*242
webNoSQL
OpenStack
OpenStackGUI
region
OrchestrationTelemetry
regionOpenStack
OpenStackKVMhypervisor
OpenStack
region
118
March 13, 2015
regionDNS
DNS
regionheat
Web Apache
DNSuser_data
Telemetryregion
HeatMongoDB shard
swift
Telemetryshard
regionregion
Orchestration
PuppetChef
OrchestrationOpenStack
OpenStack
OpenStack
OpenStackLBaaS,
OpenStackLBaaSOpenStack
119
March 13, 2015
OpenStackCDN
100%
hop
OpenStack
web
OpenStack
120
March 13, 2015
121
March 13, 2015
............................................................................................ 124
............................................................................................ 127
............................................................................................ 132
................................................................................................... 134
................................................................................................... 137
,
OpenStack(OpenStack
OpenStack)OpenStack
OpenStack(OpenStackAmazon
Web Services).
OpenStack
OpenStack
(CMP)CMPCMP
123
March 13, 2015
single CMP that can address all needs in all scenarios. Whereas most of the
sections of this book talk about the aspects of OpenStack, an architect
needs to consider when designing an OpenStack architecture. This section
will also discuss the things the architect must address when choosing
or building a CMP to run a hybrid cloud design, even if the CMP will be a
manually built solution.
124
March 13, 2015
":
OpenStack
AWS
hypervisor
hypervisor
hypervisor
125
March 13, 2015
hypervisor
:
(CMP)
Rightscale,Gravitent,Scalr,CloudFormsManageIQ.
CMP
IT
IT
(IaaS)(SaaS)
CMP
API
126
March 13, 2015
----
AWS
API API
API
API
AWSOpenStackAWSAPI
OpenStackAWS API
API
API
CMP
127
March 13, 2015
OpenStack
OpenStack,
OpenStack
DefCore
"OpenStack
OpenStackHavana
OpenStackweb OpenStackAzure
OpenStackCMP
Treats
OpenStack
vCPU,RAM
128
March 13, 2015
hypervisor32GB242GB
2GB
1.
2.
3.
4.
OpenStack
API
129
March 13, 2015
OpenStack
/
LDAPOpenStack
3
API
APISSL
hypervisor
Hypervisor(
Xen)hypervisor
"hypervisorHypervisor
hypervisor
OpenStack
CMP
130
March 13, 2015
Telemetry(ceilometer)OpenStack
(
),CeilometerNoSQL
MongoDBOpenStackCeilometer
CMP
CMP
OpenStackCeilometer
Orchestration
Orchestration
Amazon Web Services
CMP
OpenStackOpenStack
OpenStack
OpenStack
OpenStack(nova):hypervisor
Hyper-VOpenStack
AzureKVM
OpenStack(neutron)(nova-network)
Telemetry (ceilometer):Telemetry
131
March 13, 2015
Icehouse
ChefPuppet
APIOpenStack(OpenStack
)API
SDKjClouds
132
March 13, 2015
At massive scale, even when dealing with a cloud that offers an
SLA with a high percentage of uptime, workloads must be able to
recover at short notice.
133
March 13, 2015
orchestration API,
hypervisorKVM,Xen,
ESXihypervisor
134
March 13, 2015
hypervisor
NoSQL
MongoDBMySQL
OpenStack (trove)OpenStack
MySQLNoSQLAWS
AWSNoSQL
Orchestration(heat)OpenStackNoSQL
135
March 13, 2015
CMP
(HA)
API
API
(HA)--
-
region
Hadoop
HDFSIBM
Ceph,Ceph
136
March 13, 2015
OpenStack
OpenStack
()
(CMP)CMP
CMP
Telemetry
137
March 13, 2015
OpenStack
Telemetry
TelemetryCMP
CMPOpenStackAPI,
CMP
API
OpenStack
OpenStackweb
(AWS),
OpenStack-to-AWS:
BAWS
CMPAPI
CMP>
API
WebCMPEC2 API
IPIPHAProxyCMP
AWSCloudWatchCloudFormation
CMP
ManageIQ,jClouds,JumpGate
138
March 13, 2015
/
C
OpenStack
OpenStack
1OpenStackMySQL,
OpenStack
OpenStack
Swift OpenStack
For Block Storage, the replication is a little more difficult, and involves
tools outside of OpenStack itself. The OpenStack Block Storage volume
is not set as the drive itself but as a logical object that points to a
physical back end. The disaster recovery is configured for Block Storage
for synchronous backup for the highest level of data protection, but
asynchronous backup could have been set as an alternative that is not
as latency sensitive. For asynchronous backup, the Block Storage API
139
March 13, 2015
makes it possible to export the data and also the metadata of a particular
volume, so that it can be moved and replicated elsewhere. More information
can be found here: https://blueprints.launchpad.net/cinder/+spec/
cinder-backup-volume-metadata-support.
CMPCMP
DRDB
MySQL
MySQL
Swift
>Ceph
OpenStack
>
140
March 13, 2015
8
............................................................................................ 142
............................................................................................ 144
............................................................................................ 146
500
OpenStack
OpenStack
OpenStack
Raw
()() IP
141
March 13, 2015
OpenStack
()
OpenStack
OpenStack
OpenStack
web API
(CLI)API
OpenStack
142
March 13, 2015
OpenStack
API
SLA SLA
OpenStack
(OpEx)
OpenStack
143
March 13, 2015
OpenStack
OpenStack
(/)(AAA)
6 [105]
OpenStack
(PUE)
OpenStack
OpenStack
OpenStack
OpenStack
144
March 13, 2015
OpenStack (Region)
() API
OpenStack
API
()
API
OpenStack
OpenStack
OpenStack
OpenStack
CERN Rackspace
OpenStack
OpenStack
()
145
March 13, 2015
nova
SSD10GbE
OpenStack
Puppet Chef
146
March 13, 2015
OpenStack
OpenStack
OpenStack OpenStack
OpenStack
OpenStack
OpenStack
OpenStack
147
March 13, 2015
148
March 13, 2015
....................................................................... 149
........................................................................... 151
..................................................................................... 152
......................................................................................... 154
OpenStack OpenStack .................................................................. 156
......................................................................................... 158
OpenStack
(
)
OpenStack
API
149
March 13, 2015
OpenStack
KVM
ESXi
ESXi
KVM
OpenStack
ESXi
VMDK QEMU VMDK
VMFS
VMFS VMFS VMFS
OpenStack
OpenStack
150
March 13, 2015
VMware
vCenter vCenter ESXi
Icehouse VMware
(DRS)
OpenStack
OpenStack vSphere nova-compute
vCenter ESXi
( nova-compute ESXi
vCenter ) nova-compute
vCenter ESXi
vCenter vSphere
vSphere nova-compute
looking glass
BGP
OpenStack
151
March 13, 2015
OpenStack OpenStack
(ISIS) OpenFlow
Open vSwitch
VLAN
BGP
(SDN)SDN
SDN
OpenStack
OpenStack SDN
SDN SDN
OpenStack
SDN
SDN
OpenStack ML2
VLAN
OpenStack SDN
152
March 13, 2015
OpenStack SDN
153
March 13, 2015
(VDI)
OpenStack
154
March 13, 2015
OpenStack
OpenStack
155
March 13, 2015
OpenStack OpenStack
OpenStack
OpenStack OpenStack OpenStack
OpenStack
156
March 13, 2015
OpenStack
OpenStack OpenStack
VLAN VLAN
( OpenStack OpenStack)
OpenStack OpenStack
OpenStack
Heat
Heat OpenStack
OpenStack-on-OpenStack (TripleO)
TripleO OpenStack
OpenStack
KVM
In the case of running smaller OpenStack clouds for testing purposes, and
performance would not be a critical factor, QEMU can be utilized instead.
It is also possible to run a KVM hypervisor in an instance (see http://
davejingtian.org/2014/03/30/nested-kvm-just-for-fun/), though this is
not a supported configuration, and could be a complex solution for such a
use case.
157
March 13, 2015
OpenStack
CPU
OpenStack
158
March 13, 2015
159
March 13, 2015
10
Data Protection framework of the European Union: Guidance on Data
Protection laws governed by the EU.
Image Service property keys: Glance API property keys allows the
administrator to attach custom characteristics to images.
161
March 13, 2015
162
March 13, 2015
A.
................................................................................................... 163
............................................................................................ 164
OpenStack ............................................................................ 165
OpenStack ............................................................................ 165
LaunchpadBug .............................................................................. 165
The OpenStack ........................................................... 166
............................................................................................ 167
OpenStack ................................................................................ 167
OpenStackOpenStack
OpenStack
OpenStack
/
OpenStack
Ubuntu 14.04
OpenStack
163
March 13, 2015
OpenStack
OpenStack
OpenStack(HTML)
OpenStack(PDF)
OpenStack API v2
OpenStack API V2
OpenStack API v2
OpenStack API v1
The Training Guides offer software training for cloud administration and
management.
During the set up or testing of OpenStack, you might have questions about
how a specific task is completed or be in a situation where a feature does
not work correctly. Use the ask.openstack.org site to ask questions
and get answers. When you visit the http://ask.openstack.org site,
scan the recently asked questions to see whether your question has
already been answered. If not, ask a new question. Be sure to give a clear,
164
March 13, 2015
OpenStack
A great way to get answers and insights is to post your question or
problematic scenario to the OpenStack mailing list. You can learn from
and help others who might have similar issues. To subscribe or view the
archives, go to http://lists.openstack.org/cgi-bin/mailman/listinfo/
openstack. You might be interested in the other mailing lists for specific
projects or development, which you can find on the wiki. A description of
all mailing lists is available at http://wiki.openstack.org/MailingLists.
OpenStack
The OpenStack wiki contains a broad range of topics but some of the
information can be difficult to find or is a few pages deep. Fortunately,
the wiki search feature enables you to search by title or content. If you
search for specific information, such as about networking or nova, you
can find a large amount of relevant material. More is being added all the
time, so be sure to check back often. You can find the search box in the
upper-right corner of any OpenStack wiki page.
LaunchpadBug
The OpenStack community values your set up and testing efforts and wants
your feedback. To log a bug, you must sign up for a Launchpad account at
https://launchpad.net/+login. You can view existing bugs and report bugs
in the Launchpad Bugs area. Use the search feature to determine whether
the bug has already been reported or already been fixed. If it still seems
like your bug is unreported, fill out a bug report.
trace
"Juno release" vs git commit
bc79c3ecc55929bac585d04a03475b72e06a3208
165
March 13, 2015
Ubuntu 14.04
Launchpad Bug
Bugs: (ironic)
Bugs: (sahara)
Bugs: (trove)
Bugs: (heat)
Bugs: (ceilometer)
Bugs: (marconi)
The OpenStack
The OpenStack community lives in the #openstack IRC channel on the
Freenode network. You can hang out, ask questions, or get immediate
feedback for urgent and pressing issues. To install an IRC client or use a
browser-based client, go to http://webchat.freenode.net/. You can also
use Colloquy (Mac OS X, http://colloquy.info/), mIRC (Windows, http://
www.mirc.com/), or XChat (Linux). When you are in the IRC channel and want
to share code or command output, the generally accepted method is to use
a Paste Bin. The OpenStack project has one at http://paste.openstack.org.
Just paste your longer amounts of text or logs in the web form and you
166
March 13, 2015
get a URL that you can paste into the channel. The OpenStack IRC channel
is #openstack on irc.freenode.net. You can find a list of all OpenStack IRC
channels at https://wiki.openstack.org/wiki/IRC.
OpenStack
LinuxOpenStack
Debian: DebianOpenStack
Ubuntu: ubuntuOpenStack
167
March 13, 2015
6to4
IPv4IPv6IPv6
OpenStack
cinder
ceilometer
OpenStack
cinder
OpenStack
OpenStacknova
OpenStackwebhorizon
A platform that provides a suite of desktop environments that users may log in
to receive a desktop experience from any location. This may provide general use,
development, or even homogeneous testing environments.
glance
OpenStack
heat
OpenStack
horizon
webOpenStack
2()
169
March 13, 2015
IaaS
IaaS
IaaS
OpenStack
glance
IOPS
IOPS(/)
keystone
OpenStack
OSI
neutron
OpenStackOpenStack
nova
OpenStack
OpenStack
OpenStackswift
Open vSwitch
Open vSwitchApache2.0
(NetFlow, sFlow, SPAN, RSPAN, CLI, LACP, 802.1ag)
OpenStack
OpenStack
web
170
March 13, 2015
OpenStackApeche2.0
swift
OpenStack
Telemetry
An integrated project that provides metering and measuring facilities for
OpenStack. The project name of Telemetry is ceilometer.
trove
OpenStack
Xen
Xenhypervisor,
171