OpenStack

Ceilometer
Tong Li (IBM)
Brad Topol (IBM)

Agenda

What is Ceilometer?
Brief history of Ceilometer
Meters of Ceilometer
Workflow
Ceilometer Architect & Building Blocks
Collecting Process
Pipeline
Transformer
Publisher
Collector
Data store drivers
Alarm
Dispatcher
Ceilometer ReSTful API
Ceilometer Python Client
Ceilometer in Devstack
Inter-Component Security & High Availability
Who are the users of OpenStack Ceilometer
Ceilometer Road Map
Q & A (Demo maybe)

Who are we?

Tong Li
IBM senior software engineer
Twitter @email4tong
Email: litong01@us.ibm.com

Brad Topol
IBM distingushed engineer
Twitter @bradtopol
Email: btopol@us.ibm.com

What is Ceilometer?
OpenStack measurement project
Julien Danjou Ceilometer PTL
A system that keeps track of what is happening in
OpenStack clusters. It was originally created for
billing but it has been evolved into an all purpose
measurement repository. It can be extended for
purposes such as performance report,
troubleshooting, capacity analytics, smart
scheduling, optimization, ultimately BIG DATA etc.

Brief History of Ceilometer

Before year 2012 billing has been left out of OpenStack core so far
as it was not the primary problem and was non-trivial, yet almost
every OpenStack deployment needs a way to track usage
information
Started in May 2012 by developers from Canonical, DreamHost,
RedHat, Dell, Intel, AT&T, etc.
Targeted to do metering, rating, billing
Grizzly objective: The project aimed to become the infrastructure to
collect measurements within OpenStack so that no two agents
would need to be written to collect the same data.
Havana objective: remains the same as Grizzly objective plus
become an integrated OpenStack project.
Current PTL: Julien Danjou
91 contributors, 7 core reviewers

Meters of Ceilometer
(from Nova)

Meters of Ceilometer
(from Neutron)

Additional measurements:
http://docs.openstack.org/developer/ceilometer/measurements.html

Workflow
Collect

Transform

Publish

Store

Read

Collect interesting info from OpenStack components

(or any other components), create meters (or samples)
Transform meters into other meters if needed
Publish meters to any destination (including
Ceilometer itself)
Store received meters into data store
Read data via Ceilometer REST API

Ceilometer Architecture

Collecting
Notification bus (AMQP)

Nova
API

queue name: notification.info

Neutron
API

Cinder
API

Glance
API

Swift
API

Polling

Notification Handlers, Central Agents

Publishing
pipeline

Publishing

New in
Grizzly
!

Pipeline
Ceilometer
collector & agent

Meter

Transformer
Transformer
Transformer

Publisher
Publisher
Publisher

Pipeline: a set of transformers mutating meters into

something that publishers know how to send to external
systems.
Configure pipelines @pipeline.yaml

Receiver
Receiver
Receiver

New in
Grizzly
!

Transformer
Meter
Name=cpu.time
Value=5
Timestamp=T

Meter
Name=cpu.time
Value=20
Timestamp=T+1

Transformer

Meter
Name=cpu.time
Value=80
Timestamp=T+2

Transform meters into new meters!

Meter
Name=cpu.percentage
Value=9
Timestamp=T+2

New in
Grizzly
!

Publisher
AMQP
signed message
Meter

Ceilometer
RPC
Publisher

Ceilometer
Message queue

Other
Publisher

External system

Billing, monitoring,
alarming, statistics,
capacity planning

Data Store
Message Queue
ceilometer.collector.metering

Meter

Dispatchers

SQL
Database
Default backend

HBase

MongoDB
Database

New in
Grizzly

New in
Havana

Multiple backend choices

IBM DB2

Alarm

New in
Havana

Both alarm evaluator and alarm notifier are

Ceilometer services
1. Alarm evaluator uses Ceilomter public APIs to
pull meters to produce alarms according to
alarm definition, then place the alarm
message onto the alarm queue
2. Alarm notifier read the alarm queue then
inform external systems (log, rest, test)

Dispatchers

New in
Havana

Multiple dispatchers support allows Ceilometer to be extended easily with many different outlet
for all meters coming into Ceilometer. Dispatchers can be easily developed and deployed
to add more features. For example, sending email, text messages.
1. Dispatcher only needs to implement one method.
2. By default, database dispatcher is used even if the configuration file has no trace of any
dispatchers.
3. To configure multiple dispatchers:
[ceilometer.dispatcher]
file = ceilometer.collector.dispatcher.file:FileDispatcher
database = ceilometer.collector.dispatcher.database:DatabaseDispatcher

Ceilometer ReSTful API

request
Ceilometer API
Consumers
Billing, monitoring,
alarming, statistics,
capacity planning

Ceilometer API
HTTP
REST
w/ JSON &
XML

Storage abstraction
layer

Raw events:
GET /v2/meters/vcpus
Statistics (sum, average, min, max):
GET /v2/meters/vcpus/statistics

New in
Grizzly
!

Filter:
GET /v2/meters/vcpus?q.field=project_id&q.value=foobar&q.op=eq
Group statistics by period:
GET /v2/meters/vcpus/statistics?period=3600

Database

Ceilometer ReSTful API

response
Response can be in JSON or XML

Raw events:
/v2/meters/storage.object.size

Statistics (sum, average, min, max):

/v2/meters/storage.api.request/statistics

Ceilometer Python Client

Ceilometer in Devstack
devstack localrc
Enable the ceilometer metering services
enable_service ceilometer-acompute,ceilometer-acentral,ceilometer-collector

Enable the ceilometer alarming services

enable_service ceilometer-alarm-evaluator,ceilometer-alarm-notifier

Enable the ceilometer api services

enable_service ceilometer-api

Inter-Component Security

Ceilometer High Availability

Who uses Ceilometer?

https://etherpad.openstack.org/p/CeilometerFeeedback

Roadmap
Grizzly

Incubated Project
Integration
with Horizon
Agents for other
components
o Swift
o Ceph?
o Nicira?
SQLAlchemy storage
driver
Multi-Publisher
API v2
o User accessible API

o More aggregation
o Multi-dimension

Havana

Integrated Project
Integration with Horizon
Publishing meters to other
systems
Enhance SQL driver
Alarming
Integration with Heat
Deprecating APIv1
Completing APIv2
Move publishing part to
Oslo and other projects
Tighter integration with
Nova
Nova-scheduler integration

Icehouse

OpenStack Design
Summit in Hong Kong

Questions?
http://launchpad.net/ceilometer
http://docs.openstack.org/developer/ceilometer
http://wiki.openstack.org/ceilometer
Freenode:
Mailing List:

#openstack-metering
openstack-dev [ceilometer]