Professional Documents
Culture Documents
Thursday, 7 February 13
Hello
Craig Dunn
Puppet user since 2008 as an IT contractor
Started with 0.24
Joined Puppet Labs in June 2012
@crayfishX
Freenode IRC: crayfishx
Thursday, 7 February 13
Agenda
How people typically design Puppet
Real-world case study
Thinking about components
Designing Puppet for your users
Node classification
Data separation
Thursday, 7 February 13
Background
Thursday, 7 February 13
Designing Puppet
Thursday, 7 February 13
Designing Puppet
Node Classification
Modules
Thursday, 7 February 13
Down the road...
Your infrastructure grows
Business requirements will change
Your Puppet code feels bulky and high
maintenence
There will always be edge cases eventually
You decide it needs refactoring
Thursday, 7 February 13
Danger Signs
Resources being declared in two modules
You dont know where your
implementation fits
Lots of logic at a node level
Repetition and duplication
The if statement is your go-to-guy
Thursday, 7 February 13
Write good modules
Thursday, 7 February 13
Thinking beyond the
module....
Thursday, 7 February 13
Node-level logic
Thursday, 7 February 13
Node-level logic
node basil {
class { apache:
version => latest,
}
class { motd: }
class { ssh: }
class { users:
default_shell => /bin/false,
}
Class[ssh] -> Class[users]
}
Thursday, 7 February 13
Node-level logic
Thursday, 7 February 13
Designing Puppet
Provide business logic to classification
Provide an abstraction layer for
implementation of components
Make code adaptable to complex
requirements
Reduce node-level logic
Reduce functionality overlap
Thursday, 7 February 13
What is the worse
thing that is
going to happen to
your Puppet code?
Thursday, 7 February 13
Business requirements
Thursday, 7 February 13
Business logic does not
align with technology
Thursday, 7 February 13
Case study
Thursday, 7 February 13
Case study
We have 3
applications we
need to deploy using
Puppet
Thursday, 7 February 13
The business view
Application X
Application Y Application Z
Thursday, 7 February 13
Go forth and
Puppetize!
Thursday, 7 February 13
Go forth and
Puppetize!
And we jumped right in...
Thursday, 7 February 13
Things got painful
Thursday, 7 February 13
Problems
These applications arent that different
They seem to share a whole bunch of
similarities
Implementation differed on different
environments and locations
Writing 3 separate modules creates
conflicts and duplication
Thursday, 7 February 13
Our code was hacky
Thursday, 7 February 13
We are trying to code
business logic.
Thursday, 7 February 13
Stop thinking about
what it looks like
Thursday, 7 February 13
What we realised
Thursday, 7 February 13
The business view
Application X
Application Y Application Z
Thursday, 7 February 13
The technical reality
Application X
ApplicationA
Ypplication Z
Thursday, 7 February 13
We only have one
application
Implemented many
different ways
Thursday, 7 February 13
So we had an idea!
Thursday, 7 February 13
class profiles::x {
include tomcat
include mysql
include componenta
include componentb
componentb::resource { name:
ensure => present,
}
}
class profiles::y {
include tomcat
include mysql
include componenta
include componentc
include componentd
}
class profiles::z {
include tomcat
include mysql
include componenta
include componentb
include componentd
include dependancy
Class[dependancy] -> Class[componentd]
}
Thursday, 7 February 13
Use inheritance for abstraction within profiles
class profiles::application {
include tomcat
include mysql
include componenta
}
Thursday, 7 February 13
Profiles and
Components
Resources
Thursday, 7 February 13
Profiles and
Components
Resources
Thursday, 7 February 13
Profiles and
Components
Profiles : Implementation
Resources
Thursday, 7 February 13
In reality it was worse
Thursday, 7 February 13
In reality it was worse
2 different deployment types made up of
over 15 server types each
Thursday, 7 February 13
In reality it was worse
2 different deployment types made up of
over 15 server types each
10+ locations
Thursday, 7 February 13
In reality it was worse
2 different deployment types made up of
over 15 server types each
10+ locations
4 environment types
Thursday, 7 February 13
In reality it was worse
2 different deployment types made up of
over 15 server types each
10+ locations
4 environment types
Every installation was an edge case!
Thursday, 7 February 13
In reality it was worse
2 different deployment types made up of
over 15 server types each
10+ locations
4 environment types
Every installation was an edge case!
My slides werent big enough.
Thursday, 7 February 13
Lessons learned
Granularity is good
Dont assume business logic will directly
translate to technology
Abstraction is awesome.... but thats
nothing new....
Thursday, 7 February 13
Abstraction is a core
principle of coding
Functions are abstracted by methods
Methods abstracted by classes and modules
They are abstracted with libraries
Puppet is code!
Thursday, 7 February 13
Puppet is all about
abstraction
Data is abstracted by Hiera
Providers are abstracted by types
Resources are abstracted by classes
Classes are abstracted by modules
Thursday, 7 February 13
Puppet is all about
abstraction
Data is abstracted by Hiera
Providers are abstracted by types
Resources are abstracted by classes
Classes are abstracted by modules
Modules are abstracted by profiles
Thursday, 7 February 13
Focussing on
Abstraction
Weve turned business logic into a
technology stack
Can we translate that back into business
logic?
Why would we even want to do that?
Thursday, 7 February 13
UAT Cluster node
Our example configuration model:
include security
include users
include ntp
include ssh::server
include customapp
include tomcat::server
class { jenkins:
require => Class[tomcat::server],
}
include mysql
database { apptest:
ensure => present,
}
Thursday, 7 February 13
Think about the users
Meet John, Susan and Bill.
Thursday, 7 February 13
John is a Sysadmin
Thursday, 7 February 13
Susan is an application
specialist
Thursday, 7 February 13
Bill is an IT manager
Thursday, 7 February 13
What do they care
about?
Thursday, 7 February 13
In Puppet
Thursday, 7 February 13
Introducing Roles
Thursday, 7 February 13
A node can only have
one role
A role can include as many profiles as
required to define itself
If a node requires two roles, it has by
definition become a new role
Thursday, 7 February 13
A node can only have
one role
A role can include as many profiles as
required to define itself
If a node requires two roles, it has by
definition become a new role
Something couldnt be a lion and a
kangaroo at the same time!
Thursday, 7 February 13
It would be a Lingaroo
Thursday, 7 February 13
Roles
One-to-one to nodes
One-to-many to profiles
Only implement profiles
Thursday, 7 February 13
Example role
class role::uat_server {
include profiles::base
include profiles::customapp
include profiles::test_tools
}
Thursday, 7 February 13
Classification
Thursday, 7 February 13
Classification
node craig.puppetlabs.vm {
include roles::uat_server
}
Thursday, 7 February 13
Classification
Thursday, 7 February 13
The Stack
Resources
Thursday, 7 February 13
The Stack
Resources
Thursday, 7 February 13
The Stack
Profiles : Implementation
Resources
Thursday, 7 February 13
The Stack
Roles : Business Logic
Profiles : Implementation
Resources
Thursday, 7 February 13
Terminology
Thursday, 7 February 13
Naming conventions
Components should be named after what
they manage (apache, ssh, mysql)
Profiles should be named after the logical
stack they implement (database, bastion,
email)
Roles should be named in business logic
convention (uat_server, web_cluster,
application, archive)
Thursday, 7 February 13
Hiera Overview
Lets talk about data!
Thursday, 7 February 13
Managing infrastructure
Dev
Thursday, 7 February 13
Managing infrastructure
Dev
QA
Thursday, 7 February 13
Managing infrastructure
Dev
QA
Production
Thursday, 7 February 13
Managing infrastructure
Dev
QA DC1
Production
Thursday, 7 February 13
Managing infrastructure
Dev
Production
Thursday, 7 February 13
Managing data in
Puppet is hard.
Thursday, 7 February 13
Without Hiera?
if ( $::environment == dev ) {
$ntpserver = 192.168.2.1
} else {
if ( $::fqdn == host4.mycorp.com) {
$ntpserver = 127.0.0.1
} else {
$ntpserver = 213.21.6.4
}
}
Thursday, 7 February 13
With Hiera?
$ntpserver = hiera(ntpserver)
Thursday, 7 February 13
Hierarchical lookups
Thursday, 7 February 13
Separation of data from code
Thursday, 7 February 13
Pluggable Backends
Source data from multiple locations
Data source is abstracted from code
Thursday, 7 February 13
Pluggable Backends
Source data from multiple locations
Data source is abstracted from code
hiera-gpg hiera-redis
hiera-http hiera-json
hiera-mysql hiera-zookeeper
Thursday, 7 February 13
Data Separation
Thursday, 7 February 13
Profiles and Hiera
Thursday, 7 February 13
The Stack
Roles : Business Logic
Profiles : Implementation
Resources
Thursday, 7 February 13
The Stack
Roles : Business Logic
Profiles : Implementation
Hiera:
Data
Components: Resource modelling
Resources
Thursday, 7 February 13
Classification
Thursday, 7 February 13
Leveraging an ENC
You can classify your nodes however you
want
Puppet Dashboard
Enterprise Console
Foreman
Site.pp
Custom script
Thursday, 7 February 13
Leveraging an ENC
Thursday, 7 February 13
The Stack
Roles : Business Logic
Profiles : Implementation
Hiera:
Data
Components: Resource modelling
Resources
Thursday, 7 February 13
The Stack
Roles : Business Logic Classifier
Profiles : Implementation
Hiera:
Data
Components: Resource modelling
Resources
Thursday, 7 February 13
Key benefits
Reduced node-level logic to a role.
Gain the ability to be flexible with
implementation
Business logic improves managability by
non-Puppet users
Edge cases are now easy to solve
Thursday, 7 February 13
Enough Preaching!
Thursday, 7 February 13
This is not the way to
design Puppet... Its a
way.
Thursday, 7 February 13
Can I implement this
design without roles?
Thursday, 7 February 13
Can I implement this
design without roles?
Yes.
You lose the layer of abstraction that
exposes business logic
Thursday, 7 February 13
Can my roles be
defined in my ENC?
Thursday, 7 February 13
Can my roles be
defined in my ENC?
Yes.
Keeping it in code makes it versionable
Thursday, 7 February 13
Cant I just use Hiera
to define profiles?
Thursday, 7 February 13
Cant I just use Hiera
to define profiles?
Technically yes.
You lose the flexibility to implement code
logic in profiles and it may become
restrictive
You could possibly use: https://github.com/
ripienaar/hiera-puppet-nodes
Thursday, 7 February 13
The fundamental
concepts....
Thursday, 7 February 13
The fundamental
concepts....
Thursday, 7 February 13
The fundamental
concepts....
Thursday, 7 February 13
The fundamental
concepts....
Thursday, 7 February 13
The fundamental
concepts....
Thursday, 7 February 13
Thank you. Questions?
Follow me at @crayfishX
Bug me on Freenode: crayfishx
Enjoy the rest of Puppet Camp!
In memory of Giles Constant, who spent many nights debating Puppet design patterns with me over copious amounts of beer
and helped me on my journey of discovery learning how to implement Puppet properly. R.I.P
Thursday, 7 February 13