Science Officer's Blog

Thursday, November 14, 2013

Centos Fast Puppet Master

Ever been on Centos 6 and want a Puppet Master fast?

Try this:

curl -L http://git.io/cqaaQQ | sh

Wednesday, November 13, 2013

EPEL:

Enterprise Linux 5:

# rpm -Uvh http://dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

Enterprise Linux 6:

# rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm

Puppet Labs

Enterprise Linux 5:

# rpm -ivh http://yum.puppetlabs.com/el/5/products/i386/puppetlabs-release-5-7.noarch.rpm

Enterprise Linux 6:

# rpm -ivh http://yum.puppetlabs.com/el/6/products/i386/puppetlabs-release-6-7.noarch.rpm

Ubuntu:




Debian Squeeze:



# wget http://apt.puppetlabs.com/puppetlabs-release-squeeze.deb

# dpkg -i puppetlabs-release-squeeze.deb

# apt-get update



Ubuntu Precise:



# wget http://apt.puppetlabs.com/puppetlabs-release-precise.deb

# dpkg -i puppetlabs-release-precise.deb

# apt-get update

Tuesday, September 24, 2013

Puppet Time through abuse of inline_template

Stardate 91334.3

I was asked a pretty reasonable question about puppet:

Can I get access to the time in Puppet without resorting to writing a fact?

This, seemingly reasonable task, is not so easy to do. Puppet does not define this for you. However, we can use the inline_template() function for great good.

For the uninitiated, inline_template() calls out to the ERB templating processor without having to have a template file. This is very useful for simple file resources or conjunction with the file_line resource from the Puppet Labs stdlib module.

file { '/etc/motd':
   ensure  => file,
   content => inline_template("Welcome to <%= @hostname %>"),
}

However we can abuse this by doing anything in ERB that we can do in Ruby. A friend of mine, famililar with the jinja templating system from Python, remarked: 'so its like PHP, and I'm not even being derogatory.' This means we can acces time using Ruby's built in time modules.

$time = inline_template('<%= Time.now %>')

However this is being evaluated on the Puppet Master, not the node. So if the two are in different timezones, what then? The first way to improve this is to set it to utc.

$time = inline_template('<%= Time.now.utc %>')

But we can actually go further and define two variables, one for time in UTC of catalog compilation and one for local time for the checking in node. While we don't have a fact for the time on the node, we do have a fact for its timezone.

$time_utc = inline_template('<%= Time.now.utc %>')
$time_local = inline_template("<%= (@time + Time.zone_offset(@timezone)).strftime('%c') %>")

We use the strftime('%c') to strip the UTC timezone label off of the time.

Going further:

We can take this a step further by using the inline_template() function to do time comparisons:


$go_time = "2013-09-24 08:00:00 UTC" # We could seed this in hiera!

$time = inline_template("<%= Time.now.utc %>")

if str2bool(inline_template("<%= Time.now.utc > Time.parse(@go_time) %>")){

    notify { "GO GO GO: Make the changes!": }

}

What the above code does is gate changes based on time. This allows us to 'light the fuses' and only make changes to production after a certain time, after a downtime window begins for instance. Note that the Puppet clients are still going to check in on their own schedule, but since we know what time our downtime is starting, we can use Puppet to make sure they kick off a Puppet run at the right time.


$go_time   = "2013-09-24 08:00:00 UTC" # We could seed this in hiera!
$done_time = "2013-09-24 12:00:00 UTC" # We could seed this in hiera, too!

$time = inline_template("<%= Time.now.utc %>")

if str2bool(inline_template("<%= Time.now.utc > Time.parse(@go_time) %>")){

  notify { "GO GO GO: Make the changes!": }

  cron { 'fire off the puppet run':
     command => 'puppet agent --no-daemonize',
     day     => '24', # we can seed the date here in hiera, albiet more verbosely 
     hour    => '8',
     minute  => '1',
     user    => 'root',
     ensure  => 'absent',
  }

} else { 

  cron { 'fire off the puppet run':
     command => 'puppet agent --no-daemonize',
     day     => '24', # we can seed the date here in hiera, albiet more verbosely 
     hour    => '8',
     minute  => '1',
     user    => 'root',
     ensure  => 'present',
  }
}

What is this doing? We put this code out at 4pm. Get some dinner. Log in at 7:30 pm and wait for our 8:00 pm downtime. In Puppet runs before 8:00 pm a cronjob will be installed that will effecat a Puppet run precisely one minute after the downtime begins. In all Puppet runs before 8:00 pm, the resources that are potentially hazardous are passed over. But in all Puppet runs after 8:00 pm, the new state is ensured and the cronjob is removed. Then this code, which should define the new state of the system, can be hoisted into regular classes and defined types.

Monday, September 16, 2013

Custom categories with Puppet data in modules

In the old way of doing things, we would have a hierarchy in our hiera.yaml that looked something like this:


:hierarchy:
  - defaults
  - %{clientcert}
  - %{environment}
  - global

In the new way the hierarchy has been renamed categories, and each level of it is a category.

We can define category precedence in the system wide hiera.yaml, the module specific hiera.yaml, and the binder_config.yaml

The following binder_config.yaml will effectively insert the species category into the category listing:


---
version: 1
layers:
  [{name: site, include: 'confdir-hiera:/'},
   {name: modules, include: ['module-hiera:/*/', 'module:/*::default'] }
  ]
categories:
  [['node', '${fqdn}'],
   ['environment', '${environment}'],
   ['species', '${species}'],
   ['osfamily', '${osfamily}'],
   ['common', 'true']
  ]

This means we can use the species category if one is defined in a module. An example hiera.yaml from such a module is:

---
version: 2
hierarchy:
  [['osfamily', '$osfamily', 'data/osfamily/$osfamily'],
   ['species', '$species', 'data/species/$species'],
   ['environment', '$environment', 'data/env/$environment'],
   ['common', 'true', 'data/common']
  ]

Which means when we run Puppet...


root@hiera-2:/etc/puppet# FACTER_species='human' puppet apply modules/startrek/tests/init.pp 
Notice: Compiled catalog for hiera-2.green.gah in environment production in 1.07 seconds
Notice: janeway commands the voyager
Notice: /Stage[main]/Startrek/Notify[janeway commands the voyager]/message: defined 'message' as 'janeway commands the voyager'
Notice: janeway is always wary of the section 31
Notice: /Stage[main]/Startrek/Notify[janeway is always wary of the section 31]/message: defined 'message' as 'janeway is always wary of the section 31'
Notice: Finished catalog run in 0.11 seconds

You can see full example code in the startrek module.

You can pre-order my book, Pro Puppet 2nd Ed, here.

Puppet 3.3 and Data in Modules

Puppet 3.3 was released last week. As part of that release, hiera2 and puppet data in modules is available in the testing mode. I have been working with William van Hevelingen to build an example module/tutorial on Puppet data in Modules. Our example module is available at: https://github.com/pro-puppet/puppet-module-startrek. We hope to expand this further as the community learns more about how to use this new feature.

Saturday, July 20, 2013

Puppet Case insensitivity

I was helping my friend roll out some new services today and a section of his Puppet code caught my eye. This is what I saw:

...
  case $::kernel {
    'linux': {
...

This caught my eye because I had been explicitly capitalizing the 'L' in the $::kernel fact for years. I thought to myself "Is the fact capitalized?"

zeratul:~# facter -p kernel
Linux

What's going on here? Is the case operator insensitive?

case $::kernel {
  'sunos': { notify { $::kernel: }}
}
notice: SunOS
notice: /Stage[main]//Notify[SunOS]/message: defined 'message' as 'SunOS'

Wow. Is the '==' operator in Puppet case-insensitive as well?

if $::kernel == 'sunos' {
  notify { 'lasers': }
}
notice: lasers
notice: /Stage[main]//Notify[lasers]/message: defined 'message' as 'lasers'

Is this a problem with facter or puppet?

if "YES" == "yes" {
  notify { "false is true": }
}
notice: false is true
notice: /Stage[main]//Notify[false is true]/message: defined 'message' as 'false is true'

Seriously? Yep. Turns out the '==' operator is case-insensitive. The '=~' is case-sensitive, but you have to use regular expression syntax in order to use it:

if "YES" =~ /^yes$/ {
  notify { "false is true": }
}
notice: Finished catalog run in 1.30 seconds

Note that we should use '^$' to enclose the string so we don't accidentally get a substring match.

Tested on Puppet 2.7.x and 3.2.x

Saturday, June 8, 2013

Simple Openssl recepie

The Openssl command has always been very opaque to me. Whenever I am doing cert operations I feel like I am a monk in the middle ages, copying scrolls I cannot read. Last night, I learned a simple command to inspect x509 certificate files that is short enough to commit to memory. I encourage everyone to use this command whenever they encounter .pem files and I encourage you to memorize it as well.

The command syntax is:


 openssl openssl x509 -in  -text

A full example:

nibz@host $ openssl x509 -in /etc/ssl/certs/Verisign_Class_1_Public_Primary_Certification_Authority.pem  -text
Certificate:
    Data:
        Version: 1 (0x0)
        Serial Number:
            3f:69:1e:81:9c:f0:9a:4a:f3:73:ff:b9:48:a2:e4:dd
    Signature Algorithm: sha1WithRSAEncryption
        Issuer: C=US, O=VeriSign, Inc., OU=Class 1 Public Primary Certification Authority
        Validity
            Not Before: Jan 29 00:00:00 1996 GMT
            Not After : Aug  2 23:59:59 2028 GMT
        Subject: C=US, O=VeriSign, Inc., OU=Class 1 Public Primary Certification Authority
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (1024 bit)
                Modulus:
                    00:e5:19:bf:6d:a3:56:61:2d:99:48:71:f6:67:de:
                    b9:8d:eb:b7:9e:86:80:0a:91:0e:fa:38:25:af:46:
                    88:82:e5:73:a8:a0:9b:24:5d:0d:1f:cc:65:6e:0c:
                    b0:d0:56:84:18:87:9a:06:9b:10:a1:73:df:b4:58:
                    39:6b:6e:c1:f6:15:d5:a8:a8:3f:aa:12:06:8d:31:
                    ac:7f:b0:34:d7:8f:34:67:88:09:cd:14:11:e2:4e:
                    45:56:69:1f:78:02:80:da:dc:47:91:29:bb:36:c9:
                    63:5c:c5:e0:d7:2d:87:7b:a1:b7:32:b0:7b:30:ba:
                    2a:2f:31:aa:ee:a3:67:da:db
                Exponent: 65537 (0x10001)
    Signature Algorithm: sha1WithRSAEncryption
         58:15:29:39:3c:77:a3:da:5c:25:03:7c:60:fa:ee:09:99:3c:
         27:10:70:c8:0c:09:e6:b3:87:cf:0a:e2:18:96:35:62:cc:bf:
         9b:27:79:89:5f:c9:c4:09:f4:ce:b5:1d:df:2a:bd:e5:db:86:
         9c:68:25:e5:30:7c:b6:89:15:fe:67:d1:ad:e1:50:ac:3c:7c:
         62:4b:8f:ba:84:d7:12:15:1b:1f:ca:5d:0f:c1:52:94:2a:11:
         99:da:7b:cf:0c:36:13:d5:35:dc:10:19:59:ea:94:c1:00:bf:
         75:8f:d9:fa:fd:76:04:db:62:bb:90:6a:03:d9:46:35:d9:f8:
         7c:5b
-----BEGIN CERTIFICATE-----
MIICPDCCAaUCED9pHoGc8JpK83P/uUii5N0wDQYJKoZIhvcNAQEFBQAwXzELMAkG
A1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFz
cyAxIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTk2
MDEyOTAwMDAwMFoXDTI4MDgwMjIzNTk1OVowXzELMAkGA1UEBhMCVVMxFzAVBgNV
BAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFzcyAxIFB1YmxpYyBQcmlt
YXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIGfMA0GCSqGSIb3DQEBAQUAA4GN
ADCBiQKBgQDlGb9to1ZhLZlIcfZn3rmN67eehoAKkQ76OCWvRoiC5XOooJskXQ0f
zGVuDLDQVoQYh5oGmxChc9+0WDlrbsH2FdWoqD+qEgaNMax/sDTXjzRniAnNFBHi
TkVWaR94AoDa3EeRKbs2yWNcxeDXLYd7obcysHswuiovMaruo2fa2wIDAQABMA0G
CSqGSIb3DQEBBQUAA4GBAFgVKTk8d6PaXCUDfGD67gmZPCcQcMgMCeazh88K4hiW
NWLMv5sneYlfycQJ9M61Hd8qveXbhpxoJeUwfLaJFf5n0a3hUKw8fGJLj7qE1xIV
Gx/KXQ/BUpQqEZnae88MNhPVNdwQGVnqlMEAv3WP2fr9dgTbYruQagPZRjXZ+Hxb
-----END CERTIFICATE-----