Tuesday, January 29, 2013

Puppet Fact Fix after Ruby 1.9 upgrade

Stardate: 90683.2
We recently upgraded our entire infrastructure to Puppet 3. As if that wasn't ambitious enough (though I suppose Puppet 3.1 has an rc), we are slowly bringing ruby to 1.9 from 1.8.7. Surprisingly, puppet actually works under ruby 1.9. This blows my mind since I remember having to do some seriously ugly hacks to get puppet working under ruby 1.9. Unfortunately, and embarrassingly, one of my custom facts was not forward compatible.
...
Info: Loading facts in /var/lib/puppet/lib/facter/nvidia_graphics_device.rb
Could not retrieve dns_ip_6: undefined method `each' for "yermom.cat.pdx.edu has address 131.252.222.10\n":String
Could not retrieve dns_ip_6: undefined method `each' for "yermom.cat.pdx.edu has address 131.252.222.10\n":String
...
The unfortunate cause of this (other than that I didn't write 1.9 compatible code) is that string.each has been removed from ruby as of ruby 1.9. :(
For those of you unfamiliar with puppet hacking, most testing should be done through git dynamic environments. Unfortunately, there is a bug in puppet that prevents facts from being able to be tested on any branch but production. You can add the facts to git and push them to the environments directory on the puppet master, but unless they are in branch production you won't see them get filebucketted or run.
So what to do? The answer is to develop on the box itself, usually as root, though sudo is an idea. (Hopefully I can get a friend of mine to guest post on why sudo is the correct way to attain root privileges for administration, and I can counterpost on why su - is the correct way.) On ubuntu (as of 12.04, anyways) the puppet configuration dir is /etc/puppet, but the puppet var dir is /var/lib/puppet. Facts live in /var/lib/puppet/lib/facter. The best way to get information on a current puppet's installation and configuration is through:
puppet config print all | grep vardir
vardir = /var/lib/puppet
Change directory to the puppet vardir and modify the facts in place. Then run the facter utility with the '-p' argument. The '-p' argument tells facter to run all its normal facts as well as facts loaded in from puppet.

root@yermom:/var/lib/puppet/lib/facter# facter -p | grep dns
dns_ip_4 => ["131.252.222.10"]
dns_ip_6 => []

Fantastic. All is well again.

1 comment:

  1. I believe the problem is subtly different; it's not that puppet only supports syncing lib code in the production environment, more that it only looks in a single directory on the puppet master by default. This directory is shared with the agent which runs on the master, so will contain whatever files the agent put there the last time you did an agent run on the master. This is actually pretty serious, as if you do a development run on the master, you'll end up pushing development lib code out to all clients, even if they're configured to be in the production environment!

    We solved this problem by creating separate libdirs per environment, and telling the puppet master where it should look. That way, when a client does a run in the development environment, the puppet master looks there for development files.

    In puppet.conf
    [master]
    environment = production
    manifest = <%= sitedir %>/$environment/manifests/site.pp
    modulepath = <%= sitedir %>/$environment/modules/
    libdir = $vardir/environments/$environment
    pluginsync = true

    ReplyDelete