Today while poking around in the puppet source code, I came across a utility in the ext/ directory called cert_inspector. This seems to be a little utility that opens up certificates and interrogates them for useful data. This is better than what I usually do, which is incanting openssl directly. It also is capable of chewing up an entire /var/lib/puppet/ssl directory and dumping information on every cert and key it finds. See the output below:
(master u=)$: ./ext/cert_inspector ~/.puppet/ssl/certs/ca.pem
/home/nibz/.puppet/ssl/certs/ca.pem:
Certificate assigning name /CN=Puppet CA: zabava.cat.pdx.edu to key</CN=Puppet CA: zabava.cat.pdx.edu>
serial number 1
issued by /CN=Puppet CA: zabava.cat.pdx.edu
signed by key</CN=Puppet CA: zabava.cat.pdx.edu>
(master u=)$: ./ext/cert_inspector ~/.puppet/ssl/
WARNING: file "/home/nibz/.puppet/ssl/public_keys/hunner_what_r_u_doin.pem" could not be interpreted
WARNING: file "/home/nibz/.puppet/ssl/public_keys/hunner_stahp.pem" could not be interpreted
WARNING: file "/home/nibz/.puppet/ssl/public_keys/maxwell.hsd1.or.comcast.net.pem" could not be interpreted
WARNING: file "/home/nibz/.puppet/ssl/public_keys/hunner.pem" could not be interpreted
/home/nibz/.puppet/ssl/certs/ca.pem:
Certificate assigning name /CN=Puppet CA: zabava.cat.pdx.edu to key</CN=Puppet CA: zabava.cat.pdx.edu>
serial number 1
issued by /CN=Puppet CA: zabava.cat.pdx.edu
signed by key</CN=Puppet CA: zabava.cat.pdx.edu>
/home/nibz/.puppet/ssl/certificate_requests/hunner.pem:
Certificate request for /CN=hunner having key key</CN=hunner>
signed by key</CN=hunner>
/home/nibz/.puppet/ssl/certificate_requests/hunner_stahp.pem:
Certificate request for /CN=hunner_stahp having key key</CN=hunner_stahp>
signed by key</CN=hunner_stahp>
/home/nibz/.puppet/ssl/certificate_requests/hunner_what_r_u_doin.pem:
Certificate request for /CN=hunner_what_r_u_doin having key key</CN=hunner_what_r_u_doin>
signed by key</CN=hunner_what_r_u_doin>
/home/nibz/.puppet/ssl/private_keys/hunner.pem:
Private key for key</CN=hunner>
/home/nibz/.puppet/ssl/private_keys/hunner_stahp.pem:
Private key for key</CN=hunner_stahp>
/home/nibz/.puppet/ssl/private_keys/hunner_what_r_u_doin.pem:
Private key for key</CN=hunner_what_r_u_doin>
/home/nibz/.puppet/ssl/private_keys/maxwell.hsd1.or.comcast.net.pem:
Private key for key</home/nibz/.puppet/ssl/private_keys/maxwell.hsd1.or.comcast.net.pem>
Monday, December 15, 2014
Tuesday, December 9, 2014
Testing Puppet node definitions
Sometimes Puppet node definitions get a little hairy. Here is a quick trick I use to validate them manually. This is inspired by this review.
Given a regex node definition create a test file called node.pp:
node /^git(-frontend\d+)?\.openstack\.org$/ {
notify { 'match': }
}
Then, using the --certname="testnode" syntax to puppet apply, do some quick spot testing to see what happens.
Given a regex node definition create a test file called node.pp:
node /^git(-frontend\d+)?\.openstack\.org$/ {
notify { 'match': }
}
Then, using the --certname="testnode" syntax to puppet apply, do some quick spot testing to see what happens.
$: puppet apply nodedef.pp
Error: Could not find default node or by name with 'maxwell.pdx.edu, maxwell.pdx, maxwell' on node maxwell.pdx.edu
Error: Could not find default node or by name with 'maxwell.pdx.edu, maxwell.pdx, maxwell' on node maxwell.pdx.edu
$: puppet apply --certname='git.openstack.org' nodedef.pp
Notice: Compiled catalog for git.openstack.org in environment production in 0.02 seconds
Notice: match
Notice: /Stage[main]/Main/Node[git-frontendd.openstack.org]/Notify[match]/message: defined 'message' as 'match'
Notice: Finished catalog run in 0.03 seconds
$: puppet apply --certname='git48.openstack.org' nodedef.pp
Error: Could not find default node or by name with 'git48.openstack.org, git48.openstack, git48, maxwell.pdx.edu, maxwell.pdx, maxwell' on node git48.openstack.org
Error: Could not find default node or by name with 'git48.openstack.org, git48.openstack, git48, maxwell.pdx.edu, maxwell.pdx, maxwell' on node git48.openstack.org
$: puppet apply --certname='git-frontend01.openstack.org' nodedef.pp
Notice: Compiled catalog for git-frontend01.openstack.org in environment production in 0.02 seconds
Notice: match
Notice: /Stage[main]/Main/Node[git-frontendd.openstack.org]/Notify[match]/message: defined 'message' as 'match'
Notice: Finished catalog run in 0.03 seconds
This gives us the confidence to push this node definition to production without worrying about affecting existing git servers.
Monday, December 8, 2014
#puppethack
#puppethack is the new version of the Puppet triage-a-thon. It is a decentralized hackathon for open source Puppet projects. This year I participated mostly by contributing to the puppetlabs-rabbitmq module. I worked closely with Colleen Murphy of Puppet Labs on this.
When we started there were 31 outstanding pull requests. Now there are only 21. And five of those have been opened during or after the hackathon.
I am most proud of my beaker testing PR which added beaker (acceptance and integration) testing to the rabbitmq_user, rabbitmq_vhost, and rabbitmq_policy types.
Overall #puppethack was a success and I am glad I participated. I want to thank my employer, HP, for allowing me to participate in the open source ecosystem. I am looking forward to doing it next year!
When we started there were 31 outstanding pull requests. Now there are only 21. And five of those have been opened during or after the hackathon.
I am most proud of my beaker testing PR which added beaker (acceptance and integration) testing to the rabbitmq_user, rabbitmq_vhost, and rabbitmq_policy types.
Overall #puppethack was a success and I am glad I participated. I want to thank my employer, HP, for allowing me to participate in the open source ecosystem. I am looking forward to doing it next year!
Sunday, December 7, 2014
Puppet Functions in stdlib
You should read up on the Puppet Functions in puppetlabs/stdlib. Seriously.
If you consider yourself a serious Puppet user, i.e. you use it more than twice a month, you owe it to yourself to read through them. The README has a brief description of the functions that are available. Every time I read through it, I find more useful functions have been added. And with stronger protections for function composability, there is no reason not to use functions all the time, every time.
Even if all you do is learn about the existence of the validation functions, you will be able to in two lines of code make your code more robust and easier on users.
For extra credit check out the puppet-community extlib module which has more functions not deemed cool enough for puppet core.
To eat my own words, I'll now post some functions whose existence I did not know about:
If you consider yourself a serious Puppet user, i.e. you use it more than twice a month, you owe it to yourself to read through them. The README has a brief description of the functions that are available. Every time I read through it, I find more useful functions have been added. And with stronger protections for function composability, there is no reason not to use functions all the time, every time.
Even if all you do is learn about the existence of the validation functions, you will be able to in two lines of code make your code more robust and easier on users.
For extra credit check out the puppet-community extlib module which has more functions not deemed cool enough for puppet core.
To eat my own words, I'll now post some functions whose existence I did not know about:
- chomp
- chop
- defined_with_params
- diference
- delete_undef_values
- empty (OMG USEFUL)
- get_param (this changes eveeeeerything)
- private
- reject (duuuuuudeee)
- squeeze
Saturday, November 29, 2014
Bashrc: Gerrit
Gerrit is the code review and git hosting tool used by OpenStack. It is common courtesy to mark a change 'work in progress' when you have submitted it but it is not ready for others to review. Others will see the work in progress bit is set and not waste time reviewing patches that are not ready yet.
The 'git review' tool is good for submitting changes, but then I have to go to the web ui to mark a change as wip. I can use gertty for this, but again that means going and searching it out.
I have added the following function to my .bashrc:
gerrit () {
if [ $1 = "wip" ]; then
commit=`git show | grep -m1 commit | cut -d " " -f 2 2>/dev/null`
if [ -z $commit ]; then
echo "Not in git directory?"
return 1
fi
gerrit review $commit --workflow -1
return $?
fi
username=`git config gitreview.username`
ssh -o VisualHostKey=no -p 29418 $username@review.openstack.org gerrit $*
}
This function enables some pretty cool features. It takes as arguments any arguments that the gerrit ssh command line interface takes. Meaning you can do things like these:
$: gerrit ls-projects | grep puppet
openstack-infra/puppet-apparmor
openstack-infra/puppet-dashboard
openstack-infra/puppet-github
openstack-infra/puppet-httpd
openstack-infra/puppet-jenkins
openstack-infra/puppet-kibana
openstack-infra/puppet-pip
openstack-infra/puppet-storyboard
openstack-infra/puppet-vcsrepo
openstack-infra/puppet-vinz
openstack-infra/puppet-yum
openstack-infra/puppet-zuul
openstack/tripleo-puppet-elements
stackforge/puppet-ceilometer
stackforge/puppet-ceph
stackforge/puppet-cinder
stackforge/puppet-designate
stackforge/puppet-glance
stackforge/puppet-heat
stackforge/puppet-horizon
stackforge/puppet-ironic
stackforge/puppet-keystone
stackforge/puppet-manila
stackforge/puppet-monasca
stackforge/puppet-n1k-vsm
stackforge/puppet-neutron
stackforge/puppet-nova
stackforge/puppet-openstack
stackforge/puppet-openstack-cloud
stackforge/puppet-openstack-specs
stackforge/puppet-openstack_dev_env
stackforge/puppet-openstack_extras
stackforge/puppet-openstacklib
stackforge/puppet-sahara
stackforge/puppet-swift
stackforge/puppet-tempest
stackforge/puppet-trove
stackforge/puppet-tuskar
stackforge/puppet-vswitch
stackforge/puppet_openstack_builder
$: gerrit -h
gerrit [COMMAND] [ARG ...] [--] [--help (-h)]
-- : end of options
--help (-h) : display this help text
Available commands of gerrit are:
ban-commit Ban a commit from a project's repository
create-account Create a new batch/role account
create-group Create a new account group
create-project Create a new project and associated Git repository
flush-caches Flush some/all server caches from memory
gc Run Git garbage collection
gsql Administrative interface to active database
ls-groups List groups visible to the caller
ls-members List the members of a given group
ls-projects List projects visible to the caller
ls-user-refs List refs visible to a specific user
plugin
query Query the change database
receive-pack Standard Git server side command for client side git push
rename-group Rename an account group
review Verify, approve and/or submit one or more patch sets
set-account Change an account's settings
set-members Modify members of specific group or number of groups
set-project Change a project's settings
set-project-parent Change the project permissions are inherited from
set-reviewers Add or remove reviewers on a change
show-caches Display current cache statistics
show-connections Display active client SSH connections
show-queue Display the background work queues
stream-events Monitor events occurring in real time
test-submit
version Display gerrit version
See 'gerrit COMMAND --help' for more information.
We also inspect the first command to see if it is 'wip.' This allows us to create new commands to the gerrit cli without changing any code or having access to the gerrit server. What I've added is the 'wip' command which inspects the local git repository for the latest change, and marks it as wip with gerrit. This changes my workflow to look like this:
$: git review
$: gerrit wip
This is much shorter, more unixy, and doesn't require me to hop out of the terminal. Future improvements would be to identify if you are in a stack of changes and wip all of them.
The 'git review' tool is good for submitting changes, but then I have to go to the web ui to mark a change as wip. I can use gertty for this, but again that means going and searching it out.
I have added the following function to my .bashrc:
gerrit () {
if [ $1 = "wip" ]; then
commit=`git show | grep -m1 commit | cut -d " " -f 2 2>/dev/null`
if [ -z $commit ]; then
echo "Not in git directory?"
return 1
fi
gerrit review $commit --workflow -1
return $?
fi
username=`git config gitreview.username`
ssh -o VisualHostKey=no -p 29418 $username@review.openstack.org gerrit $*
}
This function enables some pretty cool features. It takes as arguments any arguments that the gerrit ssh command line interface takes. Meaning you can do things like these:
$: gerrit ls-projects | grep puppet
openstack-infra/puppet-apparmor
openstack-infra/puppet-dashboard
openstack-infra/puppet-github
openstack-infra/puppet-httpd
openstack-infra/puppet-jenkins
openstack-infra/puppet-kibana
openstack-infra/puppet-pip
openstack-infra/puppet-storyboard
openstack-infra/puppet-vcsrepo
openstack-infra/puppet-vinz
openstack-infra/puppet-yum
openstack-infra/puppet-zuul
openstack/tripleo-puppet-elements
stackforge/puppet-ceilometer
stackforge/puppet-ceph
stackforge/puppet-cinder
stackforge/puppet-designate
stackforge/puppet-glance
stackforge/puppet-heat
stackforge/puppet-horizon
stackforge/puppet-ironic
stackforge/puppet-keystone
stackforge/puppet-manila
stackforge/puppet-monasca
stackforge/puppet-n1k-vsm
stackforge/puppet-neutron
stackforge/puppet-nova
stackforge/puppet-openstack
stackforge/puppet-openstack-cloud
stackforge/puppet-openstack-specs
stackforge/puppet-openstack_dev_env
stackforge/puppet-openstack_extras
stackforge/puppet-openstacklib
stackforge/puppet-sahara
stackforge/puppet-swift
stackforge/puppet-tempest
stackforge/puppet-trove
stackforge/puppet-tuskar
stackforge/puppet-vswitch
stackforge/puppet_openstack_builder
$: gerrit -h
gerrit [COMMAND] [ARG ...] [--] [--help (-h)]
-- : end of options
--help (-h) : display this help text
Available commands of gerrit are:
ban-commit Ban a commit from a project's repository
create-account Create a new batch/role account
create-group Create a new account group
create-project Create a new project and associated Git repository
flush-caches Flush some/all server caches from memory
gc Run Git garbage collection
gsql Administrative interface to active database
ls-groups List groups visible to the caller
ls-members List the members of a given group
ls-projects List projects visible to the caller
ls-user-refs List refs visible to a specific user
plugin
query Query the change database
receive-pack Standard Git server side command for client side git push
rename-group Rename an account group
review Verify, approve and/or submit one or more patch sets
set-account Change an account's settings
set-members Modify members of specific group or number of groups
set-project Change a project's settings
set-project-parent Change the project permissions are inherited from
set-reviewers Add or remove reviewers on a change
show-caches Display current cache statistics
show-connections Display active client SSH connections
show-queue Display the background work queues
stream-events Monitor events occurring in real time
test-submit
version Display gerrit version
See 'gerrit COMMAND --help' for more information.
We also inspect the first command to see if it is 'wip.' This allows us to create new commands to the gerrit cli without changing any code or having access to the gerrit server. What I've added is the 'wip' command which inspects the local git repository for the latest change, and marks it as wip with gerrit. This changes my workflow to look like this:
$: git review
$: gerrit wip
This is much shorter, more unixy, and doesn't require me to hop out of the terminal. Future improvements would be to identify if you are in a stack of changes and wip all of them.
Saturday, November 22, 2014
Leaving Blogger
It's time to join the future and host my own blog. I'm also going to do the regular stuff of evaluating technologies and picking one, developing tools that other people have developed, etc.
I've debated doing this many times. I've always felt that I didn't want to be that person who only blogs about blogging. I feel after a couple years of pretty consistent blogging, that I won't totally ignore my blog after putting a lot of effort into it. Plus this is an excuse to build a website, something I am embarrassingly weak in.
I'm pretty sure the next location of my blog will be http://spencerkrum.com but I'm not entirely sure.
Right now the plan is to move to pelican because python and restructured text are both technologies that have crossover with OpenStack, and I like that. What I may end up writing is tooling to pull my old posts out of Blogger.
I have no idea if blogger will let me put a redirect in for my subdomain on their domain. I have to think that literally no one at google works on this now right? After reader went away I was sure this would get the axe and yet it remains...
I've debated doing this many times. I've always felt that I didn't want to be that person who only blogs about blogging. I feel after a couple years of pretty consistent blogging, that I won't totally ignore my blog after putting a lot of effort into it. Plus this is an excuse to build a website, something I am embarrassingly weak in.
I'm pretty sure the next location of my blog will be http://spencerkrum.com but I'm not entirely sure.
Right now the plan is to move to pelican because python and restructured text are both technologies that have crossover with OpenStack, and I like that. What I may end up writing is tooling to pull my old posts out of Blogger.
I have no idea if blogger will let me put a redirect in for my subdomain on their domain. I have to think that literally no one at google works on this now right? After reader went away I was sure this would get the axe and yet it remains...
Wednesday, November 12, 2014
Guest Post on Puppet-a-day
Today I have been honored to post a guest post on the puppet-a-day community blog. You can find my post here.
Big thanks to @daenney for making the puppet-a-day thing go.
Big thanks to @daenney for making the puppet-a-day thing go.
Subscribe to:
Posts (Atom)