Thursday, May 30, 2013

Cisco IOS 15 Public-key authentication

With the release of IOS 15 from Cisco, we can now use ssh public keys to authenticate to Cisco devices. It's the technology of the late nineties, today!

First create yourself a rather small key:


ssh keygen -t rsa -b 1024
It will ask you some questions, hopefully you've seen this dialog before. If you need help please feel free to comment or privately message me.

After the key has been created, copy the public string into your copybuffer.

> cat .ssh/nibz_cisco@shadow.cat.pdx.edu.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgQDMuKvC5ZVRuQw6YF5xnMZLopBVbQv5jxgHcR6BWfws3lTaqfSrKUlp3BulxA7P2snphcavf4TS+bNHFd9PKGRVpoQ8ERZtXn1+f008XUN3cxYMZXLB18ae7kfm8Sxk/bO4xWGaQAKc7jkIQY4OLIE0TsKTZGux241N6BNeLGmuLQ== nibz@shadow.cat.pdx.edu

Now add the key to cisco. This assumes the user has already been created properly. It also assumes you are running the following version of IOS:


*    1 52    WS-C3750G-48TS     15.0(2)SE             C3750-IPBASEK9-M
I have tried this on a 15.0(1) and it didn't work. Configuration commands:
fab6017a#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
fab6017a(config)#ip ssh pubkey
fab6017a(config)#ip ssh pubkey-chain 
fab6017a(conf-ssh-pubkey)#username nibz
fab6017a(conf-ssh-pubkey-user)#key-string
fab6017a(conf-ssh-pubkey
data)#$snphcavf4TS+bNHFd9PKGRVpoQ8ERZtXn1+f008XUN3cxYMZXLB18ae7kfm8Sxk/bO4xWGaQAKc7jkIQY4OLIE0TsKTZGux241N6BNeLGmuLQ== nibz@shadow.cat.pdx.edu
fab6017a(conf-ssh-pubkey-data)#exit
fab6017a(conf-ssh-pubkey-user)#end
Some notes on the above: Paste the whole public key once you get the (conf-ssh-pubkey-data) prompt. This includes the 'ssh-rsa' header and comment footer. Use the exit keyword on the (conf-ssh-pubkey-data) line, any other word will be sandwiched onto the end of the key. You can use this feature to split your key into multiple lines and input it that way. After this, cisco will hash your key and the configuration will look like:

  username nibz
   key-hash ssh-rsa 2F33A5AE2F505B42203276F9B2313138 nibz@shadow.cat.pdx.edu
This configuration can be put in other cisco configs elsewhere in your infrastructure. Happy hacking. This was performed on a Cisco3750G running IOS 15.0(2)SE

Friday, May 17, 2013

Ganeti Migration

It is common to want to do a 'Texas two-step' with your ganeti nodes. The idea being that you migrate everything off of a node, reboot it for patches, then migrate everything back, and do this around your cluster until you've patched all your hypervisors. Easier said than done!

The fast way:

# move primaries off node
gnt-node migrate $node

# move secondaries off node
gnt-node evacuate --secondary-only $node
Sometimes nodes fail to migrate. I don't know why. I just use the gnt-instance failover command to move them which requires a reboot. Sometimes the secondary disks don't move either. This is more annoying because hail crashes before making a plan for you. I have written the following script to, stupidly, move all the secondaries off of one node onto other nodes so you can reboot a node in your cluster without fear.
#!/bin/bash

evac_node=$1
if [ -z $evac_node ]; then
echo "Please specify a node to evac"
exit 1
fi

echo "Evacuating secondaries from $evac_node"

HYPERVISORS=`wget --no-check-certificate -O-  https://localhost:5080/2/nodes 2>/dev/null | grep id | awk '{print $NF}' | tr -d \",`

for instance in `gnt-instance list -o name,snodes  | grep $evac_node | cut -d " " -f 1`
do     
        current_primary=`gnt-instance list -o name,pnode | grep $instance | awk '{print $NF}'`
        target_node=`echo $HYPERVISORS | sed 's/ /\n/g' | grep -v $evac_node | grep -v $current_primary | sort -R | head -n 1`
        echo "gnt-instance replace-disks -n $target_node $instance"
        gnt-instance replace-disks -n $target_node $instance
done
Happy hacking!