Debain packaging

Stardate: 90784.6

Git-sync is a script in ruby we use at work for managing git repos. It is covered in an earlier post. I got tired of ensuring it as a file in puppet and decided to make a debian package. Here is the summary of how to make a simple debian package containing just a single file. Note that the answer to this stack overflow question is the source of most of my knowledge, so this will just be annotations and extensions to that.

Debian/Ubuntu packaging (on an ubuntu system) required me to install a single package: devscripts.

At a high level, debian packaging involves creating a 'debian' folder in your source tree and putting several metadatafiles in it. Figuring out the precise contents of these files is the challenge of packaging. I recommend you use the 'apt-get source git' command to get the source of a working package (git in this case) to compare to your own metadatafiles.

Debian/Ubuntu packaging using debuild creates files one level above your current working directory(wtf debian). So the first step is to build a build directory like:

cd ~/devel
mkdir git-sync-build

Procure the source:

nibz@darktemplar:~/devel/git-sync-build$ git clone git@github.com:pdxcat/git-sync
nibz@darktemplar:~/devel/git-sync-build$ ls
git-sync
nibz@darktemplar:~/devel/git-sync-build$ cd git-sync
nibz@darktemplar:~/devel/git-sync-build/git-sync$ mkdir debian

All of the metadata files that debuild, the utility that will actually build the .deb, needs are going to be in the debian directory. 

The first file to create is the debian/changelog file. This file is created with the dch utility. Run it from the git-sync directory. It will open vim and it will look like this. Many fields here need to be changed.
dch --create

PACKAGE (VERSION) UNRELEASED; urgency=low

  * Initial release. (Closes: #XXXXXX)

 -- Spencer Krum   Thu, 07 Mar 2013 01:40:18 -0800


PACKAGE refers to the name of the package. Replace the word PACKAGE with the name you want your package to register itself as. In my git-sync case I will use 'git-sync'. The package name must be lower case. The VERSION must be replaced with a version number. I'm using 1.0.1 for this, since it is the second release of git-sync, but the changes are very minor. There are long articles on the internet about version numbering. It's not my place to comment here. The RELEASE variable needs to be replaced with a debian or ubuntu codename such as 'precise' or 'wheezy'. I have no idea what urgency is, but setting it to low doesn't seem to hurt anything. Maybe this is how you tell apt/dpkg about security updates. The initial release stuff is fine. The name is a bit tricky. Later on we will gpg sign the package. Make sure the name and email in the changelog match exactly the name and email on your gpg key, else the debuild utility won't attempt to have you gpg sign it at all. My changelog looks like this:
git-sync (1.0.0) precise; urgency=low

  * Initial release. 

 -- Spencer Krum   Wed, 06 Mar 2013 16:46:14 -0800


Next create a debian/copyright file:
Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: myScript
Upstream-Contact: Name, 

Files: *
Copyright: 2011, Name, 
License: (GPL-2+ | LGPL-2 | GPL-3 | whatever)
 Full text of licence.
 .
 Unless there is a it can be found in /usr/share/common-licenses


I elected for the apache2 license and to use the two paragraph version of that license. I also gave credit where it was due here. Fill out this file as you see fit.

Next create a debain/compat file: 
nibz@darktemplar:~/devel/git-sync-build/git-sync/debian$ echo 7 > compat

Next create the rules file. This file seems to be the work-doer in debian packaging. It is evaluated by make, which is picky, so make sure that indented line is a real tab(copying from my blog will probably fail). The --with python is... well I have no idea. I traced it to a python.pem (pem is a perlism) deep within /usr. Since I am packaging a ruby script I just removed it.

Example from stackoverflow
#!/usr/bin/make -f

%:
    dh $@ --with python2

git-sync version
#!/usr/bin/make -f

%:
	dh $@

Next make the control file. Make the natural substitutions here. I guessed on section and it just sorta worked.
nibz@darktemplar:~/devel/git-sync-build/git-sync/debian$ cat control 
Source: git-sync
Section: ruby
Priority: optional
Maintainer: Spencer Krum, 
Build-Depends: debhelper (>= 7),
               ruby (>= 1.8.7)
Standards-Version: 3.9.2
X-Ruby-Version: >= 1.8.7


Package: git-sync
Architecture: all
Section: ruby
Depends: ruby, ${misc:Depends}, ${python:Depends}
Description: Git syncing script, pull based
  Git-sync allows git repositories to be kept in sync via git
  hooks or other means. Pull based, able to handle force pushes
  and submodules

Next make the install file. I went with the default in the stackoverflow post. I attempted to make some simple modifications to it(moving the file to /usr/local/bin) and that made it fail so this file is evidently pretty finicky.
nibz@darktemplar:~/devel/git-sync-build/git-sync$ cat debian/install 
git-sync usr/bin


Now you can build the debian package.
nibz@darktemplar:~/devel/git-sync-build/git-sync$ debuild --no-tgz-check

If all went well, it should ask you to decrypt your gpg key twice and build a package in the directory one level up.
nibz@darktemplar:~/devel/git-sync-build/git-sync$ ls ..
git-sync                git-sync_1.0.0_amd64.build    git-sync_1.0.0.dsc
git-sync_1.0.0_all.deb  git-sync_1.0.0_amd64.changes  git-sync_1.0.0.tar.gz

You now have a shiny .deb file that can be installed with dpkg -i git-sync_1.0.0_all.deb

It is easy to put this in a launchpad PPA if you have a launchpad account. From your launchpad homepage (a shortcut is https://launchpad.net/~ if you are signed in). Press the "Create new PPA". Fill out the form. 

Next build a source package. Launchpad PPAs take source packages and build binary packages on launchpad servers. Build it with:
nibz@darktemplar:~/devel/git-sync-build/git-sync$ debuild -S

It should go through the gpg motions again and build a source file.
Then you should be able to run something like(with your launchpad username and name of PPA):
dput ppa:krum-spencer/git-sync-ppa git-sync_1.0.0_source.changes


Happy Packaging!

Git-sync

Stardate: 90784.4428183

Where I work(read as: play) we use a lot of git. As an operator we often have a service running with its configs in git. A common pattern we use is to have a post-receive hook on the git repository set up to update the git checkout on a remote server. We accomplish this through a post-receive hook that sshes into the remote server and calls a script called git-sync with some options. The git-sync script github project forked from the puppet-sync script project that we use specifically for puppet dynamic git environments. Hunner <3 More dynamic git environments with puppet. Finch <3.

A hook for a project goes in the hooks/post-receive file of the git server's bare repo. Lets look at one now:

Example git post-receive hook

#!/bin/sh
## File: akwardly incorrect

PWD=`pwd`
REPONAME=`basename $PWD | sed 's/.git$//'`
REPO="git@gitserver.cat.pdx.edu:$REPONAME.git"
DEPLOY="/etc/nagios3/conf.d/flat"
SSH_ARGS="-i /shadow/home/git/.ssh/git@gitserver.cat.pdx.edu"
GITSYNC="nagios@nagios.cat.pdx.edu"
SYNC_COMMAND="/usr/local/bin/git-sync"

while read oldrev newrev refname
do
  BRANCH=`echo $refname | sed -n 's/^refs\/heads\///p'`
  if [ $BRANCH != "master" ]; then 
    echo "Branch is not master, therefore not pushing to nagios"
    exit 0
  fi
  [ "$newrev" -eq 0 ] 2> /dev/null && DELETE='--delete' || DELETE=''
  ssh $SSH_ARGS "$GITSYNC" "$SYNC_COMMAND" \
    --branch "$BRANCH" \
    --repository "$REPO" \
    --deploy "$DEPLOY" \
    $DELETE

done

ssh nagios@gitserver.cat.pdx.edu '/etc/init.d/nagios3 reload'

The hook will exit before doing anything if the branch is not

master

and if it is, will run the git-sync script remotely on the nagios host, then go back in to bounce the nagios service.

The git-sync script essentially performs a

git fectch; git checkout HEAD

It doesn't worry itself with merging, and it is submodule aware.

A file, .git-sync-stamp, must be created by the administrator of the system. This is how git-sync knows it is in charge of managing the repository. It is definitely not recommended that you add this file to git. However, that should more or less work if you never want to think about it. I also wrote this puppet defined type to manage the stamp, initial vcsrepo, and public_key_file for you.

A puppet defined type to initalize gitsync managed folders

define gitsync::gitsync(
  $deploy,
  $repo,
  $user,
  $source,
  $public_key,
  $public_key_type
){

  ssh_authorized_key { "${user}-${name}-gitsync":
    user   => $user,
    ensure => $present,
    type   => $public_key_type,
  }

  vcsrepo { $deploy:
    ensure    => present,
    provider  => git,
    user      => $user,
    source    => $source
    require   => Ssh_authorized_key["${user}-${name}-gitsync"],
  }

  file { "${deploy}/.git-sync-stamp":
    ensure  => present,
    owner   => $user,
    mode    => 0644,
    require => Vcsrepo[$deploy],
  }

The last thing to note is that I didn't write git-sync. I've modified it but it was mostly written by Reid Vandewielle and others. Marut <3 Enjoy