Sunday, May 10, 2015

Managing patchset stacks with git-review

In OpenStack, we use gerrit and git-review to propose changes to the repository. The workflow for that is pretty complicated, and downright confusing if you are coming from the github workflow.

One of the places where it gets hard/complicated/annoying to use our workflow is if you have multiple dependent changes. I have a technique I use, that I will present below.

The situation: You have two patches in a stack. There is a bug in the first patchset that you need to fix.

The simple play: Checkout the patchset with 'git review -d <review number>', ammend and git-review. The problem with this is that now you need to go rebase all dependent patchsets against this new review. Sometimes you can get away with using the 'rebase' button but sometimes you cannot.

What I do: I use 'git rebase -i HEAD~2' and use 'edit' to change the commit that needs to be changed, rebase goes ahead and auto-rebases everything above it (pausing if needed for me to fix things), then I can 'git review' and it will update all the patchsets that need to be changed.

This approach works for any sized stack, but using it on a two-stack is the simplest example that works.



The git log before we start:

commit e394aba4321f6d30131793e69a4f14b011ce5560
Author: Spencer Krum <nibz@spencerkrum.com>
Date:   Wed May 6 15:43:27 2015 -0700

    Move afs servers to using o_p::template
    
    This is part of a multi-stage process to merge o_p::server and
    o_p::template.
    
    Change-Id: I3bd3242a26fe701741a7784ae4e10e4183be17cf

commit 3e592608b4d369576b88793377151b7bfaacd872
Author: Spencer Krum <nibz@spencerkrum.com>
Date:   Wed May 6 15:38:23 2015 -0700

    Add the ability for template to manage exim
    
    Managing exim is the one thing sever can do that template cannot.
    This is part of a multi stage process for merging server and template.
    
    Change-Id: I354da6b5d489669b6a2fb4ae4a4a64c2d363b758


Note that we have two commits and they depend on each other. The bug is in 3e592608b4d369576b88793377151b7bfaacd872. We start the interactive rebase below, first with a vim session then with output on the command line. The vim session:

$ git rebase -i HEAD~2


  1 e 3e59260 Add the ability for template to manage exim
  2 pick e394aba Move afs servers to using o_p::template
  3 
  4 # Rebase af02d02..e394aba onto af02d02
  5 #
  6 # Commands:
  7 #  p, pick = use commit
  8 #  r, reword = use commit, but edit the commit message
  9 #  e, edit = use commit, but stop for amending
 10 #  s, squash = use commit, but meld into previous commit
 11 #  f, fixup = like "squash", but discard this commit's log message
 12 #  x, exec = run command (the rest of the line) using shell
 13 #
 14 # These lines can be re-ordered; they are executed from top to bottom.
 15 #
 16 # If you remove a line here THAT COMMIT WILL BE LOST.
 17 #
 18 # However, if you remove everything, the rebase will be aborted.
 19 #
 20 # Note that empty commits are commented out


Note that the 'top' commit in the rebase view is the 'bottom' commit in the git log view, because git is stupid. We change the 'pick' to 'e' for 'edit' meaning stop at that point for ammending. And the shell output:

Stopped at 3e592608b4d369576b88793377151b7bfaacd872... Add the ability for template to manage exim

You can amend the commit now, with

  git commit --amend

Once you are satisfied with your changes, run

  git rebase --continue

 (master|REBASE-i 1/2)$: git st
rebase in progress; onto af02d02
You are currently editing a commit while rebasing branch 'master' on 'af02d02'.

  (use "git commit --amend" to amend the current commit)
  (use "git rebase --continue" once you are satisfied with your changes)



nothing to commit, working directory clean


Then we make changes to modules/openstack_project/manifests/template.pp (not shown) and continue the rebase:


 (master *|REBASE-i 1/2)$: git st
rebase in progress; onto af02d02

You are currently editing a commit while rebasing branch 'master' on 'af02d02'.

  (use "git commit --amend" to amend the current commit)
  (use "git rebase --continue" once you are satisfied with your changes)
 
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)
  modified:   modules/openstack_project/manifests/template.pp
no changes added to commit (use "git add" and/or "git commit -a")
 
 (master *|REBASE-i 1/2)$: git add modules/openstack_project/manifests/template.pp
 (master +|REBASE-i 1/2)$: git rebase --continue
[detached HEAD 6ca26e9] Add the ability for template to manage exim
 1 file changed, 7 insertions(+)
Successfully rebased and updated refs/heads/master.


Then we publish our changes with git-review:
(master u+2)$: git review

You are about to submit multiple commits. This is expected if you are
submitting a commit that is dependent on one or more in-review
commits. Otherwise you should consider squashing your changes into one
commit before submitting.

The outstanding commits are:

2bc78a8 (HEAD, master) Move afs servers to using o_p::template
6ca26e9 Add the ability for template to manage exim

Do you really want to submit the above commits?
Type 'yes' to confirm, other to cancel: yes

remote: Resolving deltas: 100% (4/4)
remote: Processing changes: updated: 2, refs: 2, done    
To ssh://krum-spencer@review.openstack.org:29418/openstack-infra/system-config.git
 * [new branch]      HEAD -> refs/publish/master

With that we have changed a commit deep in the stack, rebased any commits above it, and published our changes to the gerrit server.

Overview of Puppet in OpenStack Infra

Last week I gave this presentation at the PDX Puppet Users group. It is an overview of how we use Puppet in the OpenStack Infra project. There is no video or audio recording.

Presentation