Easily manage a chain of commits and their respective Pull Requests using git

This blog post was originally posted as a Q&A style answer on Stack Overflow. I recently edited the answer on Stack Overflow, with a slightly improved method, so I felt like sharing it on my personal blog would be beneficial too.

(A Q&A style question on Stack Overflow is a question that you answer at the same time as you ask the question, kind of like writing a blog post but right on Stack Overflow instead, under the guise of a question)

The original question

I have chain of commits, each commit with its own branch, but they are all part of the same feature.

When I get comments on my Pull Requests (GitHub/Bitbucket/Whatever) I want all the following Pull Requests to get my fixes with as little manual work as possible.

In this scenario no one else is using my branches so I am free to force push to them if I need to.

I used to work at a major Android device manufacturer, and I loved the way Gerrit handled this with ease. Gerrit isn't widely used outside of Android though so I want to find a good workflow that covers the 90% case, even in a large-ish team.

I usually solve this in one of two ways:

1. Do fixup commits, push them to their respective branch and then merge upwards however many times is needed until all the branches have all the fixes. This can be a lot of work, depending on how long the chain of commits is.

2. Do a git rebase -i and then do git push origin <sha>:remote-branch as many times as needed. This is also a lot of work, and afterwards there is a mismatch between the local and remote branches (which can be remedied by git reset but that adds even more work). This can also be error prone since it's easy to misspell shas or branch names.

Is there an easier way of managing a chain of commits that are destined for Pull Requests?

The answer

I found a really good method for handling this on [William Chargin's blog][https://wchargin.github.io/posts/managing-dependent-pull-requests/], for what he calls "Dependent Pull Requests" which is a good way of describing it.

Edit 2021-11-08: I have started using git notes for managing the remote-branch instead of adding it to the commit message. This makes the git commit message history much cleaner.

I have modified the original script, and since it uses an MIT license I have followed the guidelines and added my name as well.

The following modifications were made to the script:

• Make it compatible with macOS by using a regexp that works on the macOS version of sed.
• Use the git notes command git notes show to get the remote-branch message instead of git show.
• Use a more generic label for the branch name matcher.
• Default to an empty branch prefix.

Preface

To follow along in this answer you need to be reasonably skilled using git, and you have probably used git rebase -i enough to be comfortable with it.

If this is your first git rodeo, it might be wise to get some experience and come back another time.

Potential drawbacks

The method gets you close to the convenience of Gerrit, as long as you are aware of the potential drawbacks:

• Force pushing isn't always a good strategy.
• All the updated branches will have their tests run again if you have CI (which is good, but also bad if your CI is expensive).
• Watch out for remote branch name collision.
• Some reviewers prefer commit-by-commit changes.
• Probably other stuff too!

Method

Prerequisites

For this method to work you must configure git to use these settings (I like to configure it globally, but you can remove --global if you want it only for a specific repository)

git config --global notes.rewriteRef refs/notes/commits
git config --global notes.rewrite.amend true
git config --global notes.rewrite.rebase true

Branch strategy

All your commits live on a single local branch which does not necessarily need to be pushed except if you want to keep a remote backup while your work is in progress (recommended).

Add as many commits you need to build your logical chain of dependent pull requests. I once used a chain of 11 commits.

$ git log --oneline
aaaaaaa3 (HEAD -> my-backup-branch) Feature: Step 3
aaaaaaa2 Feature: Step 2
aaaaaaa1 Feature: Step 1
aaaaaaaa (origin/master) Probably some merge commit
...

Manage your commits

For each commit in the chain you must add a git notes that tells the script which remote branch that it should be pushed to. The remote branch does not need to exist. Add the git notes when creating the commit or at any time before pushing your changes.

$ git notes add -m "remote-branch: my-feature-step-3"
$ git log
commit aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa3 (HEAD -> my-backup-branch)
Author: Erik Zivkovic <...>
Date:   Fri May 19 19:47:53 2020 +0200

    Feature: Step 3

Notes:
    remote-branch: my-feature-step-3

[... more commits]

Ready for pushing and creating/updating PRs

I would recommend that you do NOT do a git rebase -i fixup AND use the script at the same time. Perform any changes and then test your chain of commits first, then you push. That is the way I recommend doing it.

Make sure that all the commits in the chain have a unique remote-branch: <branch name> line!

To finish it all off call git rebase -i with the -x "git push-to-target" parameter.

$ git rebase -i aaaaaaaa -x "git push-to-target"
[This will open your chosen git editor with a git rebase window]
pick aaaaaaa1 Feature: Step 1
exec git push-to-target
pick aaaaaaa2 Feature: Step 2
exec git push-to-target
pick aaaaaaa3 Feature: Step 3
exec git push-to-target

The magic, of course, is that git rebase stops after each pick command and runs the exec command, which:

• Reads the git notes message.
• Finds the remote-branch line.
• Force Pushes (--force-with-lease) the current HEAD (which points to the pick) to remote-branch.
• If the remote branch does not exist, it is created. Otherwise it is updated.

If everything worked, you should see updated remote branches in git log.

$ git log --oneline
aaaaaaa3 (HEAD -> my-backup-branch, origin/my-feature-step-3) Feature: Step 3
aaaaaaa2 (origin/my-feature-step-2) Feature: Step 2
aaaaaaa1 (origin/my-feature-step-1) Feature: Step 1
aaaaaaaa (origin/master) Probably some merge commit
...

Pull Requests

If you did not have any Pull Requests, now is the time to create them using whatever service you have (Github, Bitbucket, ...), and have the branches point to each other in order.

If the Pull Requests already existed, they should now be updated and tests running again (if you have CI).

More commits, handling review comments, etc.

• If you get review comments, just git rebase -i and do any fixups.
• If you do more feature work, just keep adding commits on top and add unique remote-branch: labels in a git notes for each commit.
• Finally push again with git rebase -i <sha> -x "git push-to-target" (after testing your changes locally, should go without saying).

Cleanup

When everything is merged, you might want to delete your remote backup branch.

Tips

• Tip: Add the script to your path, then you can use it as git push-to-target
• Tip: git config --global alias.ptt push-to-target
• Tip: git rebase -i <SHA> -x "git ptt"

Additional Notes

Note: I have contacted the developer by email to inform him of my changes and ask what he thinks of them, but I have not yet received an answer.

Script

The script should work on Linux/macOS/WSL. I have only tested it on macOS.

The majority of this script was written by William Chargin, I just made it work on macOS and made some minor changes that seemed good to me.

File: git-push-to-target

#!/bin/sh
#
# git-push-to-target: Push this commit to a branch specified in its
# commit description.
#
# Copyright (c) 2017 William Chargin. Released under the MIT license.
# Copyright (c) 2020 Erik Zivkovic.
#
# Originally https://wchargin.github.io/posts/managing-dependent-pull-requests/
#
# Usage: For a commit add a git notes line `remote-branch: branchname` then in a rebase do `exec git push-to-target`
#
# Tip: Add this file to your path, then you can use it as git push-to-target
# Tip: git config --global alias.ptt push-to-target
# Tip: git rebase -i <SHA> -x "git ptt"
#

set -eu

DIRECTIVE='remote-branch'  # any regex metacharacters should be escaped
BRANCH_PREFIX=''           # Add a branch prefix if needed

target_branch() {
    directive="$( \
        git notes show HEAD \
        | sed -n 's/^'"${DIRECTIVE}"': \([A-Za-z0-9_.-]*\)$/\1/p' \
        ; )"
    if [ -z "${directive}" ]; then
        printf >&2 'error: missing "%s" directive\n' "${DIRECTIVE}"
        return 1
    fi
    if [ "$(printf '%s\n' "${directive}" | wc -l)" -gt 1 ]; then
        printf >&2 'error: multiple "%s" directives\n' "${DIRECTIVE}"
        return 1
    fi
    printf '%s%s\n' "${BRANCH_PREFIX}" "${directive}"
}

main() {
    if [ "${1:-}" = "--query" ]; then
        target_branch
        return
    fi
    remote="${1:-origin}"
    branch="$(target_branch)"
    set -x
    git push --force-with-lease "${remote}" HEAD:refs/heads/"${branch}"
}

main "$@"