Subversion Merge Tracking – The Basics

Subversion 1.5 will support Merge Tracking. This blog post provides a high level overview of what Merge Tracking is, why it is important and what it means to your organization. I’ll begin with some very basic explanations; if you are familiar with branching and merging, skip paragraph 1.

1. What’s branching and merging?

Development teams often work on several parallel lines of development, called ‘branches.’ A branch starts with making copies of all the files of your development project (or a directory) and maintaining these copies separately as you go forward. Files start as identical copies but over time they will differ from each other as developers make changes on the different branches.

Why branch? You might create a branch in Subversion for the maintenance of a released product while simultaneously working on the next release. Why? The next release will have new features but on the maintenance branch you only accept bug fixes. Another use case is the development of a complex new feature that you know will destabilize your development builds. By doing that development on a separate branch, you isolate the other developers from the disruptive work that would break nightly builds for everybody. Subversion’s own Merge Tracking feature was developed on a branch for about a year.

Mt1
Graph 1. Mainline development (trunk) with two branches

Subversion has always supported branching and unlike many other systems it does not actually copy the files, which would rapidly grow the size of the repository. Instead, Subversion creates a quick reference to the existing directory and from there on just tracks the changes to the files on the branch and trunk. The result is that creating a branch is very quick and the repository hardly grows in size.

At times, code needs to be merged. For instance, a developer fixes a bug in the maintenance branch. You will want to merge that fix into the main development line, otherwise your next release will still have the bug. Another example is the development of a new feature on a branch. As said, Subversion’s own Merge Tracking feature was developed on a branch. In February ’07 the feature was stable enough to be merged into trunk.

Mt2
Graph 2. Merging code

Let’s look at a real simple merge example. This is our code:

Code on trunk (the main codeline)
main(){    printf("hello, wordn");}

Now we make a branch. From a user point of view we have two instances of the code:

Code on trunk (the main codeline) Code on branch
main(){ printf("hello, wordn");}
main(){ printf("hello, wordn");}
“Word” should actually be “World”; we have a bug. A developer fixes it on the branch. The files are different now.
main(){ printf("hello, wordn");}
main(){ printf("hello, worldn");}
At some point in time the bugfix is merged into trunk
>> svn merge: Subversion merges the change from branch to trunk.
main(){ printf("hello, worldn");}
main(){ printf("hello, worldn");}

2. Subversion and Merging

Just like branching, Subversion has always supported merging and can do a lot of it automatically if you ask it to. But Subversion does not “remember” when you merged what code from where into what branch, and when. While branching and merging works well in Subversion, adding Merge Tracking solves some limitations:

  • Repeated merges:  Say you have a feature branch that you keep in synch with trunk. Without Merge Tracking you have to carefully (and manually) record which revisions you have already merged. This can be tedious and when you make mistakes you might forget to merge certain changes or create what is called “spurious conflicts”, which basically means that files aren’t properly merged and the resulting revision has errors.
  • Auditability.  When you merge a feature branch back to trunk, the history of trunk only records that a merge happened, but not what was merged. It is therefore hard to find out afterwards what exactly was merged when into trunk.

3. What this means to you?

Development teams have branched and merged successfully with Subversion for years but Merge Tracking offers several benefits:

  • Merge Tracking adds auditibility/traceability (what code was merged when and where?). This is required by some organizations for governance purposes.
  • Merge Tracking reduces errors and admin overhead. Teams that merge often will see increased productivity when Subversion has Merge Tracking.
  • It is important to merge often. Two files that start the same will look increasingly different over time. The more different they are, the more difficult it is to merge them. If you merge often, incremental differences are smaller and merging will be easier. Merge Tracking will make it easier to merge often.
  • Many companies that use Subversion choose to have limited branching strategies. They therefore do not benefit from a good branching strategy that leverages parallel development. For example: they might choose to develop a risky new feature on the main development line rather than on a separate branch. The result can be that team members have to deal with builds that are more often broken than needed
  • The ultimate benefit: some companies have not moved to Subversion yet and have stayed with traditional and more costly tools; they are waiting for Merge Tracking.

4. Merge Tracking in Subversion 1.5

Subversion 1.5 records what happens during a merge: it ‘tracks merges’ so the next time you merge Subversion remembers what happened during previous merges. The feature supports the following use cases:

  • Repeated merge: merge a branch into another branch this week, do it again next week. Subversion will remember what was already merged and only merge the new changes.
  • Automated merge with conflict resolution. Subversion can do most of your merges automatically, but merging inevitably involves conflicts that Subversion’s internal merge algorithm cannot resolve. If so, Subversion will ask the user to resolve the conflict manually. Mark Phippard blogged about this.
  • Cherry picking: merge only one or a few changes on a branch, rather than all changes.
  • Record manual merge: sometimes users will merge something manually (copying code from one file to another by using an editor). Subversion 1.5 has functionality to explicitly add information about manual merges to the history database so its Merge Tracking information stays complete.
  • Rollback Merge: undo a merge. Merges are often not perfect and you may find out afterwards that something is broken. Subversion lets you undo the merge.
  • Merge auditing: merge data is automatically added to the commit log (Mark blogged about this as as well).

5. GUI Clients and Merge Tracking

Merge Tracking will become really powerful when the GUI clients for Subversion support it. GUI clients will leverage the Merge Tracking feature to make merge easy to use and approachable to all users.  CollabNet is working on a Merge Tracking GUI client for Eclipse, expect a preview of that within the next few days, here on openCollabNet.

Subversion 1.5 will have call-backs for clients. E.g.: sometimes Subversion cannot merge two files automatically and it needs the developer to resolve a “merge conflict”. Subversion will call the client and the client can decide what to do. For example, Subclipse will take the conflicting files and send them to Eclipse’s graphical diff tool so the user can resolve the conflict (it actually does a 3-way diff, but that is beyond the scope of this post).

6. Merge Tracking Early Adopter Program

To accelerate the adoption of the new Merge Tracking feature in Subversion 1.5, CollabNet has started a Merge Tracking Early Adopter Program on openCollabNet. In this program you can find:

  • Links to Subversion’s Merge Tracking design documents.
  • Subversion 1.5 pre-release binaries that include the new Merge Tracking feature.
  • A sample Subversion repository that includes merge tracking history.
  • A forum to discuss the feature with CollabNet’s Subversion committers and with others.
  • Defect reporting and enhancement requests.
  • Soon we will post the GUI client as well.

You can find the program at: http://merge-tracking.open.collab.net

7. What’s next for you?

The proposition of Merge Tracking is straight forward: “You did not have it in Subversion before, now you do.” How do you get ready?

  • If you held back on branching because you do not want to deal with the overhead of tracking merges manually, you might need to rethink your branching policies to ensure that you make full use of the benefits of parallel development on different branches. On September 26th CollabNet will organize a webinar about branching strategies. CollabNet Subversion consultants Bob Jenkins and Auke Jilderda will explain different branching strategies and show how Subversion 1.5 can support them. Sign up here.
  • Merge Tracking improves the quality of your merges and adds traceability. Join the Merge Tracking Early Adopter program and learn the new feature now. Also, download our client when it is available and try it out. It will help you to get ready for Subversion 1.5
  • If you have not deployed Subversion yet because you are waiting for Merge Tracking, now is the time to jump off your legacy tool and onto Subversion.

8. Background: CollabNet and Merge Tracking

CollabNet was strongly involved with the development of the Merge Tracking feature:

  • CollabNet organized a customer summit for requirements gathering (Jan. 2006).
  • CollabNet employees wrote most of the specifications.
  • CollabNet led the development effort of the Merge Tracking feature.
  • We started the Merge Tracking Early Adopter Program.
  • CollabNet will develop GUI clients for Merge Tracking.

Check back within the next few days for more info about our GUI client and sign up for the Branching and Merging for Subversion 1.5 webinar.

Tagged with: , , , , , , , , ,
Posted in Subversion
13 comments on “Subversion Merge Tracking – The Basics
  1. John says:

    The italicised text is really hard to read (or maybe I’m getting old) – italics should really only be used for individual words or brief comments, or large blocks in a foreign language that you don’t really expect anyone to read (c.f. the Elvish poetry in Lord of the Rings).

  2. The only italics I see are in the “About the Author” section at the bottom.

  3. Nazar says:

    This is a very welcome and significant development indeed. Great work and we all look forward to this feature being incorporated into clients like TortoiseSVN.
    Thank you very much.

  4. arjan says:

    I have been waiting for this feature for a long time. It’s undoubtedly one of the most important contributions to SVN since its initial release. Really great work this!
    Currently merging is indeed always a troublesome effort, especially for me since it’s highly desirable at our company that we merge often between our “live branch” and development branch. I now have to manually write down revision numbers in the commit logs and use these for the next merge. This is quite tedious and in fact doesn’t even always work correctly.
    Keep up the good work! 🙂

  5. Nathan says:

    This is great news!
    But for this to be _really_ useful, SVN should support “true rename.”
    See: http://subversion.tigris.org/merge-tracking/requirements.html
    (under the subtitle “Track Renames in Merge”)
    … and according to the bugtracker ( Issue 898), it looks like it won’t be in 1.5. That’s disappointing.
    Without true renames, merging renamed files will continue to result in a risk of data loss.

  6. Christophe Meresse says:

    Chapter 2: Auditability. When you merge a feature branch back to trunk, the history of trunk only records that a merge happened, but not what was merged. It is therefore hard to find out afterwards what exactly was merged when into trunk.
    -> Subversion does not even record that a merge happened, it only records that a modification happened (No way to know that the modification comes from a svn merge command…)

  7. Christopher, As of SVN 1.5 that will no longer be true. See this post which was linked to above:
    http://blogs.open.collab.net/svn/2007/06/merge_auditing_.html

  8. Indeed, but Christophe’s feedback is technically correct though because in this particular paragraph I discussed Subversion up to 1.4. It would have been more accurate to write that Subversion currently only records the change and does not know by itself that it was the result of svn merge or not. Most developers will “track” the merge in the log message.
    Mike Pilato wrote a great post about the importance of detailed log messages: http://blogs.open.collab.net/svn/2007/05/the_subversion_.html.
    Tx,
    Guido

  9. The preview of CollabNet’s GUI merge client is now available at http://merge-tracking.open.collab.net. I encourage you to test it out and test Subversion 1.5 as well. The more people help the community with testing Subversion 1.5, the better the release will be when it is available.
    Tx,
    Guido

  10. Brandon McCabe says:

    This is all great, however Subversion is still missing good interactive merge support. Merge tracking simply cannot compare. The CVS team client for Eclipse has great interactive merge support, and I can’t see how a Subversion and plugins like Subclipse and Subversion have not made an effort in this direction? Although I realise Subversive attempted it. This GUI client is not going to help much on very complex or large merge issues.

  11. Hi Brandon,
    The GUI merge client does more than merge tracking, it gives you access to all of Subversion’s merge features. Have you tried the client and evaluated it? We’d love to hear from you what specific features and capabilities you are looking for. Please post a message in the forum at http://merge-tracking.open.collab.net/servlets/ForumMessageList?forumID=57. One of the things we really want from openCollabNet is to engage with users on the roadmap. If you post a message in the forum, others can chip in with their ideas and requirements, and CollabNet engineers and product managers can participate in the discussion.
    Thanks,
    Guido

  12. Derek Mahar says:

    This is great news, but why did this feature take almost two years to develop? Linus Torvalds implemented merge in Git in just two weeks (see http://en.wikipedia.org/wiki/Git_(software)#Early_history)! Git, now just two and a half years old, has very sophisticated merging. Subversion, however, is seven years old and only now has Merge Tracking! What took so long?

  13. Kevin Dietz says:

    MergeMagician, http://www.mergemagician.com, is focused on automated merging for Subversion and other SCM tools.

Leave a Reply

Your email address will not be published. Required fields are marked *

*