Merging from Foreign Repositories

For many folks, the forthcoming Subversion 1.5 release gets billed as something like "merge tracking and some other stuff". It’s probably true that the work put into the new merge tracking feature outweighs the investments in other individual areas. But of course, Subversion 1.5 isn’t about merge tracking alone. There are several other major features coming in this release. And as you’d expect, the Subversion community has made countless bugfixes over the past year-and-a-half, too. But today I want to briefly talk about one half-bugfix, half-feature, that might have slipped in under your radar: merges from foreign repositories.

Subversion has for some time now allowed you — kinda, sorta — to perform merge operations where the sources of the merge live in one repository but the target working copy (the thing you are merging to) comes from a different ("foreign") repository. After all, since Subversion models merges as essentially the application of a diff, you might expect that the source of those diffs wouldn’t matter all that much. But this is a feature that nobody really talks about: the Subversion public API doesn’t mention it; the Subversion book doesn’t mention it; Subversion’s release notes don’t brag about it. In fact, the only reason I even knew the feature existed was because of code comments I came across while hacking on Subversion’s merge tracking feature. Why the shroud of secrecy? Well, it might be one of those accidental features — the ones that weren’t specifically intended to work, but mostly did anyway. Or perhaps the reason is that it only worked sometimes. At any rate, merges from foreign repositories is practically a hidden feature.

Apparently empowered by the spirit of "by golly this ought to work", one openCollabNet community member found this feature, though and found its shortcomings. User "argeman" posted the following to the merge-tracking beta program forums:

i have tested the svn 1.5 alpha2 a bit (converted some repositories and played around with those). it works completely great until now. The only thing that is still not working (it never worked in previous subversion-releases): If i try to merge changes from a different repository, that will not work for added files.

I’d never really played with merges from foreign repositories myself, but immediately recognized the value of the feature. I can summarize it in two words: "vendor branches". Vendor branches are a common way of maintaining private customizations to somebody else’s code without mirroring that code change-for-change. There are different ways to make this work. For example, you might use individual branches which are pure mirrors of a given vendor’s release package contents, and then another branch which contains the currently employed version of the vendor package plus your private modifications. Your upgrade your customized package by applying the diffs between pure vendor releases to your customized copy. Alternatively, you might keep in a version control branch only the customized copy of the vendor package, and use diffs generated as patches between unversioned vendor package contents to upgrade that branch. And some masochists avoid the powers of version control altogether and simply maintain custom patch files which need to be updated with each new release of the vendor’s package. (If you’re one of those, there are people who can help you deal with your illness.)

The ability to merge from foreign Subversion repositories, then, provides a sort of hybrid approach similar to the first and second methods I previously mentioned. In this approach, you need to maintain only that single customized copy of the vendor’s package instead of also tracking pure mirrors of the vendor’s release (because the pure versions can be readily obtained from the vendor’s Subversion repository). But now you get to use version control tools to merge the differences between those pure versions into your customized copy, instead of dealing with messy patch files.

Unfortunately, "argeman" wasn’t posting to say, "Hey, I just found this feature and it works great!" The post was to tell us that the feature had potential, but failed to handle files which needed to be added as a result of the merge. Here, however, is the joy of working on open source software. It took me all of 40 minutes to change Subversion’s code to fix the problem and compose a regression test, plus some additional time thereafter doing some related follow-up fixes. So when Subversion 1.5 ships, I expect merges from foreign repositories to be supported at the same basic level that intra-repository merges are. Renames in the merge source will still cause the same complications that they always have in Subversion, and the merge tracking logic will be bypassed when merging from foreign repositories, but the merge should complete successfully in the commonmost situations.

C. Michael Pilato

C. Michael Pilato is a core Subversion developer, co-author of Version Control With Subversion (O'Reilly Media), and the primary maintainer of ViewVC. He works remotely from his home state of North Carolina as a software engineer for CollabNet, and has been an active open source developer since early 2001. Mike is a proud husband and father who loves traveling, soccer, spending quality time with his family, and any combination of those things. He also enjoys composing and performing music, and harbors not-so-secret fantasies of rock stardom. Mike has a degree in computer science and mathematics from the University of North Carolina at Charlotte.

Tagged with: ,
Posted in Subversion
14 comments on “Merging from Foreign Repositories
  1. Rup says:

    Wow, neat! Which parts of the system need to be 1.5+ to make this work – just your client, or your client + your server, or your client + your server + their server? Thanks.

  2. C. Michael Pilato says:

    This functionality *should* require only a 1.5-class client (and some quick testing seems to confirm as much). And when I say “client”, I mean command-line client. Not that other Subversion front-ends can’t have this feature, too, of course. I just can’t promise that they will.

  3. Kim Sullivan says:

    One step closer to distributed version control systems 😀 But I’m afraid you’re opening a can of worms for yourself, people will want merge tracking even when merging from remote repositories. If I understand it correctly, people will have to track merges from remote repositories “the old way”, via commit logs? Maybe that’s why the feature was hidden in the code – while enabling it is technically possible, it’s sort of a pandora’s box.

  4. I’m using SVK and foreign branches when I want to do merge from other repositories.

  5. C. Michael Pilato says:

    Kim, I’m not sure I agree with the Pandora’s box analogy. People want two things: the ability to merge at all from foreign repositories, and the ability to track those merges. Providing support for the former without the latter does not preclude us providing the latter in the future.
    Consider the alternative, where people not only still have to track merges “the old way”, but they also have to perform those merges using patch application which a) requires addition tools, b) doesn’t make use of the new interactive conflict resolution feature, c) doesn’t carry property changes, and d) doesn’t carry file/dir addition/deletion changes (tree changes).
    I appreciate the warning, but I personally think users will welcome this as an enhancement, and as a positive first step toward the ideal.

  6. Daniel says:

    Florent, how do you do that with SVK? Is there a howto somewhere that explains how to use foreign branches?

  7. A great feature indeed, making customisation of open source projects which are using SVN themselves an easy exercise (without wasting space on the local SVN server). But I found that the merge regularly reports conflicts although there shouldn’t be any.
    I’d pay you a drink if you can spare another 40 minutes to fix this, Mike 😉
    See http://subversion.tigris.org/issues/show_bug.cgi?id=1957 and http://subversion.open.collab.net/ds/viewMessage.do?dsForumId=4&dsMessageId=167883.

  8. Apologies, my fault. It all works fine and this is one of the coolest features of SVN since the introduction of the externals ! (I had my config enabling the svn:eol-style=native auto-prop which caused the conflicts. All it needed was a –no-auto-props with the svn add). So please accept my apologies about the fuss and accusing you of a bug. You’ve added a very, very handy feature, keep the good work coming 🙂

  9. C. Michael Pilato says:

    @Diego: do I still get the drink? 😉

  10. Baptiste says:

    Hi, small question about this feature: I just tried using it and I can’t make it work.
    I get the following error:
    $ svn merge svn://svnserver/repo1/trunk/project-core svn://svnserver/repo2/trunk/project.core
    > svn: No such revision 1754
    I suppose this might be due to my subversion server version: my server is a 1.4 one, and my client is 1.5.2. Should this work? Should I upgrade my server to subversion 1.5.x to be able to merge from foreign repositories?
    If it should work, what should have been the right command?
    Thanks a lot.

  11. C. Michael Pilato says:

    @Baptiste: Actually, I suspect the problem is much less sinister. The command you invoked is interpreted by Subversion to mean, “Please calculate the difference between svn://svnserver/repo1/trunk/project-core@HEAD and svn://svnserver/repo2/trunk/project.core@HEAD, and apply that difference to the working copy located at “.” (the “implicit dot target”, we call it).
    This is not a valid request to make of Subversion. A supported “foreign repository merge” requires that both of the merge sources live in one repository, and that the target (the working copy) reflects another. But your two merge sources are in two different repositories (repo1 and repo2), and that’s not supported. I suspect that Subversion has merely determined that HEAD for one of those sources is r1754, and then — having presumed that the other source is from the same repository — tried and failed to reference r1754 for that URL.
    Did you by chance expect that maybe your command meant, “Merge changes from repo1 to repo” ? If so, know that Subversion doesn’t support URL *targets* of merges. All merges must be aimed at working copies (on your local computer), and then committed.

  12. Ted Jordan says:

    I am looking at doing some project specific modification to some open source packages. Some of these modifications will be applicable to other projects so I was hoping to maintain a separate repo of open source packages with modifications common to all my projects. Foreign repository merges are exactly what I was looking for. Mike, thank you so much for getting this to work.

  13. Ilya says:

    all is ok, except that it doesn’t commit properties on added files (

Leave a Reply

Your email address will not be published. Required fields are marked *

*