Upcoming Subversion 1.5 Feature: Changelists

One quality of a typical engineer is a heightened sensitivity to potential problem areas. For software engineers, that often means spotting possible bugs in our peripheral vision while working on some nearby chunk of source code. This is both a blessing and curse. You want your bugs fixed, but unfortunately we sometimes get derailed from our primary task by these little excursions into Perfection.

Another fact of a developer’s life is that there are always more things to work on than time to do so. And everything is of the utmost priority to someone. And so here again, we find ourselves working on multiple things at the same time, making slow-yet-incremental progress on each of them so at least we can say about them all with a clear conscience and a forced smile, "As it turns out, I’m working on that right now."

To my managerial readers, I regretfully admit that I have absolutely no solution for those problems. After all, this is a technical blog. But I can (and will) tell you about an upcoming Subversion feature that might reduce the context-switching costs for those developers: changelists.

Whatever the reason, it is commonplace to find that a developer is working at any given time on multiple different, distinct changes to a particular bit of source code. And often each change isn’t nicely contained in some module, safely separate from the other changes. The groups of changes might overlap, modifying different files in the same module, or even modifying different lines in the same file. Now, there are various work methodologies that developers can employ to keep these tasks organized. Some use separate working copies of the same repository to hold each individual change-in-progress. Others might choose to create feature branches in the repository, and use a single working copy that is constantly switched to point to one such branch or another. Still others use diff and patch to backup and restore uncommitted changes to and from patchfiles associated with each change. Each of these methods has its pros and cons, and to large degree, the details of changes being made dictate the methodology used to distinguish them.

But Subversion 1.5 brings a new feature called "changelists" which adds yet another method to the mix. Changelists are basically arbitrary labels applied to working copy files for the purpose of associating multiple files together. Gmail users are familiar with this concept already. Gmail doesn’t provide the traditional folders-based email organization mechanism. In Gmail, you apply arbitrary labels to emails, and multiple emails can be said to be part of the same group if they happen to share a particular label. Viewing only a group of similarly labeled emails then becomes a simple user interface trick. Many other Web 2.0 sites have similar mechanisms — consider the "tags" used by sites like YouTube and Flickr, "categories" applied to blog entries, and so on. Folks understand today that organization of data is critical, but that how that data is organized needs to be a flexible concept. The old files-and-folders paradigm is too strict for many applications.

So Subversion’s changelist support allows you to:

  • create changelists by applying labels to files you want to be associated with that changelist,
  • remove those labels, and
  • limit the scope of the files on which its subcommands operate to only those bearing a particular label.

For example, say that as part of my vast array of things to accomplish today, I need to fix some minor issues in ViewVC related to diff-viewing operations. My work leads me to change a couple of files.

$ cd projects/viewvc
$ svn status
M      lib/vclib/ccvs/__init__.py
M      lib/vclib/__init__.py
$

But while testing this change, I notice something else that’s not working right in the nearby annotation support code. There are, as previously mentioned, many things I can do at this point in order to switch contexts and fix this secondary issue. We’ll use changelists for the purposes of illustration.

First, I want to create a changelist and associate with it the two files I’ve already changed. These helps me to distinguish those changes from the ones I’m about to make for this secondary issue.

$ svn changelist diff-fixes lib/vclib/ccvs/__init__.py
Path 'lib/vclib/ccvs/__init__.py' is now a member of changelist 'diff-fixes'.
$ svn changelist diff-fixes lib/vclib/__init__.py
Path 'lib/vclib/__init__.py' is now a member of changelist 'diff-fixes'.
$ svn status

--- Changelist 'diff-fixes':
M      lib/vclib/ccvs/__init__.py
M      lib/vclib/__init__.py
$

As you can see, the output of ‘svn status’ reflects this new grouping.

Now, I start fixing the secondary issue, which requires that I change a third file. I’m not yet ready to commit either change, though, so I’ll make a changelist for my second change, too.

$ svn changelist blame-fix lib/vclib/ccvs/blame.py
Path 'lib/vclib/ccvs/blame.py' is now a member of changelist 'blame-fix'.
$ svn status

--- Changelist 'blame-fix':
M      lib/vclib/ccvs/blame.py

--- Changelist 'diff-fixes':
M      lib/vclib/ccvs/__init__.py
M      lib/vclib/__init__.py
$

Now, that visual grouping alone is nice, but not entirely useful. If I wanted to see just the diff-related code changes I’d made, I would still need to explicitly name the files I modified for that change on the ‘svn diff’ command line.

$ svn diff lib/vclib/__init__.py lib/vclib/ccvs/__init__.py

That’s not so bad for two files, but what about for twenty?

Fortunately, Subversion 1.5’s changelist support is more thorough than that. I can use the new --changelist option to limit the scope of many subcommands to just those files associated with a given changelist. The result is a much simpler command line, and a much smaller chance of overlooking one or more of the files I intended to examine.

$ svn diff --changelist diff-fixes
Index: lib/vclib/ccvs/__init__.py
===================================================================
--- lib/vclib/ccvs/__init__.py (revision 1157)
+++ lib/vclib/ccvs/__init__.py (working copy)
@@ -112,8 +112,8 @@
     temp2 = tempfile.mktemp()
     open(temp2, 'wb').write(self.openfile(path_parts2, rev2)[0].getvalue())

-    r1 = self.itemlog(path_parts1, rev1, {})[0]
-    r2 = self.itemlog(path_parts2, rev2, {})[0]
+    r1 = self.itemlog(path_parts1, rev1, {})[-1]
+    r2 = self.itemlog(path_parts2, rev2, {})[-1]

     info1 = (self.rcsfile(path_parts1, root=1, v=0), r1.date, r1.string)
     info2 = (self.rcsfile(path_parts2, root=1, v=0), r2.date, r2.string)
Index: lib/vclib/__init__.py
===================================================================
--- lib/vclib/__init__.py (revision 1157)
+++ lib/vclib/__init__.py (working copy)
@@ -240,7 +241,7 @@
   def readline(self):
     return self.fp.readline()

-  def close():
+  def close(self):
     try:
       if self.fp:
         self.fp.close()
$

As you might expect, I can then provide the same --changelist option to the ‘svn commit’ operation.

$ svn ci -m "Fix some diff-related oopses." --changelist diff-fixes
Sending        lib/vclib/ccvs/__init.py
Sending        lib/vclib/__init.py
Transmitting file data ..
Committed revision 1158.
$

Meanwhile, my other changelist remains undisturbed.

$ svn st

--- Changelist 'blame-fixes':
M      lib/vclib/ccvs/blame.py
$

The new --changelist option is present on many of the Subversion command-line client’s subcommands (commit, diff, info, lock, log, proplist, propget, propset, status, update, …). And, of course, there is also the new ‘svn changelist’ subcommand for adding and removing (via a --remove option) files to and from changelists. You can assign files to changelists before you even modify them. And Subversion doesn’t force you to use changelists universally, or even at all. As in most things, Subversion just gives you the tools, and lets you decide whether and how to use them. All in all, though, this new feature should provide Subversion users — especially those with Perforce experience — an easier way to manage those multiple simultaneous works-in-progress.

There are some limitations of the changelist support, though. First, the feature has path-level granularity. If you find yourself with overlapping changes which touch the same file, you’re going to have to work out a different way to deal with that overlap. Also, a changelist is private to a particular working copy — there’s no way to automatically share your changelist definition with other users of the repository. But given the problem the feature exists to solve, that shouldn’t be any big deal — if you need to share works-in-progress with others, use versioned branches.

C. Michael Pilato

C. Michael Pilato is a core Subversion developer, co-author of Version Control With Subversion (O'Reilly Media), and the primary maintainer of ViewVC. He works remotely from his home state of North Carolina as a software engineer for CollabNet, and has been an active open source developer since early 2001. Mike is a proud husband and father who loves traveling, soccer, spending quality time with his family, and any combination of those things. He also enjoys composing and performing music, and harbors not-so-secret fantasies of rock stardom. Mike has a degree in computer science and mathematics from the University of North Carolina at Charlotte.

Tagged with: , , , , , , , ,
Posted in Subversion
15 comments on “Upcoming Subversion 1.5 Feature: Changelists
  1. Rene L. says:

    This is a great new feature, I (and maybe the PHP developers) can’t wait for it 😉
    Is there a content-based granularity planned?

  2. Rob Russell says:

    Interesting idea, I could see myself using this. One question that wasn’t clear to me from your example though: can a file be part of multiple changelists? I’m guessing yes since you used content tagging sites as an analogy. For example: if lib/vclib/__init__.py was added to the ‘blame-fix’ changelist, would it show up in both groups after running svn status?

  3. C. Michael Pilato says:

    Rob: Unfortunately, you’ve just highlighted a glaring oversight in my post. I had intended to explicitly point out that while part of the beauty of tagging sites is that a given object can have multiple tags, Subversion does not yet support that. Somewhere between the top of the post and bottom, I managed to forget that subtle (yet important) detail. A previous draft even had an example showing to change which changelist (singular) a file was associated with, but that got edited out.
    Here’s some command-line interaction that demonstrates the effect of trying to add a file to multiple changesets:
    $ svn cl foo plaza.jpg
    Path ‘plaza.jpg’ is now a member of changelist ‘foo’.
    $ svn cl bar plaza.jpg
    Path ‘plaza.jpg’ is now a member of changelist ‘bar’.
    $ svn cl baz plaza.jpg
    Path ‘plaza.jpg’ is now a member of changelist ‘baz’.
    $ svn info plaza.jpg | grep Changelist
    Changelist: baz
    $

  4. Rob Russell says:

    Thanks for the clarification. Sounds like it’s still a usable feature, just needs a little more forethought when I make a Changelist.

  5. Mike L. says:

    To be honest, I don’t understand why changelists where added to the *command line* client, because IMHO it’s the task of a Subversion front-end. See smartSVN 3 for an example.

  6. C. Michael Pilato says:

    Subversion’s changelist functionality was (as is most Subversion functionality) designed modularly, with the business logic implemented in the core Subversion libraries so it can be shared by all programs which use those libraries, and leaving the UI bits to upper layers of the software. That means that the slew of Subversion front-ends which use Subversion (including ‘svn’) need only implement a bit of UI stuff and *poof*, they have changelist support. And what’s more, they have changelist support that is cross-tool compatible — I can create a changelist with ‘svn’, and still make use of it in TortoiseSVN (or vice-versa). I doubt that this can be said of rapidSVN’s changelist support. Of course, that doesn’t mean that the individual tools wouldn’t do a better job of implementing changelist support. Certainly they wouldn’t necessarily be limited by some of the constraints present in Subversion’s core implementation.
    As for why the functionality was provided in the command-line client, I believe that was largely because the developer that added the feature was used to having something similar in his Perforce command-line client.

  7. Dan C says:

    This sounds like a handy feature — it will certainly reduce the amount of time I spend svn diffing before checkin to pick out the files that I actually want to check in.
    As Rene L mentioned, even more helpful would be something along the lines of darcs interactive hunk-picking (too often I find myself with unrelated changes inside a single file).

  8. kamleshjkl says:

    changelist is a great feature. I have used it in Perforce and I definitely want it in Subversion.

  9. Michael Haggerty says:

    > $ svn cl foo plaza.jpg
    > Path ‘plaza.jpg’ is now a member of changelist ‘foo’.
    > $ svn cl bar plaza.jpg
    > Path ‘plaza.jpg’ is now a member of changelist ‘bar’.
    > $ …
    Yikes, there is no error or warning if you move a file from one changelist to another? This sounds like trouble. If I’ve modified a file as part of changelist A, then (forgetting changelist A) add it to changelist B, then commit changelist A, then my commit is probably broken.
    What would be really sweet would be a way to stash a selected changelist away somewhere (e.g., as a patch) and unapply the corresponding changes from the working copy. This would allow me to test a spontaneous mini-fix before committing it, without having to worry that the “main” work-in-progress is affecting the results my unit tests (the main work-in-progress might prevent my code from even compiling and/or running, or even worse, the mini-fix might work *only* because of changes made in the work-in-progress).
    I’ve been working with quilt a lot lately, and appreciate its ability to apply/unapply patches easily. With quilt, it is easy to “pop” the main work-in-progress from the stack of applied patches, implement the quick fix, test, commit, then “push” the main patch back onto the working copy. The SVN changelist concept is obviously somewhat different than quilt’s model (for example, quilt imposes a linear ordering of the patches under its control), but the analogy could nevertheless be useful.
    “Stashable” changelists would also be a way to allow a single file to participate in multiple changelists at the same time, provided all but one of the changelists is “stashed” away. Well, I guess that isn’t really “at the same time”, but rather “in quick alternation” 🙂

  10. C. Michael Pilato says:

    Great feedback, Michael. I’ve taken the liberty of pointing the Subversion developers at this comment — watch for discussion around it on dev@subversion.tigris.org.

  11. Michael Haggerty says:

    I just created Subversion issues #2947 and #2948 to document my suggestions.

  12. James Alexander says:

    How to update with changelist? Send me a command line arg.

  13. James Alexander says:

    when i use: svn update –changelist ch-list-name
    svn: Unknown changelist ‘ch-list-name’
    What is problem on this command? But I committed files using ‘ch-list-name’.

  14. C. Michael Pilato says:

    There are a few possible reasons for what you are seeing that I can think of, James.
    First, did you commit with –keep-changelist? If not, the files in the changelist that you committed will have been removed from the changelist post-commit. When you then try to update using that changelist name, it can’t be found as a label on any of your files.
    Secondly, did you commit those files from a different working copy? Changelist names are per-working-copy, and do not persist in the repository or across working copies. So a file in changelist “foo” on your work laptop might not be in that changelist on your home computer, even if both machines have checkouts of the same repository location.
    Lastly, you might be seeing the problem I recently became aware of. I turns out (and I didn’t know this when I wrote my blog post) that “–changelist foo” expands to “crawl $CWD looking for files in the changelist foo”. Yep, I said “$CWD”. So, if you are sitting in /home/james, and you run ‘svn update –changelist foo /projects/myproject’, Subversion (today) will crawl *not* /projects/myproject, but /home/james, looking for items in the changelist “foo”. That, I claimed in a recent dev-list post[1] is an entirely unexpected behavior that is destined to confuse our users. So, we’re working on fixing that right now.
    [1] http://svn.haxx.se/dev/archive-2007-12/0666.shtml

  15. Eder Sousa says:

    I need know how to install Subversion im my computer and um network company.
    Please Help !

Leave a Reply

Your email address will not be published. Required fields are marked *

*