This is the second blog entry resulting from a recent webcast focused on the topic of “Subversion Best Practices – Branching Successfully.” I answered about half of those questions in the previous entry a month ago and will do the other half in this posting. Again feel free to ask more questions via our discussion forums.
We already have a working SVN+Apache. Why change to Subversion Edge?
There a couple of very good reasons to consider this change. First, it is great that you’ve got the combination working well today, but how easy is it going to be for you to keep them up-to-date and in sync? Subversion Edge is a package of Subversion, Apache, and ViewVC, along with any dependencies that have been tested and certified to work correctly together. Once installed, you are informed if a newer package is available that again has been certified to work properly. And the upgrade can be automated so that you can keep everything in the stack current, compatible and certified to work together.
Second, Subversion Edge comes with a powerful and secure Web-based console that can help you create and manage Subversion repositories; allow you to browse your repositories, via ViewVC; give you the ability to view and manage both Subversion and Apache logs; have visibility into server health statistics at-a-glance; provide you with the ability to assign user roles and permissions to access your repositories; and allow you an easy integration with LDAP. All this comes at the same low price of free.
How does CollabNet Subversion Edge relate to your CollabNet TeamForge platform?
CollabNet Subversion Edge uses some of the same code as the TeamForge platform, but it is solely focused on supporting the Subversion server aspect of the larger ALM (Application Lifecycle Management) functionality. TeamForge can use Subversion Edge as Subversion servers that it manages. This makes it easy for organizations to go from having the free Subversion administration benefits of Edge to utilizing the large ALM functionality of TeamForge without having to really impact their Subversion servers.
Are there plans for Subversion to address its issue with supporting cyclic merges?
Yes. There’s been a lot of work addressing the core functionality of merge tracking and tree conflicts that has taken a lot of time since those features were introduced, but it has always been the plan to address the cyclic merge issue as well as continue to implement additional functionality that is a part of user’s definition of merge tracking. CollabNet has plans to look at this issue in the 1.8 release, but we can’t guarantee it will be resolved in that release. We will help work to get it supported as soon as is feasible.
We have no change manager or testing manager (due to staffing limitations) and currently use a locking model but will be moving to Subversion. I’m interested to hear how you would handle this common (for us) scenario: – Multiple long term projects starting on various dates – Weekly “scheduled” changes that are not project-related – support only of a transactional system – Emergency changes (law changes, etc) not related to weekly cycle or project cycle. – “Other” changes — usually mini projects on-the-fly How can you use branching safely and effectively – testing – apply changes?
The answer, for better or worse, is a lot of branching. You want and need isolation for each of the various “releases” that your process entails. That means branches for all the development efforts you list with the potential exception of the emergency changes which could be done on the trunk (as quick fixes applied to what is already in production). Obviously the existence of numerous releases being in process simultaneously calls for the use of the stable trunk model that was discussed. All branches would likely come from that stable trunk.
The long term projects would lend themselves well to the idea of cascading changes with each regularly merging changes from its planned predecessor. The most imminent release would regularly merge from the trunk (assuming emergency bug fixes were either done there or merged there to reflect the latest in production). When a long term release was complete, it would be merged into the trunk to reflect the latest in production (and be ready for emergency and weekly bug fixes) and the next planned release would then merge regularly from the trunk rather than from a predecessor’s branch.
The weekly changes would also branch off of the trunk and like the most imminent long term release, they would need to merge in changes from the trunk. When the release was complete, it would be merged to the trunk. The “other” changes would be treated much like the weekly changes since there is no definite relationship about where it would fall in relationship to other releases in process. That means it would merge in changes from the trunk along the way. If a timeline was known, then it should mimic one of the long term releases in getting changed merged in from the release that is planned to proceed it and the release that is planned to follow it would merge in changes from this “other” branch. When released, the “other” changes branch would be merged into the trunk.
While there is considerable branching in your situation, it is necessary to support the isolation required. It doesn’t have to really be complicated in process however as the clear relationships between branches and the trunk can drive a very consistent and controlled process to propagate changes (i.e., either the branch regularly merges in its predecessor release branch or the trunk).
Should the number of branches be subsequently limited? Consider Windows N-way merge.
There isn’t a need for an arbitrary branch number limit. As I said in the webcast, you branch because the work needs isolation so if you have lots of simultaneous needs for isolation, then you need lots of branches. Subversion looks to merge the diff of two trees to a third revision tree (i.e., what is in your working copy) so you work through things usually one branch at a time.
Your diagram of the agile release methodology is helpful, but we have multiple release branches active at the same time. Is there a diagram that you could share or email that would illustrate this type of scenario?
That should be the stable branch model that was visualized prior to the agile release method. In that model you have multiple releases in process at the same point in time that’s the scenario that model is designed to address. Let me know if you still need a diagram for it, but you should be able to get the slides from the presentation.
At what point is the use of a Release Branch overkill for a small organization with regard to maintaining bi-directional merge situations?
I’m not sure what exactly we’re talking about with bi-directional merge situations so my answer may not be what you’re looking for so feel free to follow-up in our discussion forums.
Small organizations often follow the unstable trunk model as they are unlikely (and to some extent unable) to support multiple releases in process simultaneously. There are also cases when the release branch, that we discussed being created around the time of feature complete, may be overkill as well. A foundational definer of process for me is the KISS principle. I am all about keeping things simple so if the release branch isn’t needed, then don’t create it. That said, be careful that you’re not avoiding it thinking you are getting around merging or having to address conflicts. If work starts on the next release before you reach production, then without a branch that work is at risk in someone’s working copy (which isn’t likely backed up) or work isn’t started waiting on that production release. Most projects will still need to branch to address production bugs while the next release is being worked on the trunk so again be sure you’re not skipping the release branch for the wrong reasons.
How can we merge multiple release running parallel and one goes in production earlier other still stays and goes later?
I’m not certain I understand this question, but I’ll answer based on what I think is being asked. So the scenario I envision is that there are two releases being developed simultaneously and one goes into production. Prior to that happening how could the two be merged? The only way to do that successfully is to know (or be relatively certain) of which one will go into production first. In that case, that one should be merged to the release that will be made at a later date since it will have to have all that’s been put into production previously. If you don’t know which will precede the other, then you’ll just have to delay the merge until that decision is made (perhaps as late as when the first one is designated production-worthy). That leaves you open to having significant issues arise once you do the merge that you’ll have to work out that may delay the second release, but you don’t have a choice.
If you’ve done a reintegrate merge of a branch back to trunk (creating say revision 10) and thus obsoleted that branch, but then revert via a reverse merge of revision 10, can you reintegrate that branch again??
Yes. Keep in mind that Subversion doesn’t do a historical delete of a branch in obsoleting it so the branch is still out there just not accessible via the latest revisions. You would most likely rollback the deletion of that branch and do the reintegrate again.
Once conflicts are resolved, does Subversion provide some sort of history on how conflicts were resolved?
No. There is no history of there being conflicts let alone how they were resolved. If this is the result of a merge (versus an update), then you could do a diff to see what is different between the two revisions that were merged, but there is no history retained about the conflict. I’m not aware of any version control tool that does keep this history though there could be one that does.
Is there a general or best practice for the use of a promotion model that allows you to pull/deploy code at different ‘promotion levels’ from the same branch?
For most projects, a promotion model needs to address the whole of what you expect to be included in a release. You expect that issues/defects will arise as you take your release through the promotion model and thus you expect to apply fixes. The release branch can support these expectations if you use tags applied to that branch to identify the candidates for each stage/phase in your promotion model.
You can do your work on the release branch and then as you get to a point for the first stage of your promotion model, you apply a tag to the appropriate revision on the release branch (normally the latest or “HEAD” revision). The build is done on the tag and if issues arise, then you go back to the release branch and make your changes. A new tag is applied for the second candidate for the first stage and a build done. If that candidate passes that stage, a new tag is applied to the same revision for the second stage. I won’t go on with this further as I hope you have the idea. The tag is what you use to deploy to your environments.
Can we create a branch with nested working copies in any of the methods?
I’m not sure I completely understand the question so I’ll do my best to answer it a couple of ways that I think it might be meant and you can follow up through the discussion forums if I’m incorrect in my understanding of what you’re asking.
Can you create a branch where externals are used? Yes, the svn:external property will be “copied” along with the directory when you create the branch with the same value as the source. You can then keep that value or change it according to what you want for that branch. This effectively supports multiple nested working copies via the externals.
Can you create a branch where the switch operation has been used on directories or files? Yes, this may make the branch contents more difficult to understand after you’ve done it and for you to be sure you are merging appropriately to it, but Subversion will allow you to branch from the structure you have in your working copy versus a path in the repository.
How do you do integration testing that should include multiple feature branches without committing to merge into the release branch?
So you want to do some type of “what if” testing? I’m personally not too keen on this type of testing as you’re testing something that you don’t see as complete and that may not be released together as they stand. I think integration testing should happen when features are pretty much feature complete. Not to say that there won’t be issues that arise to be addressed, but the code should be solid and therefore merged to the release branch.
If you’re determined to do this type of testing (and there can be good reasons for doing so), then you can do the merging to a working copy and not commit the results. You’d establish a working copy from the release branch and then execute merges from each of the feature branches, resolving any conflicts with each merge before proceeding to the next one. You will end up with code that you can build and test that includes the feature branches, but isn’t committed to the release branch. You can either abandon that working copy or use the revert operation to get it back to a revision that had been committed on the release branch.
Could you describe a little bit more about “tags” and briefly about how they should be used? I would like to know more on when to tag.
I’ve lumped two questions together here which show the interest and, frankly, the lack of understanding of what the purpose is for tagging (trust me this is a consistent theme I’ve seen in my 14+ years in configuration management). I’ll try to do a blog post in the next few weeks focused on this topic and at some future date we’ll dedicate a webinar to just this topic. Let me try to give you a foundational understanding at this point at least.
Tags are the ability for you to place a meaningful name on a revision of files that either has historic value or that multiple users need to find quickly and easily. Subversion’s general approach to version control provides a similar snapshot approach where a global revision number relates to the overall state of the repository after the applying of a single commit. However, revision 12345 may not be meaningful to others especially if they are reviewing the development months or years later. Instead, a tag that indicates that revision 12345 was release 2.5.0 is much more useful to those reviewers.
Subversion implements tags as “cheap” copies so they are in fact additional paths in your repository. Subversion has no internal concept of tags so while you want and need this concept we’ve called a tag, you have to be careful to create it where you and others will be able to later find and understand it. That normally calls for a /tags directory.
This gets us to the core of your question which is how and when tags should be used. Most teams seem to pretty consistently use them to mark the revision that has been anointed as production-worthy. And that’s a very important time to tag. We need to make it easy for everyone to know what reached that designation both now and historically. However, it is also important to know the process that the code went through to get that designation. Development normally has a process that calls for an extended period of general development where we do unit testing and potentially continuous integration. At some point though, we get serious about the release and begin to treat specific revisions differently by putting them through formal testing processes. Sometimes we merely designate a candidate release that we send through black box testing where if it passes, it is deemed production worthy; and if it fails, we make our fixes and identify a new release candidate. Other times we have a very defined process of multiple stages (e.g., QA->UAT->System Test->Production).
These candidates or stages are potentially very important to track as there are many questions that they can help us answer if we have identified them. We can better identify where our process may need improvement or what specific files are the most troublesome. There is a wealth of information that can be harvested if these revisions are appropriately tagged.
Lastly, there are arguments for tagging both before and after building for the initial stage in your process (the assumption and best practice is you only build for the first stage and the bits are promoted to subsequent stages). It is my belief that tagging first is the best practice for a couple of reasons. First, you have an opportunity to identify another place in your process that may need improvement (i.e., the handoff from development to release management). If you include a counter at the end of that first stage tag name, you can see if that handoff is an issue by noting how many times it took before the official/formal build could be executed successfully. Second, you are that much more certain about what was used in a build if you are using the tag itself to establish the working copy (or export) upon which you build.
How about automated unit and module testing? That’s our bottleneck at the moment. Any ideas on getting started?
That’s a pretty broad question so I’ll try to answer it in as simple a manner as I can and assume you’ll ask follow-up questions through our forums if you have additional detailed questions.
Many organizations utilize some type of continuous integration process where such testing is executed after a successful build and prior to interested parties being given a status of the current state of development. Most of the CI tools will allow you to automate unit testing (like that done by junit for an open source example) and any other tool testing you may have available to your organization. The best place to start is around functional/regression testing. Later you can grow into performance/load testing.
Binaries – we also need to have the development environment saved “tagged for a revision” because just source does not an equal binary.
Absolutely. There are definite situations where the scope of what needs to be version controlled includes the development environment (e.g., OS release, patches, compiler, libraries, etc.). Since these additional items are reasonably static in nature, it is a good practice to put them in a different repository and use the svn:externals property to link them to the source code. Make sure the mapping guarantees an “immutable” revision of these items when you go to apply the tag on the source code repository as the tag will be associated with the property rather than being applied to the target of the external itself. This isn’t a requirement for the majority of development environments, but it is an important one to control for those who have the requirement.