Archive for September, 2008

Developing in streams, releasing from streams

September 25th, 2008

By Matthew Laudato

When I read the following blog, Developing in branches, releasing from trunk about a software configuration management development process that was constrained and tortured by branches, I had an ‘aha’ moment. Actually, I had three moments. They went something like this:

1. Branches are obsolete, so why does anyone still use them?

2. People use them because they are unaware of the alternatives – specifically streams.

3. I must act quickly to save my coding comrades from any further branch related pain.

If there is a cyberspace equivalent of shouting from the rooftops, then this blog post is it. My goal is to help the author to simplify his development process, and to free him and his team from the time-wasting, error-prone, outdated branch-induced hysteria that is clearly evident in his post. OK, so maybe that last sentence was a little hysterical, but you get the point.

In the post (which I encourage you to read first), the author outlines a branch-centric process that goes something like this:

- create issues in JIRA for feature (defect, etc) work

- create a Subversion branch for each JIRA issue, named by the JIRA issue ID (eg, SUP-3452). Developers write code on these feature branches.

- create a JIRA release (for example, R4.2) and target the issues created above to this release

- create a Subversion branch (off of the ‘trunk’) with the same name ‘R4.2′.

- Write a bunch of scripts that look at the JIRA release, find all the issues for that release, look at Subversion for the branches that are named for said issues, merge these branches to the release (R4.2) branch.

- Do some testing on the R4.2 branch

- Tag the release R4.2 branch

- Release software

- Merge the R4.2 tag back into the trunk

Rinse, repeat for the next release.

For some reason, I’m reminded at this point of a character in Tom Robbins’ “Even Cowgirls Get the Blues”. The character had a little business card that he would hand to people. One one side it read, ‘I believe in everything, nothing is sacred’. On the other side it read, ‘I believe in nothing, everything is sacred’. When I think about the author’s process, one side of my brain says ‘there is nothing wrong with this process’, and almost immediately, the other side screams ‘there is everything wrong with this process’. The truth is somewhere in the middle, and it all comes down to branches.

The problem with this process is that branches get in the way. The idea of isolating features in your SCM tool is a good one. Although I don’t personally agree with the idea of dedicating an SCM construct per-feature, there are plenty of people who do. (See Stream-per-task paradigm, for example by Dave Thomas). But that’s not the point. Whether you put one feature or ten in it, a branch constrains your ability to work efficiently. Once you create a branch, it loses all contact with the outside world, most importantly, with its own history and its relationship to other branches. The author even admits jokingly that a ‘release manager’ will likely be needed to handle the merges. Ya think?

Here’s an alternative process that uses AccuRev streams:

- Create issues in JIRA as before for the desired features and fixes and assign to developers. Use AccuBridge for JIRA to sync those issues into AccuRev

- Create a stream for R4.2

- Create (or reparent existing) developer workspaces off of this stream.

- When developers finish coding, they use ‘promote to issue’ to push the code to the R4.2 stream (automatically creating a change package that associates the code changes with the JIRA issue, and automatically updating the JIRA issue with an annotation about said code changes).

At this point, the R4.2 stream contains everything that you want to release. If you decide to back out a feature, you can use the revert by change package feature to remove it from the stream. The code changes are always available in the developer’s workspace if you need to revive them in the future. Once it is time to release, you can create a snapshot of the R4.2 stream to preserve it, and then you can go about working on the next release. If R4.2 and R5.0 (The Next Big Release) are happening in parallel, you could create an R5.0 stream as a child of R4.2, so that the R5.0 stream automatically inherits all of the R4.2 changes.

The advantages are many:

- Fewer and simpler merges

- Right-sized feature isolation and management

- Inherent parallelism through code inheritence

- Fewer process steps

- Built-in automation and thus fewer opportunities for manual errors.

I could go on, but I hear the footsteps in the hallway. There are people out there who don’t want you to know about streams. “We’ve always used branches,” they’ll say. “We’re too smart to use some vendor’s solution,” others will say. I say, don’t listen to them. Fight the power. Walk right up to them and say “Why branch when you can stream?” And then go ahead and watch your team spend less time fighting with branches and merges, and more time writing features and fixes for your customers.

1 comment »

Posted in AccuRev, Best Practices, version control

Tags:

Dr. Strangecode, or How I Learned to Stop Worrying and Love Old Code

September 17th, 2008

by Chris Boran

Several years ago I happened to be browsing in my favourite local bookstore and one book in particular caught my eye: Martin Fowler’s Refactoring: Improving the Design of Existing Code

This is the book that changed my whole career. Up until that point, I had lived in a constant state of fear of change. I viewed old code as a house of cards – if I wasn’t very careful, it was all going to come down around me. Whenever my boss asked me for advice about what to do in a given area of the code, my answers were almost always similar to:

The risk of doing anything but the smallest possible change is huge – so either we do something that is an ugly hack that we will regret later, or we need to take the whole thing apart and re-implement the whole subsystem from the ground up.

And predictably, my boss’s answer was always something like:

Okay, we will live with the hack for now, but in the next release we will make time to do it right.

Of course every release we were forced to make many similar decisions. When the next release would come, the newly conceived product features would get priority over fixing code that was already working passably. In practice we might get lucky and be able to spend lots of time rewriting a single small subsystem, but introduce ugly hacks that would put 4 other systems on the map for future re-implementation.

The end result is affectionately referred to as a Big Ball of Mud. Yup, it is every bit as pleasant as it sounds. Life is just so miserable when you come to work every day knowing you are going to have to pack another layer of mud on the ball. You gripe about it. Your teammates gripe about it. Your boss gripes about it. Somehow you never seem to make a whole lot of forward progress.

As I read the book, I was at first skeptical. I thought that bad old code needed to be thrown away and not revitalized! But I wanted to see if it would work, and so I set out to try it. I was very careful to follow the techniques exactly as they are laid out in the book. I made sure not to use the word refactor when I really meant rewrite. I was careful to refactor the code every time I saw something I wanted to change and not just note it for later. I made sure that my refactorings were small. I took my time with it.

Obviously working this way required that I also be doing very good unit testing, but I had already bought into Test Driven Development. I was already writing unit tests for code before fixing bugs in an attempt to prevent regressions. Running these tests after each refactoring was not a big challenge for me.

Bit by bit I discovered the truth. By applying the refactoring techniques, I could take pieces that I thought needed to completely rewriten and make them better while I was fixing bugs in that area. I could kill two birds with one stone.

Then I discovered Eclipse. The built-in refactoring browser captivated me. Suddenly there was a good, fool proof way to do many of the common refactorings, and automation to keep them introducing new errors. My commitment to refactoring was completed and the defect rates in the code that I was responsible for maintaining declined dramatically. I was a convert. Since that time refactoring has been a cornerstone technique in my arsenal. I no longer lived in fear and loathing of old code!

One refactoring that I have still found to be painful, despite Eclipse’s facilities, is renaming files/classes. In most software configuration management (SCM) tools, renaming files can have unfortunate unintended side effects because the identity of a file is its path. This leads to a great many developers to name a class once, and then never change its name – even if that name does not make sense and the design of the code and the classes primary purpose and responsibilities change.

Luckily for me, I am a long time Accurev user. In Accurev, files are not identified by their pathnames, but rather by a unique id. This makes it possible to quickly and easily rename files with no negative side effect. However this process was inconvenient – I would have to drop out of Eclipse, rename the file with Accurev’s tools, and then refresh my workspace. Not ideal, but it worked. That is why I was so pleased when the Accurev-Eclipse Plugin was released – it integrated the Eclipse Refactoring browser’s rename actions with Accurev’s capabilities to make the whole experience seemless. Accurev has helped me to maintain well thought out, easy to understand designs despite constant evolution to those designs.

Person Martin Fowler’s
Right click for SmartMenu shortcuts

7 comments »

Posted in AccuRev, Agile, Best Practices, Customer Guest Blogs, Tips and Tricks

Tags:

Whither branches?

September 2nd, 2008

First, for all of you English majors out there, yes, the title is a deliberate play on words. My initial inclination was to use ‘wither’ instead of ‘whither’, to imply that SCM systems that still used branches were dried up and rotting legacies of a less enlightened past. But since this post is a response to a blog I read about the limits of branches, I though ‘whither’ was more appropriate, as I am really talking about the overall disposition of branches and how they should be used.

The author of “Is branching the answer?” poses several interesting and important observations about the limitations of branching as a means to push source code through a development process. My goal in this post is to show the reader how AccuRev addresses those issues.

To begin with, the author correctly notes that branches are often used to identify stages in a development process, and that merges are then used to ‘promote’ code through these branches as the code evolves. My first comment is on private branches, which AccuRev replaces with private developer workspaces. Private workspaces are essentially full SCM environments on a developer desktop, as opposed to the copies that private branches in, say, Subversion are. The practical difference is then when you promote from an AccuRev workspace, merges are required only for those changes that are in conflict with existing changes in the parent stream. The general case is that promotes are clean and do not require merges at all.

The author then lists four problems with the approach of using branches to ‘automate sharing of code changes within a team’. For most file and branch based systems, I tend to agree with the author, so my comments are on how AccuRev addresses these issues:

1. It encodes the team’s process into the file history of code making it difficult to quickly adjust the process as team conditions change.

With standard branches, that is true. With AccuRev, streams represent how code flows through the system. File histories reflect the individual ancestry of files. Since streams can be easily reparented, there is nothing difficult about changing the process.

2. Using a merge to promote changes introduces the possibiliy of an incomplete or bad merge that hides its effect, namely that the code isn’t promoted, or worse additional code is injected that doesn’t belong.

Promotion is a fundamental operation in AccuRev. Code changes are designed to be pushed up the stream hierarchy, from developers, to QA, all the way up to release. The only required merges are those that arise naturally from conflicts, and AccuRev’s merge tools make it straightforward to assure that merges when required are performed accurately. There are absolutely no ‘merges that are simply code promotions’. Also, it is impossible in AccuRev for anything ‘incomplete’ to get promoted. AccuRev uses atomic transactions for all operations, so success or failure is binary.

3. It clouds the history of file changes with multiple merges that make it difficult to understand if changes have correctly been merged and makes it hard to understand what is in a build of the software.

See previous response. One additional comment – there is no magic bullet for merges. AccuRev helps you identify when a merge is required, but to my knowledge there are no tools that help you decide whether changes have been correctly merged from a semantic point of view. Put another way, AccuRev can tell you that lines 3 and 4 are in conflict and require a merge, but only a human being can know how to properly resolve the conflict. That’s why they pay us the big bucks! As for what is in a given build, atomic transactions in AccuRev and the TimeSafe architecture assure that the contents of a given stream are unambiguous at any point in time.

4. It makes it hard to promote code to software teams that don’t have a close common branch. For example, you want to merge a fix from one project to another but the only common branch is the main line and merging there will hit production before you want to.

Streams make it easy to isolate lines of development from each other, especially for this common case. There is no need for a ‘common branch’ in order to do this. You simply place the changes of interest on the change palette and select the destination stream.

As to the author’s final point about Telelogic Synergy and change set routing, I will just say that AccuRev change packages provide a logical grouping of changes that can be promoted, reverted and routed in a similar fashion.

Overall, this was a very interesting post and I congratulate the author on identifying and commenting on the problems with branches in release management.

3 comments »

Posted in AccuRev

Tags: