Software Configuration Management

Posts Tagged ‘tags’

Pattern for Floating Labels with Streams

September 1st, 2009

I’ve encountered a number of use cases in the field where people enjoy the use of ‘floating’ labels. Implemented correctly, floating labels are a good example of convention-over-configuration usually eliminating repetitive, manual build system updates. Implemented incorrectly, they can be a traceability curse. In this writeup I’ll show two examples of how to implement a floating label pattern in AccuRev.

Deprecated Concept? If you’re used to using floating labels as ‘markers’ on a branch [in another CM system] for stable code or feature complete state, you’ll find that the AccuRev stream hierarchy and associated workflow is a much better solution – in some cases, eliminating the need for floating labels altogether. In my experience, floating labels have been used because the traditional CM system didn’t support process – so you had to roll your own – ultimately leading to lots of scripting and a solution that doesn’t scale.

Stories of Misuse. Every now and then I hear stories about how an urgent quick fix or missing doc change -needed- to be included in the “release-4.0.1″ that just got labeled in source control a few days prior. In some traditional CM systems, you have the ability to commit new file changes, un-label the old version and re-label the new version. As easy as this sounds, this anti-pattern instantly breaks traceability [do your labels maintain history?] and can quickly lead to confusion as “release 4.0.1″ has multiple meanings depending on when the label was pulled, either before or after the re-label. Example. I once asked someone why they routinely re-label individual files and their answer was – “because creating a new full label takes about 30 minutes.” Why so long? Because most CM systems label the individual files (or references) in a given configuration; the more files you have, the longer it will take to label the entire configuration. I’ve even heard stories about creating labels (and branches!) that take upwards of an hour each run. Unless you are using a time/txn-based CM system, labeling build events is required if you need to track and compare changes between builds over time. If you are waiting 30-min per label, forget about the agile “10-minute” build – you haven’t even gotten to the compile step yet!

Valid Cases. In practice there are cases where having a floating label makes sense, typically related to your build system. Most cases have to do with marking transient build configurations. For example, when doing nightly or continual integration builds, let the build script or CI server simply pull/poll a single label and let the developers or build engineer determine what content is bound to the label. This way, the build server can be “dumb” and just get whatever the developer or build engineer say it should have. This eliminates the need to configure your build system with each-and-every new build label – especially painful for those build systems requiring manual edits of config xml files – ugh.

The Pattern. Two examples are presented below showing how floating labels can be used to support continuous integration with streams.

Static Floating Label. In this example, continuous builds are performed on the single stream App_Integration. No need to change your CI configuration. New builds are kicked off whenever changes are promoted [by team leads, etc] from the child project streams.

: Static Floating Label

The CI/build tool is configured to monitor the single App_Integration stream. As the configuration changes, the name of the build stream remains unchanged – hence, the App_Integration stream is the ‘floating label’.

When doing very-frequent builds (e.g. per-promote or even nightly) this pattern works well as long as you have the discipline to immediately fix any broken configurations.

Dynamic Floating Label.

In this example, individual build configurations for App_Integration are marked with snapshots. New snapshots are created whenever changes are promoted [by team leads, etc] from child project streams, or on schedule (e.g. Nightly).

: Dynamic Floating Label

The stream Int_Good_Build is dynamically reparented to either the latest snapshot (e.g. latest build) or reparented to an alternate (e.g. prior) snapshot representing the last known good build. The ability to move this stream, on-demand and as needed, represents a dynamic ‘floating label’ pattern. Simply reparent the stream, update the attached workspace and the files on disk will mirror the current release snapshot configuration. The same pattern can be applied to production releases where internal snapshots capture detailed per-configuration changes, but the reparented floating label can represent the latest public release.

Summary. In all, I’m a big fan of creating snapshots for each build, and removing old[er] ones along the way. There are two camps on the subject, and this will be left for another blog. Thus, I like the visibility of knowning which snapshots have been created (implying a purpose), and having a ‘marker’ stream as a floating label (e.g. context) to identify the current build being tested and/or delivered to an Int or Test machine. Lastly, having recent snapshots immediately available assist with debugging why the latest build is broken – diff by snapshot. sweet.

/Happy Coding/ dave

Share/Bookmark

1 comment »

Posted in Best Practices, Patterns

Tags: AccuRev CI continuous integration floating labels floating tags labels tags

Developing in streams, releasing from streams

September 25th, 2008

By Matthew Laudato

When I read the following blog, Developing in branches, releasing from trunk about a software configuration management development process that was constrained and tortured by branches, I had an ‘aha’ moment. Actually, I had three moments. They went something like this:

1. Branches are obsolete, so why does anyone still use them?

2. People use them because they are unaware of the alternatives – specifically streams.

3. I must act quickly to save my coding comrades from any further branch related pain.

If there is a cyberspace equivalent of shouting from the rooftops, then this blog post is it. My goal is to help the author to simplify his development process, and to free him and his team from the time-wasting, error-prone, outdated branch-induced hysteria that is clearly evident in his post. OK, so maybe that last sentence was a little hysterical, but you get the point.

In the post (which I encourage you to read first), the author outlines a branch-centric process that goes something like this:

- create issues in JIRA for feature (defect, etc) work

- create a Subversion branch for each JIRA issue, named by the JIRA issue ID (eg, SUP-3452). Developers write code on these feature branches.

- create a JIRA release (for example, R4.2) and target the issues created above to this release

- create a Subversion branch (off of the ‘trunk’) with the same name ‘R4.2′.

- Write a bunch of scripts that look at the JIRA release, find all the issues for that release, look at Subversion for the branches that are named for said issues, merge these branches to the release (R4.2) branch.

- Do some testing on the R4.2 branch

- Tag the release R4.2 branch

- Release software

- Merge the R4.2 tag back into the trunk

Rinse, repeat for the next release.

For some reason, I’m reminded at this point of a character in Tom Robbins’ “Even Cowgirls Get the Blues”. The character had a little business card that he would hand to people. One one side it read, ‘I believe in everything, nothing is sacred’. On the other side it read, ‘I believe in nothing, everything is sacred’. When I think about the author’s process, one side of my brain says ‘there is nothing wrong with this process’, and almost immediately, the other side screams ‘there is everything wrong with this process’. The truth is somewhere in the middle, and it all comes down to branches.

The problem with this process is that branches get in the way. The idea of isolating features in your SCM tool is a good one. Although I don’t personally agree with the idea of dedicating an SCM construct per-feature, there are plenty of people who do. (See Stream-per-task paradigm, for example by Dave Thomas). But that’s not the point. Whether you put one feature or ten in it, a branch constrains your ability to work efficiently. Once you create a branch, it loses all contact with the outside world, most importantly, with its own history and its relationship to other branches. The author even admits jokingly that a ‘release manager’ will likely be needed to handle the merges. Ya think?

Here’s an alternative process that uses AccuRev streams:

- Create issues in JIRA as before for the desired features and fixes and assign to developers. Use AccuBridge for JIRA to sync those issues into AccuRev

- Create a stream for R4.2

- Create (or reparent existing) developer workspaces off of this stream.

- When developers finish coding, they use ‘promote to issue’ to push the code to the R4.2 stream (automatically creating a change package that associates the code changes with the JIRA issue, and automatically updating the JIRA issue with an annotation about said code changes).

At this point, the R4.2 stream contains everything that you want to release. If you decide to back out a feature, you can use the revert by change package feature to remove it from the stream. The code changes are always available in the developer’s workspace if you need to revive them in the future. Once it is time to release, you can create a snapshot of the R4.2 stream to preserve it, and then you can go about working on the next release. If R4.2 and R5.0 (The Next Big Release) are happening in parallel, you could create an R5.0 stream as a child of R4.2, so that the R5.0 stream automatically inherits all of the R4.2 changes.

The advantages are many:

- Fewer and simpler merges

- Right-sized feature isolation and management

- Inherent parallelism through code inheritence

- Fewer process steps

- Built-in automation and thus fewer opportunities for manual errors.

I could go on, but I hear the footsteps in the hallway. There are people out there who don’t want you to know about streams. “We’ve always used branches,” they’ll say. “We’re too smart to use some vendor’s solution,” others will say. I say, don’t listen to them. Fight the power. Walk right up to them and say “Why branch when you can stream?” And then go ahead and watch your team spend less time fighting with branches and merges, and more time writing features and fixes for your customers.