Continuous Integration: The temptation of the Antipattern

April 30, 2008

You’ve started using Continuous Integration (CI), but it’s not quite meeting your needs. You’ve started out simply enough, getting a build to happen on Linux or some other common environment. You’ve gotten comfortable enough to start using it in more and more places.

You hear the whispers of potential. Maybe they come from coworkers, maybe you hear them in your head, maybe you’re hearing the antipattern. Yeah he’s talking to you, and he’s very tempting. Go ahead, add more tests, he says. Automated documentation, yeah that will work. Run some performance and benchmarking. Run some reports it can take it he says. Worse yet, the antipattern is sitting in the cube next to you. The antipattern doesn’t like to check in his code until it is ‘just right’, and then it is a big chunk of messy merged globby spaghetti code I wouldn’t eat and your product shouldn’t either.

The antipattern is tempting. Even the “good book” of CI talks about being careful of the antipattern, and then goes on to say go ahead and add more to your CI process. Very very tempting.

Beware of the temptation! Don’t fall into the hands of the antipattern. It only results in disappointment. There are people out there just waiting to show that CI isn’t successful. They are waiting for the first real failed build, delayed delivery, or time spent working on a tool instead of the product to kill the usefulness.

I should know, because I’ve given into temptation. We had a good CI setup going, our build processes were running smoother, broken builds were being caught before they became critical. And then I added benchmarking. Worked great. We were getting additional reports on leaks and bottlenecks. And we had time to attack them and slowly make them less and less of a problem. We were being lean. But then I heard the whisper. Run reports it said. Build some doc it said. So I tried. I wanted automated reports, one less thing to do manually right? But I wanted them with every CI build. Now the builds were taking 1/2 hour to an hour. Not so bad. Until code started breaking. You see, at the same time that I was tempted so were others. And it was easy. Large chunks of code were being added, destabilizing builds. Tests were failing over and over until people stopped listening to them. There was no buffer between changes and the rest of the developers. Yeah, we were able to clean up the mess using AccuRev, but the CI damage had been done. Benchmarking fell by the wayside while I tried to get reporting done. And the build took days to stabilize even while we scraped the code clean of the problem areas. And this was when the antipattern was able to leverage our misstep. Leaks got into the code. Not enough to destabilze, but enough that product performance could suffer. Instead of catching it early we caught it late, and the cost was that much higher when we fixed the problems.

That was over 2 years ago, but I’m here to repent. I had forsaken the usefulness and message of CI. The temptation was to strong for me then, but I and our product have recovered. Non-critical automation is no longer a dream to be shared with CI, but as a post operation run more infrequently. Stability and Performance of our product our paramount and CI is helping us keep them in line. I have exorcised the antipattern and remain vigilant.


Forks, Feuds, and Friends - The Unix Family Version Tree

April 25, 2008

It’s Friday… so I dug up one of my all-time favorite web gems: The Unix Family Version Tree. Ever wonder when Unix started? Or the relationship between BSD and System V? Or how closely related Mac OSX is (or is not!) to Linux? Click the chart to see the full-size version.

Unix Family Tree

Here are some notable shortcuts: SystemV, Linux, BSD, HPUX, Solaris, MacOSX, GNU/Hurd, Plan 9, Atari Unix, iPhone OS, Windows OS.

Even for those new to Unix (out on the leaves of the tree!), this time-line has a wealth of interesting information showing the history and relationships of most unix varietals. It will also help explain the usability challenges of switching between divergent OS’ from navigating file-systems to loading drivers to managing hardware.

While this reference isn’t exactly about AccuRev or its software development process automation per-se, it’s a great example of how divergent software can grow over time and how tracking changes between active mainline and previous releases becomes really tricky — unless you have good tools (like AccuRev!).

/happy friday/ - dave


Retargeting Features to Future Releases

April 24, 2008

In today’s agile world, the ability to determine what goes in a release and what does not, is a mandatory requirement for your SCM system to easily support.  There are many use cases where this scenario applies, such as:

  1. Using feature streams to isolate changes
  2. Promoting ony stable changes to the next stream
  3. Creating a hierarchy based upon iteration based development

All of these scenarios are a kind of move forward process.  You decide through a promotion to merge changes in with what is going to be in the release.  But, what happens when a decision is made right before the release to remove functionality that has already been merged in and needs to be moved to the next release?  There goes the schedule, right?

This is where your SCM tool can come to the rescue, if it has streams, change packages, and drag & drop.  Of course I’m talking about AccuRev.

Here’s how you do it:

Let’s say I have 4 features/issues that have been promoted into the 4.0 release and are going through a final round of testing before being released.  Marketing then decides that the writable crosslink feature is going to be deferred until the 5.0 release.

 

Step 1 - Select the issue from the v4.0 stream and drag to the v5.0 stream to retarget the feature to the next release.

Step 2 - Revert the feature out of the v4.0 stream using the “revert by change package” opertion.  This requires a workspace to be connected directly to the stream to revert.  I just reparented one of my workspaces there.  Select the feature in the v4.0 stream and press the “revert by change package” button, select the attached workspace.

 

Step 3 - The “revert by change package” operation does not actually remove the change package from the stream.  The operation triggers a “reverse merge” so that you can pull the changes out of the files.  Once, the reverse merge has been completed, the new versions are promoted back into the stream to complete the “revert by change package” operation.

 

Step 4 - Promote the results of the reverse merge

 

The change package has been removed from the v4.0 Stream


Agile - The Soft Hum of Many Well-Intentioned Voices

April 23, 2008

If you listen closely, you can almost hear the soft hum of thousands of well-intended voices all intoning the mystical phrase “Agile Development” like a magical mantra that will make everything faster, better and appear more attractive.  This buzz word is coming from managers and their bosses, from PMs and VPs and CMs (Configuration Managers) and other folks with 2-letter title abbreviations, from developers and testers and even the customers.   “We must be Agile!”– so they say.

 

As you may have noticed, if you repeat any word or phrase long enough, it tends to lose all meaning.  Unfortunately this seems to be the case with concept of Agile Development. 

 

I once attended a meeting wherein a VP announced that we were going to do agile development “as of today.” There was a lot of cheering and a lot of smiling and a few hands were shaken.  And at the very back of the room, there were a few of us that sat there quietly trying mightily to conceal our shock/disbelief/cynicism and sheer apprehension at the thought of what was about to happen to us. 

 

You see – Agile development is more than just throwing smaller chunks of code into Production faster.  It takes planning, involvement, a solid architecture, good supporting tools – in short, A WHOLE LOT OF WORK – to make agile processes really work for you.  You can’t reap the reward without doing the work first– and if you try, all you’re going to wind up with is a great, big mess. (Not to mention a staff with their updated resumes out on DICE)

 

While this post is written a bit tongue-in-cheek, the message is serious.  If you want to be agile, make an investment in the process:

 

1)      Know your code architecture:  Having all 73,000 files in version control is not the same as KNOWING the architecture of your code. You can’t be truly agile if you don’t know the inter-dependencies of your own code. 

2)      Know your end-users wants vs. needs: Actively involve the end-users in the release scope.  This is A LOT harder than it sounds.  It takes a good relationship with the end users to separate out their desires from their actual needs, and balance the content of the releases across the two.  Building this relationship is a fundamental component of agility.

3)      Implement Tools that support Agile methods:  There is nothing agile about merging branches of code all over kingdom-come.  There is nothing agile about having to manually determine what files changed since last Friday at noon, or depending on checksum to figure it out.  Choose your tools wisely, implement them appropriately for your individual situation, and enforce the process globally across all groups, management levels and situations…and do so knowing that everything is subject to change without notice.

 

I highly recommend AccuRev to support agile development methodologies.  It provides a level of flexibility that I’ve never encountered in any other tool, while still enforcing process through an indelible history of every event, and user defined process criteria.

 

AccuRev is the ideal tool for distributed development teams, with fast remote updates, the option of full or partial updates to the development workspace, and flexible, developer-defined and controlled sharing of in-process work.

 

I’ve setup a lot of projects using a lot of different software configuration management tools, and AccuRev is by far and away, my favorite choice for a SCM tool – particularly when supporting agile processes.

 

In closing, here are some words of wisdom from an old-hat Configuration Manager:

 

1)      If they tell you, “Just load the CM tool on the development server for now.  We’ll find you a permanent server later” – DON’T fall for it.

2)      When a prospective employee describes their environment as “dynamic” just know in advance that’s a euphemism for “chaos.”

3)      There is no such thing as a “Planned Emergency.”

4)      If your manager says, “We’re implementing agile methodologies, and we’re buying ClearCase, because it’s the best,”….well, in that case, I’ll be seeing your updated resume on DICE…

 

 

 

Fran Schmidt is a veteran CM, who’s survived over a decade in the Software Configuration Management field through a combination of good humor, constant education on the newest technologies, and sheer stubbornness.


foo. bar. baz. ???

April 18, 2008

Thank goodness it’s Friday.

This week I was creating a diagram and needed a few placeholder words. foo. bar. baz. The usual suspects. But I found myself needing a few more. I’ve used baq before as a 4th but wasn’t sure how ‘formal’ it was. Oddly enough, I can’t find a current reference to it! Egads. But I digress.

Wikipedia has a nice writeup of meta-syntactic variables and mashes some old-skool verbiage into a suggested “formal” list. Here is their list:

Wikipedia: A “standard list of metasyntactic variables used in syntax examples” is: foo, bar, baz, qux, quux, corge, grault, garply, waldo, fred, plugh, xyzzy, thud.

From a distance, this list looks pretty interesting. Though, ‘foo’ probably works better than ‘thud’ in a formal presentation. In my opinion, this list breaks down. Why? Because only ‘fred’ can be typed with one hand (QWERTY) and the words are too terse! I may be biased as a left-hander, but ‘fred’ is the only word that can be typed by either hand in isolation! Otherwise, they are either too obtuse for presentation or frustratingly difficult to smoothly type… regardless of their pristine historical roots. Seriously, try typing “xyzzy”. Ugh. Would you label a diagram element as ‘plugh’? Useless, perhaps. Room for improvement? Indeed!

Do you have any suggestions for what could be used after ‘baz’?

/happy naming/ - dave