Archive for November, 2008

Pattern for Private Prototype Development

November 13th, 2008

by Dave Thomas

Your team has been coding a feature for a few days, across dozens of files, and everyone is excited with progress.  Deep into development, you find an interesting 3rd party library that may simplify your work and possibly take the team’s feature to the next level.  But it will require adding new files, moving some directories, and refactoring some critical code.  However, you’re not ready to commit your current changes because they are unfinished and will surely break everyone else.  Your changes need to be saved before prototyping in case of a rollback but also be integrated with prototype development.  And what if the new library is a bust?  If you start co-mingling the library integration with unfinished code, the potential revert process will be a complete nightmare and waste of time.

Private Versioning. With AccuRev’s private workspace, you can always commit your changes early, often, and safely without sharing amongst your peers.  But in this case you have two logical development efforts, the 2nd effort depends on the first and both need commits to preserve evolving changes.   How do you keep them cleanly separated and continue to work on both in parallel?

Stream Inheritance.  AccuRev streams have an intriguing and very powerful feature called inheritance.   Similar to how an OO subclass implicitly inherits methods from parent and grandparent classes, a child stream implicitly inherits versions of files & directories from parent and grandparent streams.   Taking advantage of inheritance, we can use streams to independently manage logical changes and cleanly maintain change dependencies without physically co-mingling files.

The Pattern.  Prototyping changes without co-mingling files can be done by simply creating a series of ‘personal’ or ‘private’ development streams (though, they are just regular dynamic streams). This pattern will create a “Feature”, “MyDevelopment”, and “MyPrototype” stream sub-hierarchy.  See Picture.

click to enlarge

click to enlarge

From your Integration stream (or equivalent), start by creating a “Feature” stream that will collect all changes from your entire team.  [See related blog, Stream-per-Task Pattern].  The majority of team members may have their workspaces directly from here.  Next, create a child stream called “MyDevelopment” that will track your personal ongoing development activity.  Finally, create a grandchild stream called “MyPrototype” to track changes that will be discarded or retained depending on the level of success.

The prototype development activity is committed, shareable, integrated, and yet cleanly segregated from both active feature and private development.

The “Feature” stream is the collection point of all in-progress development activities for the given feature from the entire team.  Developers will promote here frequently possibly kicking off an automated build with success/fail notification.   The “MyDevelopment” stream provides a collection point for all of your personal development changes.  This stream may be considered a “private” development stream simply because no other developers will likely use it – set a stream lock to be sure.  A lightweight Continuous Integration build (i.e. compilation only) may be performed on “MyDevelopment” for sanity sake or just compile and promote as a practice.  The “MyPrototype” stream is a collection point for all prototype changes. Even as new changes are promoted to “Feature” and “MyDevelopment”, the “MyPrototype” stream will automatically incorporate those changes (via inheritance) and the prototype developers will merge changes as necessary. The prototype development activity is committed, shareable, integrated, and yet cleanly segregated from both active feature and personal development. Also, by using a stream for prototype work, multiple developers can contribute and collaborate. If the prototyping work is deemed successful, the files can be promoted to “MyDevelopment”. If the prototyping efforts don’t work out – no problem – just remove the “MyPrototype” stream and re-purpose the workspaces.

The beauty of this pattern is that it isolates development activity by purpose without co-mingling physical file changes.   It lets the prototype developers go nuts and shoot from the hip while the regular feature developers (with the deadline!) work unimpeded and without fear of rampant changes — and everyone stays up-to-date.   And with no limit to stream depth, teams can perform prototyping efforts in parallel and/or perform prototyping based on existing prototypes by adding another child stream!  Furthermore, the pattern works for any size development activity or team, even for us Team-of-One developers with tons of ideas, fast fingers, and a few green screen x-terms (2 space, 80-char wrapping of course – <chuckle>).

This is a perfect example of how AccuRev empowers the developer to take control of their own development.  Creating streams is extremely easy and the stream browser provides unprecedented visibility into the entire development process.  With the right amount of security in place (Locks, ACLs, Triggers), the critical release streams (left side of the stream structure) can be locked down by the CM Admins, but the development related streams (right side) can be fair game for developers to create an environment that suits their purpose, such as prototyping.

Does anyone have a good story to tell about how this pattern (or equivalent) helped with a major refactoring effort or library upgrade?

/happy prototyping/ – dave

How We Manage Continuous Integration 2.0

November 11th, 2008

I work for a large software company, and we’ve used AccuRev to facilitate using a large scale distributed Continuous Integration model.

AccuRev makes this possible with the Stream approach to managing different codebases.  Developers run builds using the same build scripts used by the core team for production builds that ultimately are packaged and shipped out of Engineering.

These build scripts not only build and package and kit the product, they also run thousands of xUnit tests written to run fast and fail fast.  Developers that encounter failures immediately know where to fix the code to pass the tests.  We also use test driven development.

Each day, developers promote their changes to a task stream.  We use Scrum, so a task stream correlates in most cases to a Scrum team.  This team runs automated builds / tests at their task stream level and when stories are done and accepted and passing, promote the appropriate changes to the integration stream.

The integration stream is built every afternoon, and any test failures run during the build are quickly addressed by the team.  Our Continuous Integration software provides a failure email with the modifications made that day with AccuRev user names.  Developers can then go into AccuRev using the StreamBrowser and the Version Browser and determine the root cause.

Fixes are then promoted back to the integration stream, and the full nightly build in most cases runs successfully.  We fail all integration builds on test failure as we believe in Continuous Integration.

Each week our qa level stream is built and we repeat the same process. Developers handle the promotes, the central release team does not promote code for teams.  As code promotes up the hierarchy from task to integration to qa the frequency of broken builds, due to test errors or compilation, decreases.

This Multi-stage Continuous Integration approach is easy with AccuRev due to stream inheritence.  If you used a branch / merge solution you would need to staff a central team just to manage the commits to source control, and manage code that is “done”.

Use Case: Fixing the Broken Build

November 4th, 2008

by Rob Mohr, AccuRev

In one of many travels and customer visits, I came across a very cool way that AccuRev was helping to improve the way development teams do their work. To be more specific, this group was using Change Packages integrated with the JIRA Issue Tracking system to manage changes across their various product releases. They also used CruiseControl for continuous integration that would kick off nightly builds and notify the team with the results of the build.

From what they told me, the success of builds has significantly improved since they started using AccuRev because of the ability for the developers to work in their own private workspaces where they can integrate and unit test before promoting their changes for the rest of the team. Although broken builds are, for the most part, a thing of the past, they will still occur once in a while and need to be fixed ASAP.

Here is how they do it with AccuRev

The stream structure below is a simpler view of their overall software development process, but will be sufficient to show the use case.

Promoting to the Integration Stream

To start, the 4 developers below have made changes in their workspaces that will be promoted and associated to 4 different issues.

As you can see below, the integration stream (EntSoft_Client_Int) is “aware” of which issues are active in the stream. These are the new “change packages” introduced in the stream to be included in the next nightly build.

Build Fails in the Integration Stream

The next morning, the team is notified that last nights build failed via an email notification from CruiseControl. They have also scripted CruiseControl to automatically enable a time based stream called the “Temp_Fix_Build” stream and assign the appropriate transaction number to rollback the change packages from last night.

Assign the Developer to Fix the Build

One of the developers creates a workspace on the Temp_Fix_Build and “change palettes” over each change package one at a time.  This gives them the ability to mix and match change packages together to determine which one of them is the problem.

Problem Solved

After the culprit is fixed, the repaired change package(s) are promoted back into the integration stream for all to share.