Posts Tagged ‘version control’

Pattern for Versioning Generated Objects

May 5th, 2009

After building your software, do you check-in your generated binary  files? How about the output from test runs? If your software runs on multiple platforms or your test runs take hours/days to execute, you may want to consider storing the output — especially if binary reproducibility is critical.

Example. Consider shipping an application to a customer who 2 years later reports a defect. Can you reproduce their build “today”? Surely you have the exact versions of source files. But are you using the exact build file? Probably. How about the original version of the compiler? Maybe. But probably not. Don’t forget that your compilers get upgraded too — their optimization algorithms or bugfixes can change the binary execution format of your application. Thus, compiling source from 2 years ago may result in an equally functioning application at the user-level, but at the byte-level, things may have changed dramatically — and at a level where runtime defects (performance/memory) rear their ugly heads.

Myth #1: Committing generated  files results in longer checkout times. No developer wants to checkout source code and wait for or be inundated with megabytes of .o, .class, .jar, .war files that they are either never used or are going to be rebuilt anyway.  The AccuRev Truth: Use include/exclude rules on streams and workspaces to control which streams have access to generated objects and who will receive them during checkout.

Myth #2: Committing binary files slows down your CM system. Traditional SCM systems combine both meta data and content resulting in slower performance over time as the number of files increase (think labeling).  The AccuRev Truth: AccuRev stores meta-data separate from file contents and uses indexes to lookup and retrieve contents.  For example, transactions are labeled not files.  Using a card catalog (index lookup) to find your books is always faster than walking the isles (linear scan).

Myth #3: Storing generated artifacts will bloat the repository. Back in the day of wild-west coding, there was little rhyme or reason for where files were saved in the source tree.  The build system would simply compile the files it found, save the generated output right next to the source file, and as long as everything linked & compiled — it worked.  But in todays complex world of multi-layer software architectures, tiered deployments, mixed technologies, and sophisticated build tools, following a convention is almost a necessity (think  ruby on rails, maven, etc). The AccuRev Truth: Organizing the top-level source tree and configuring your build tool can make it very easy to carve out source vs. binary vs. tests vs. scripts, etc.  Using include/exclude rules, end-users can decide at the stream or workspace level what parts of the file tree need to be visible.

The Pattern. In this pattern for versioning generated artifacts, I’ll show how streams can be used to store generated files only in the appropriate stage of development and prevent unwanted exposure to developers.  Two options are present that can also be used in combination.

Option #1: sub-configurations

Option #1: sub-configurations

Option #1: Store and track generated artifacts as sub-configurations isolated from the mainline.   From a baseline snapshot such as a test build or release candidate, create a new child stream to store the generated artifacts.  Then create a second snapshot that represents both source code and generated artifacts. For a single “configuration” you now have two snapshots – one for source only and a second for source + binary.  Furthermore, you can diff these two snapshots to know exactly how the binary configuration is different from the source configuration.  You might also consider storing compiler files, debugging output, test output,the compilers themselves (!), etc.

Option #2: include/exclude rules

Option #2: include/exclude rules

Option #2: Store and track generated artifacts directly in mainline but exclude them from downstream access using stream-level exclude rules.   The top-most streams that need access to both source and binary will include the majority or entire filesystem footprint in their configurations.   The first stream that does not need access to generated objects will likely be the candidate to set an exclude rule on the folder(s) that contain those files.  The exclude rule is inherited to all children and grandchildren.

When using exclude rules, it is easiest to set a single rule on a top-level ‘./build’ or ‘./generated’ folder rather than creating a rule for each sub-folder in a large source tree.  Traditionally, make based build systems would generate the compiled files in-line with the source code.  Lately, ant based build systems would package all generated artifacts in a separate sub-tree off the root.  Regardless of your build tool, it’s best to have all generated artifacts in their own tree – it makes it easier to exclude as well as safer to clean!

In practice I see both patterns in use and both have equal merit depending simply on the situation at hand.  Option #1 is commonly used when generated artifacts are not to be included in the official release.  For example, transient or secondary artifacts such as test cases, debugging output, reports, etc.  These files are not promoted up to the release stream.  Option #2 is usually used when the generated artifacts are expected to be included in the official release snapshot.  Thus, they are promoted up through the test/build/release streams.  The build system for these types of ‘uber’ configurations may have multiple release targets creating different levels of release packages such as ‘minimal’, ‘app’ , ‘app-with-tests’ and ‘full’.  That is to say, the CM system may have all possible files but you can choose what actually gets deployed.  Ultimately, storing everything in the CM system may likely be the right choice for audit and reproducibility.

/Happy Coding/

Getting the Most out of AccuRev’s Windows Explorer Integration

March 18th, 2009

As organizations become more advanced in their use of software configuration managment (SCM) they typically expand its use to include more than the software developers.  A common addition is around audibility and compliance of the entire development process which may require them to version more than just the source code, but also any requirements, project plans, design artifacts and documentation.  This enables the labeling of the big picture of who, what, where, why and how the software was developed.

With this expansion there are now more users of the SCM tools, many of whom have never used a version control solution before. A few examples of these new user types may be hardware designers, graphic designers, product managers, or the documentation team. These types of users are working with different types of files, and work in different tools than the traditional software developers and because of that may use version control tools in different ways.  These users may have historically created versions of files using naming convention such as:

ReleaseNotes_v1.doc, ReleaseNotes_v2.doc etc. all on their local computer.

Obviously this can be an error prone process and also a single point of failure if something were to happen to that user’s computer.  Since many of the files they work on are binary (.doc, .pdf, .ai, .dwg, .vsd etc.) they may want to work on these files in a serial mode so they don’t have to try to manually merge them when someone else makes changes to them in parallel.  So instead of the traditional use of AccuRev, taking advantage of our parallel development features, these users may want to work in exclusively locked workspaces or at least have exclusive locks on certain files to protect them.

In order to make this new type of user successful they need to use an interface that is familiar to them. With many source code control tools they may have to write scripts, manage configurations or interact with a command line interface to be able to get their work done. This type of interaction will likely not work well for these types of users, as many of them may not have the technical know how to work in those environments.  AccuRev can make this transition much more fluid and familiar for these users with the AccuBridge for Windows Explorer. This interface gives the user access to the AccuRev commands needed from within the familiarity of Windows Explorer as seen in exhibit 1.

AccuBridge for Windows Explorer

AccuBridge for Windows Explorer

» Read more: Getting the Most out of AccuRev’s Windows Explorer Integration

AccuRev is a 2009 Jolt Award Finalist

January 22nd, 2009

AccuRev is once again pleased to inform everyone that its flagship product, AccuRev, has been selected as a Jolt Award Finalist for Dr. Dobb’s 19th Annual Jolt Product Excellence Awards in the Change and Configuration Management category. 

For the past 18 years, the Jolt Product Excellence Awards have been presented annually to showcase products that have “jolted” the industry with their significance and made the task of creating software faster, easier, and more efficient. The awards presentation is sponsored by JOLT, the fabled soft drink quaffed by software developers for sustenance during project development marathons.

“The Jolt judges have selected these finalists from the nearly 300 qualified nominations that were submitted online. The submissions represent the many innovative tools available for every phase of the software development lifecycle,” said Amber Ankerholz, Jolt Award Project Manager. “This year’s finalist round was extremely competitive and we appreciate the effort all of the participants put into showcasing their products’ features for the judges.”

In the next round of the Jolt Award process, the judges will examine the finalists according to the standard criteria of audience suitability, productivity, innovation, quality, ROI, risk, and flexibility. They focus on products that are ahead of the curve, universally useful, rich in functionality or that were solutions to problems in their product space.

Winners are announced during the Jolt Awards ceremony that takes place on March 11 at SD West 2009 Conference & Expo at the Santa Clara Convention Center.

Build Management with AccuRev and Maven

June 25th, 2008

Build management is a long-overlooked area of software development. As is often the case with underserved areas, the open source community has several good solutions. Maven is among the most interesting and functional solutions for Java developers. I had the chance to take the AccuRev-Maven m2eclipse integration out for a test drive recently. This is a recently announced integration (read the press release here if you are interested, and see my video demo here).

The integration follows the pattern of other software configuration management (SCM) integrations with m2eclipse. A backend provider interface was written to support AccuRev, and the provider was integrated into the m2eclipse environment via the SCMHandler mechanism. I used m2eclipse in conjunction with AccuBridge for Eclipse (the AccuRev-provided Team plugin) to materialize Maven projects into a new AccuRev workspace from within Eclipse, and then did some basic Maven build and AccuRev SCM operations.

To get started, I needed to install m2eclipse from Sonatype. I used the Eclipse Update Manager to do this, by creating a new remote site in Update Manager for Sonatype (http://m2eclipse.sonatype.org/update/). I already had the AccuBridge for Eclipse plugin installed, also obtained via Update Manager (http://www.accurev.com/download/eclipseupdate/32/). After restarting Eclipse, I was ready to test.

In my test environment, I had an AccuRev stream called maven_int that I had pre-loaded with (of all things) the source and test code for the Maven integration itself. My goal was to create a new AccuRev workspace based on this stream from within Eclipse that was Maven- and AccuRev-aware. To do this, I used the Eclipse Import functionality and selected the “Checkout Maven Projects from SCM” option. Since I was using AccuRev, I chose the AccuRev provider from the drop down list box. Like other integrations with Maven, AccuRev supports a URL format for specifying where to obtain code. The URL lets you specify the AccuRev user and login credentials, as well as the exact depot and stream from which to import the code. I also chose the make workspace checkout option, which tells the plugin to create an actual AccuRev workspace in a specified location.

After pressing the Finish button on the Import wizard, m2eclipse called out to AccuRev, created a new AccuRev workspace, and created new Eclipse projects (based on whatever Maven POM files were found in the maven_int stream) containing the source code. I then associated the Eclipse projects with AccuRev using the Team Share option so that I would have full access to AccuRev functionality.

Since the projects were all Maven-based, building was done by right clicking on the POM file and selecting one of the build options. To convince myself that the AccuRev functionality worked, I did some edits to a few files, saved the changes, and then used the AccuRev promote option to version that file in the AccuRev repository.

Overall, the integration worked as I expected. The installation of the plugins was fast, it was easy to materialize Maven-based projects from AccuRev, and all of the important Maven and AccuRev functionality was available in the respective plugins. If you are an AccuRev user and want to use Maven, then the combination of m2eclipse and AccuRev integrated inside of the Eclipse environment is a practical desktop tool for managing your local builds.

 

Take Back Control: Using LDAP for SCM Authentication

May 15th, 2008

Yesterday, just for fun, I counted the number of times that I logged into a computer or website. Once to login to my PC. Once more to connect to the company network. Three times for the 3 different UNIX boxes I needed to work with. And (if you promise not to tell my boss) once to login to a popular online shopping site to cancel a book order that apparently was lost in shipping. That’s six times in a day – and a slow day at that.

All this logging in got me thinking about security, authentication and of course, software configuration management (SCM) systems. Most SCM users, unless they are in the computer security business or are otherwise paranoid, don’t think about what goes on when they type in their user name and password and press Enter. In this post, we’ll peel back the covers a bit and show how you can use LDAP as the authentication mechanism for the AccuRev SCM system.

Starting with version 4.6, AccuRev introduced the notion of a ‘custom’ authentication mechanism. If you boil it all down, there are only three things that you need to do in order to use LDAP authentication with AccuRev:

1. Tell the AccuRev server that you want to use custom authentication

2. Create users in AccuRev and in LDAP

3. Write a special AccuRev trigger that authenticates the users against an LDAP server

Let’s look at each of these in turn. First, a word of caution. As with any change to a shared production system, it is best to practice this in a safe environment. If you don’t have a spare AccuRev server laying around, you can always download the free 30 day, 5-user evaluation kit and use it to fine tune your new authentication process.

Now to the details. To tell the AccuRev server that you want to bypass the built-in authentication mechanism and use a custom method, execute the following command:

accurev authmethod custom

Next, you’ll need to create some AccuRev users. In this example, we’ll assume that you’re already using LDAP for other applications, and therefore user entries already exist in the LDAP server. A typical user in an LDAP server might look like this in LDIF format:

dn: cn=James T. Kirk,o=engineering,dc=enterprise,dc=com
objectclass: top
objectclass: person
objectclass: inetOrgPerson
objectclass: organizationalPerson
cn: James T. Kirk
sn: jtkirk
mail: jtkirk@enterprise.com
userPassword: jtkirk

Well, typical if they happen to be the captain of the most famous starship ever! But I digress.

In AccuRev, we need to decide how this user will be represented. In this example, we’ll use the LDAP ‘commonName’ attribute, which is shown above as ‘cn’, as the AccuRev username. Here’s the command we’ll use to create that user in AccuRev:

accurev mkuser “James T. Kirk”

At this point, we have a user represented in LDAP, and that same user represented in AccuRev. All we need to do is to tell the AccuRev server how to authenticate this user via LDAP. We do this via an AccuRev trigger known as the ’server_auth_trig’. Here is some sample code for a server_auth_trig that does just that:

use Net::LDAP;
use Net::LDAP::Util qw(ldap_error_text);
use Net::LDAP::Constant qw(LDAP_SUCCESS
            LDAP_CONNECT_ERROR
         );

use XML::Simple;
use strict 'vars';

# Server info for contacting LDAP
my $LDAP_HOST = "localhost" ;
my $LDAP_PORT = "389" ;

# We explicitly list the subtree DN to use when rewriting the incoming username as an LDAP Bind DN.
my $ldap_baseDN = "o=engineering,dc=enterprise,dc=com" ;

# Default attribute name for binding.  This attribute is concatenated
# with the incoming username and the ldap_baseDN above to form
# a Bind DN.
my $ldap_bind_attribute = "cn" ;

sub main
{
    my ($xmlinput);
    my ($command, $ip );
    my ($username, $password );
    my ($result);

    # populate array using XML::Simple routine, reading from stdin
    $xmlinput = XMLin('-', forcearray => 1, suppressempty => '');

    # set variables
    $command = $$xmlinput{'command'}[0];
    $ip = $$xmlinput{'ip'}[0];
    $username = $$xmlinput{'username'}[0];
    $password = $$xmlinput{'password'}[0];

    # First, establish a connection to the LDAP server
    my $LDAP = Net::LDAP->new($LDAP_HOST, port => $LDAP_PORT) ;
    unless ($LDAP) {
                print "Unable to connect to LDAP server on host $LDAP_HOST at port $LDAP_PORT.\n" ;
                return LDAP_CONNECT_ERROR;
    }

    # Now that we are connected, rewrite the username as a DN and attempt to bind to the server
    my $ldap_bindDN = $ldap_bind_attribute . "=" .$username . "," . $ldap_baseDN;
    print "Attempting to bind as: $ldap_bindDN\n" ;

    my $mesg = $LDAP->bind($ldap_bindDN, password => $password) ;

    # 'code' method contains any error code from the bind call
    # including success, so we return it to the caller
    my $return_code = $mesg->code;
    print "LDAP_BIND returned: $return_code\n";

    # Now unbind to free the connection
    $LDAP->unbind;

    # return the auth code to the AccuRev server
    exit ($return_code);

}

# run main routine
&main();

The main trick is to ‘rewrite’ the incoming user name in the form of an LDAP Distinguished Name, or DN, and then to use that DN and the incoming password to bind to the LDAP server. Binding is a fancy word for logging into the server. Typically this is done by providing a DN (to uniquely identify the user) and credentials (in this case, a password).

As we said earlier, we’re using a convention in this example that the incoming user name represents the ‘commonName’, or cn, attribute of the user’s LDAP entry. We then construct a string by concatenating the cn with a hard-coded base DN, the latter representing the subtree within the LDAP server where the users exist. The resulting DN in this example is:

cn=James T. Kirk,o=engineering,dc=enterprise,dc=com

which is represented in the example as the perl variable $ldap_bindDN. If the bind is successful, the trigger returns a 0, and the user is logged into AccuRev. If the bind fails, the trigger returns a non-zero code, and the user login is rejected.

There you have it. A few simple steps and you can use the industry standard LDAP mechanism to provide authentication for your AccuRev users. LDAP is used in all sorts of enterprises, from education to technology companies to government, and so is AccuRev, so we’re glad to provide a way for our customers to use this powerful and ubiquitous authentication mechanism.

Why Use AccuRev for Document Management?

May 8th, 2008

Let’s face it, process-centric SCM is not just about the process, code and issue tracking, it’s also about document management. Simply storing documents in a shared network space is no longer sufficient; documents related to a product release need to be versioned and tracked. We’re talking about a wide range of documents here — UML diagrams, meeting notes, functional specifications, wireframe prototypes, product plans, slide decks, planning spreadsheets, and, of course, end-user documentation from release notes to complete online help systems.

AccuRev’s stream-based, process-centric approach makes it easy to manage your product-related documents right alongside your code and issues. Add a couple of folders to your workspace — call them planning, doc, or whatever you like. Add your planning documents to the folders, and promote everything. Now your product documents are part of the stream.

  • When you create a child stream for the next release, those planning documents are included automatically because of AccuRev’s built-in inheritance. Just update the documents for the new release and promote. Product managers and others can create their own workspaces where they update those documents as needed.
  • Want to track general-purpose planning documents that aren’t related to a specific release? Create your child stream under the root, and store all those documents there. Create child streams and workspaces based on document evolution, not product releases.
  • Want to free your product managers and technical writers from using the AccuRev client interface? Now there’s the AccuBridge for Windows Explorer (for AccuRev V4.5.4 and newer), which provides AccuRev functionality from a right-click menu in Windows Explorer, and icon decorations to indicate AccuRev status.

Documents represent knowledge; they are part of the company’s history. They provide records of decisions and the context in which those decisions were made. They show the reasons behind the implementation decisions, the backtracking, the roads not taken, and the plans for the path ahead. When these important documents are not versioned and tracked, information is lost.

Globally Distributed Developers, Under a Single Roof

May 7th, 2008

One of the most common and problematic challenges that exists in today’s software development environments is how best to support a Globally Distributed Development organization. In ye olden days, you had the entire team co-located in the proverbial cube farm under a single monolithic roof. If Brad wanted you to review the code he just wrote, he would literally turn around in his chair, ask you to come in and look over his shoulder.

Times have definitely changed. Now, your team might be headquartered in Boston, separate R&D sites in California and London, with some specialized groups in Bangalore and Shanghai. But that’s not necessarily the hard part. Where it gets complicated is when all these developers are trying to work on the same source code at the same time. Mastership issues questions, latency, mismatched process across sites; communication problems, lack of project visibility; these things all lead to significant decrease in productivity, not to mention the chaos for those trying to manage the effort.

Enter AccuRev. Uniquely architected to support remote and geographically distributed development (GDD), there are several key built-in capabilities that make the challenges of the past disappear. Consider the following graphic:

  • AccuRev’s Stream Browser presents a dynamic visual representation of the software development process that is both fundamentally tied to the source code itself as well as being flexible and enforceable. At a single glance, *anyone* working on the project anywhere knows exactly what the process in place is. (See example AccuRev annotated screen shot here from the Alaska Airlines success story).
  • Geography has zero impact on your position in the process; a developer in the UK can happily use a Workspace on the same parent stream as a developer in the United States. For low bandwidth locations, AccuReplica can provide LAN-quality access without introducing the traditional mastership and latency problems of other replication solutions.
  • The private nature of the Workspace means that these remote developers can “share” code while still determining when to deliver their changes publicly.

Here’s the scenario: Developer ibergman works out of London, while jtalbott works in Boston. However, they are both part of a virtual team working on ComponentC. With AccuRev, the normal boundaries and limitations of time and space – not to mention being constrained by an inadequate SCM tool – no longer apply. Okay, I took some verbal liberties with the “time and space” bit, but it’s actually not too far from the truth.

In London at noon, ibergman wraps up a section of code she’s been working on and performs a Keep, which in AccuRev is a private check-in. The change is versioned yet stays within the confines of the Workspace, not yet ready for public consumption. But ibergman wants some validation, and asks jtalbott to review her code. Using instant messaging (IM), she pings him and catches him as he’s having his first sip of coffee, probably a colombian supremo. In other tools, how would someone review a private change that was just committed on the other side of the ocean? Would they even have private commits in the first place? In AccuRev, the moment ibergman performed that Keep in London, a visual identifier is available to anyone viewing the StreamBrowser, such as to jtalbott in Boston. So jtalbott clicks on the icon, and now has immediate access to operations like View, to see the file, and Diff, to compare against any previous revision of this file:

Did I mention that jtalbott’s access to these operations is instantaneous, as soon as ibergman performs the Keep? He can even take her version and send it into his own workspace if he finds it interesting enough to want to do additional development on.

So the previously mentioned problems of mastership, latency, visibility, communication, and most importantly Process, have all gone away. No more waiting on a 24-hour turnaround to get that Shanghai code copied into the branch. No more working in the dark not knowing exactly where your piece of code fits into the puzzle. Each team can regain the responsibility of merging in their efforts into common integration points, using a well-defined process implemented with streams.

It’s a remarkably simple and elegant solution to a complex and challenging problem. Of course, it’s still not going to solve the amusing problem of both the London and Boston developers feeling like they are superior to each other, but at least now they can actually review each other’s code real-time to help figure that one out ;-)

Continuous Integration: The temptation of the Antipattern

April 30th, 2008

You’ve started using Continuous Integration (CI), but it’s not quite meeting your needs. You’ve started out simply enough, getting a build to happen on Linux or some other common environment. You’ve gotten comfortable enough to start using it in more and more places.

You hear the whispers of potential. Maybe they come from coworkers, maybe you hear them in your head, maybe you’re hearing the antipattern. Yeah he’s talking to you, and he’s very tempting. Go ahead, add more tests, he says. Automated documentation, yeah that will work. Run some performance and benchmarking. Run some reports it can take it he says. Worse yet, the antipattern is sitting in the cube next to you. The antipattern doesn’t like to check in his code until it is ‘just right’, and then it is a big chunk of messy merged globby spaghetti code I wouldn’t eat and your product shouldn’t either.

The antipattern is tempting. Even the “good book” of CI talks about being careful of the antipattern, and then goes on to say go ahead and add more to your CI process. Very very tempting.

Beware of the temptation! Don’t fall into the hands of the antipattern. It only results in disappointment. There are people out there just waiting to show that CI isn’t successful. They are waiting for the first real failed build, delayed delivery, or time spent working on a tool instead of the product to kill the usefulness.

I should know, because I’ve given into temptation. We had a good CI setup going, our build processes were running smoother, broken builds were being caught before they became critical. And then I added benchmarking. Worked great. We were getting additional reports on leaks and bottlenecks. And we had time to attack them and slowly make them less and less of a problem. We were being lean. But then I heard the whisper. Run reports it said. Build some doc it said. So I tried. I wanted automated reports, one less thing to do manually right? But I wanted them with every CI build. Now the builds were taking 1/2 hour to an hour. Not so bad. Until code started breaking. You see, at the same time that I was tempted so were others. And it was easy. Large chunks of code were being added, destabilizing builds. Tests were failing over and over until people stopped listening to them. There was no buffer between changes and the rest of the developers. Yeah, we were able to clean up the mess using AccuRev, but the CI damage had been done. Benchmarking fell by the wayside while I tried to get reporting done. And the build took days to stabilize even while we scraped the code clean of the problem areas. And this was when the antipattern was able to leverage our misstep. Leaks got into the code. Not enough to destabilze, but enough that product performance could suffer. Instead of catching it early we caught it late, and the cost was that much higher when we fixed the problems.

That was over 2 years ago, but I’m here to repent. I had forsaken the usefulness and message of CI. The temptation was to strong for me then, but I and our product have recovered. Non-critical automation is no longer a dream to be shared with CI, but as a post operation run more infrequently. Stability and Performance of our product our paramount and CI is helping us keep them in line. I have exorcised the antipattern and remain vigilant.

Agile – The Soft Hum of Many Well-Intentioned Voices

April 23rd, 2008

If you listen closely, you can almost hear the soft hum of thousands of well-intended voices all intoning the mystical phrase “Agile Development” like a magical mantra that will make everything faster, better and appear more attractive.  This buzz word is coming from managers and their bosses, from PMs and VPs and CMs (Configuration Managers) and other folks with 2-letter title abbreviations, from developers and testers and even the customers.   “We must be Agile!”– so they say.

 

As you may have noticed, if you repeat any word or phrase long enough, it tends to lose all meaning.  Unfortunately this seems to be the case with concept of Agile Development. 

 

I once attended a meeting wherein a VP announced that we were going to do agile development “as of today.” There was a lot of cheering and a lot of smiling and a few hands were shaken.  And at the very back of the room, there were a few of us that sat there quietly trying mightily to conceal our shock/disbelief/cynicism and sheer apprehension at the thought of what was about to happen to us. 

 

You see – Agile development is more than just throwing smaller chunks of code into Production faster.  It takes planning, involvement, a solid architecture, good supporting tools – in short, A WHOLE LOT OF WORK – to make agile processes really work for you.  You can’t reap the reward without doing the work first– and if you try, all you’re going to wind up with is a great, big mess. (Not to mention a staff with their updated resumes out on DICE)

 

While this post is written a bit tongue-in-cheek, the message is serious.  If you want to be agile, make an investment in the process:

 

1)      Know your code architecture:  Having all 73,000 files in version control is not the same as KNOWING the architecture of your code. You can’t be truly agile if you don’t know the inter-dependencies of your own code. 

2)      Know your end-users wants vs. needs: Actively involve the end-users in the release scope.  This is A LOT harder than it sounds.  It takes a good relationship with the end users to separate out their desires from their actual needs, and balance the content of the releases across the two.  Building this relationship is a fundamental component of agility.

3)      Implement Tools that support Agile methods:  There is nothing agile about merging branches of code all over kingdom-come.  There is nothing agile about having to manually determine what files changed since last Friday at noon, or depending on checksum to figure it out.  Choose your tools wisely, implement them appropriately for your individual situation, and enforce the process globally across all groups, management levels and situations…and do so knowing that everything is subject to change without notice.

 

I highly recommend AccuRev to support agile development methodologies.  It provides a level of flexibility that I’ve never encountered in any other tool, while still enforcing process through an indelible history of every event, and user defined process criteria.

 

AccuRev is the ideal tool for distributed development teams, with fast remote updates, the option of full or partial updates to the development workspace, and flexible, developer-defined and controlled sharing of in-process work.

 

I’ve setup a lot of projects using a lot of different software configuration management tools, and AccuRev is by far and away, my favorite choice for a SCM tool – particularly when supporting agile processes.

 

In closing, here are some words of wisdom from an old-hat Configuration Manager:

 

1)      If they tell you, “Just load the CM tool on the development server for now.  We’ll find you a permanent server later” – DON’T fall for it.

2)      When a prospective employee describes their environment as “dynamic” just know in advance that’s a euphemism for “chaos.”

3)      There is no such thing as a “Planned Emergency.”

4)      If your manager says, “We’re implementing agile methodologies, and we’re buying ClearCase, because it’s the best,”….well, in that case, I’ll be seeing your updated resume on DICE…

 

 

 

Fran Schmidt is a veteran CM, who’s survived over a decade in the Software Configuration Management field through a combination of good humor, constant education on the newest technologies, and sheer stubbornness.

When SCM meets Web 2.0 – Cool Widget at Orbitz

April 17th, 2008

In my role as an AccuRev Systems Engineer, I have the welcome opportunity to visit many companies looking to find a better software configuration management (SCM) tool, as well as many companies who have already achieved that goal by choosing AccuRev. I run into various degrees of sophistication, from those that were previously versioning their source by creating .bak directories to those that have embraced modern capabilities and are truly stretching their SCM environment beyond anything previously achievable.

One of the coolest examples I’ve ever seen of the latter is at Orbitz. I’m sure you’re familiar with the travel giant, and they use AccuRev to manage the source code behind their heavily visited website. Their team is full of highly skilled and intelligent people and they are definitely one of the most advanced AccuRev shops I’ve had the pleasure of working with.

While visiting them recently, I was very impressed with a component of an intranet wiki they showed me. They were actually running RSS feeds from AccuRev streams! There are a number of practical applications for this kind of functionality, but first and foremost it allows you to keep teams up to date on changes being made in the codebase. So you can subscribe to various collaboration streams in your project, as well as perhaps “watching” other streams that you might have interest in. This can also provide support for a continuous code review process. Leads on a project can easily review the code that is created during the project by watching the stream that the team is working in. By doing this review over the lifetime of the project, they can more easily give feedback earlier in the process. And since the feeds are simply aggregated as they arrive, they can decide when the best time to check out the changes are.

How is this done? Well, with AccuRev’s command-line it’s actually a lot easier than you might think. Their RSS feeds are generated from a Rails app they have written for internal use. Getting the initial feed is quite easy. The user’s feed reader hits a URL that includes the stream they are watching. This URL is parsed by Rails and sent to a Rails controller that queries for the last 20 transactions in that stream:


[snip]
Parameters: {“format”=>”atom”, “action”=>”hist”, “id”=>”orbitz-api-foo-ci”, “controller”=>”accurev”}
Accurev command (1.226238): accurev ’show’ ’streams’ ‘-fix’ ‘-s’ ‘orbitz-api-foo-ci’ ‘-fx’
Accurev command (0.592468): accurev ‘hist’ ‘-s’ ‘orbitz-api-foo-ci’ ‘-t’ ‘now.20′ ‘-fx’
Accurev command (1.195955): accurev ’show’ ’streams’ ‘-fix’ ‘-s’ ‘orbitz-api-foo-ci’ ‘-fx’
[snip]

Once that information is loaded, they render an Atom feed to summarize the transaction information. Items like transaction type, user, comments, and number of files are displayed. Additionally, building a page that summarizes the transaction lets them show file status like modified, added, or removed. A tiny bit more complicated, but again can be easily accomplished with AccuRev CLI. Here’s a snippet of that processing:


[snip]
Parameters: {“stream”=>”orbitz-api-foo-ci”, “action”=>”transaction”, “id”=>”4587″, “controller”=>”accurev”}
Accurev command (0.937849): accurev ’show’ ’streams’ ‘-fix’ ‘-s’ ‘orbitz-api-foo-ci’ ‘-fx’
Accurev command (0.356075): accurev ‘hist’ ‘-s’ ‘orbitz-api-foo-ci’ ‘-t’ ‘4587.1′ ‘-fx’
Accurev command (0.603777): accurev ‘hist’ ‘-fx’ ‘-p’ ‘orbitz-api-foo’ ‘-k’ ‘defunct’ ‘/./build.xml’ ‘/./foo.txt’ ‘-fx’
Accurev command (0.333936): accurev ‘anc’ ‘-p’ ‘orbitz-api-foo’ ‘-v’ ‘orbitz-api-foo-ci/3′ ‘-1′ ‘/./foo.txt’ ‘-fx’
Accurev command (0.357128): accurev ‘anc’ ‘-p’ ‘orbitz-api-foo’ ‘-v’ ‘orbitz-api-foo-ci/44′ ‘-1′ ‘/./build.xml’ ‘-fx’
[snip]

Finally, that summary page provides the ability to view the file contents and run diffs. Here’s a sampling of how the unified diff gets generated:


[snip]
Parameters: {“eid”=>”56″, “action”=>”view_diff”, “previous_version”=>”721/45″, “controller”=>”accurev”, “virtual_version”=>”106/44″, “path”=>”/./build.xml”, “depot”=>”orbitz-api-foo”}
Accurev command (0.309664): accurev ‘cat’ ‘-p’ ‘orbitz-api-foo’ ‘-e’ ‘56′ ‘-v’ ‘106/44′
Accurev command (0.637650): accurev ‘cat’ ‘-p’ ‘orbitz-api-foo’ ‘-e’ ‘56′ ‘-v’ ‘721/45′
[snip]

Orbitz has built in a number of other optimizations and naturally this post only reveals a glance into what they’ve accomplished. But to give you an idea of the kind of volume this can support, there are anywhere from 9,000 – 10,000 requests a day for the RSS URL during the work week! Their users have given very positive feedback regarding the usefulness of this page and more enhancements are in the works, like links in from issue tracking systems.

So the next morning you come in to work, get your coffee, and fire up your Google Reader to see what’s new in the world you care about, and think about how cool it would be to automatically be able to check on all the latest code changes from your outsourced team overseas…