Posts Tagged ‘software development process’

Security Primer – Anatomy of the AccuRev Admin Trigger

March 27th, 2009

by Rob Mohr

Do you have the “Admin Trigger” installed and running in your AccuRev environment?  I hope so!

The “Admin Trigger” is the best way for you to restrict those non-ACE’d (AccuRev Certified Engineer) users from wreaking havoc on the process you’ve meticulously designed and implemented for your organization.  Make sure you lock it down!  It’s really simple to do!

Now, I’m not talking about taking flexibility away from your developers, they’ll need to control certain aspects of the process too.  It’s up to you where the line is drawn in the stream hierarchy and the Admin Trigger is the chalk.

The equator is commonly established on the integration stream.  Since the globe in this case is AccuRev, the line of demarcation runs north and south.  To the West (upstream from integration), ACE’rs set the rules on workflow, code promotion, stream configurations, and general access control.  To the East (downstream from integration), developers and development teams are free to make their own decisions to best support their process, products, projects, components, patches, bug fixing and development activities.

Have you read the private prototyping or stream-per-task blogs by Dave Thomas?  These are good examples of how dev teams and developers control how their activities are organized using streams while adhering to the overall enterprise software development life cycle (SDLC).

The Admin Trigger is a simple if-then-else perl script that fires on the server whenever certain commands are  executed.  Out-of-the-box, the script restricts “admin type” commands such as creating users, groups, depots, etc without needing additional customization.

 @cmdlist = qw/mkuser chref chdepot chslice lsacl addmember
                  rmmember mkgroup mkdepot mktrig rmtrig
                  setacl write_schema/;
    # is the user command in the above list?
    if (grep $_ eq $command, @cmdlist) {
    ...

The admin type commands are typically global in nature, meaning, that a single Admin group is granted permission for these commands.  Stream creation has a more granular scope allowing different groups to control their development process and stream management capabilities.

Simply list the streams in the trigger that only Admins have the ability to “manage.”  By default, other streams not listed are managed by the development teams themselves.  There are 2 sections in the trigger to set this up depending upon the commands to control.

Restricts: lock, unlock, chstream, incl, excl, incldo

    $admin_stream{"replace_with_admin_stream"} = 1;
    $admin_stream{"replace_with_admin_stream"} = 1;
    $admin_stream{"replace_with_admin_stream"} = 1;
    ...

Restricts: mkstream, mkws

    $basis_stream_deny{"replace_with_basis_stream_to_deny"} = 1;
    $basis_stream_deny{"replace_with_basis_stream_to_deny"} = 1;
    $basis_stream_deny{"replace_with_basis_stream_to_deny"} = 1;
    ...

Inside the trigger logic, each command is evaluated and will allow the operation to complete or not.

For example, the following section validates the “mkstream” command:

if ($command eq "mkstream") {
...
 # only a user listed as an administrator can create a new stream
 # based on an existing stream in the "basis_stream_deny" list
 if ( defined($basis_stream_deny{$stream2}) and `$::AccuRev ismember $principal "$admingrp"` == 0 ) {
   print TIO "Basing a new stream on existing stream '$stream2' disallowed:\n";
   print TIO "server_admin_trig: You are not in the $admingrp group.\n";
   close TIO;
   exit(1);
  }
}

There are other facilities in AccuRev to control the process and workflow too. Stream Locks grant users/groups the ability to promote to and from streams and Access Control Lists (ACLs) grant access to entire depots and subhierarchies.  Setting up these security measures combined with the Admin Trigger provide your organization with the flexible and granular security model it needs for the optimum development process.

Drop me a note and let me know the creative ways you’re using the Server Admin Trigger.

AccuRev is a 2009 Jolt Award Finalist

January 22nd, 2009

AccuRev is once again pleased to inform everyone that its flagship product, AccuRev, has been selected as a Jolt Award Finalist for Dr. Dobb’s 19th Annual Jolt Product Excellence Awards in the Change and Configuration Management category. 

For the past 18 years, the Jolt Product Excellence Awards have been presented annually to showcase products that have “jolted” the industry with their significance and made the task of creating software faster, easier, and more efficient. The awards presentation is sponsored by JOLT, the fabled soft drink quaffed by software developers for sustenance during project development marathons.

“The Jolt judges have selected these finalists from the nearly 300 qualified nominations that were submitted online. The submissions represent the many innovative tools available for every phase of the software development lifecycle,” said Amber Ankerholz, Jolt Award Project Manager. “This year’s finalist round was extremely competitive and we appreciate the effort all of the participants put into showcasing their products’ features for the judges.”

In the next round of the Jolt Award process, the judges will examine the finalists according to the standard criteria of audience suitability, productivity, innovation, quality, ROI, risk, and flexibility. They focus on products that are ahead of the curve, universally useful, rich in functionality or that were solutions to problems in their product space.

Winners are announced during the Jolt Awards ceremony that takes place on March 11 at SD West 2009 Conference & Expo at the Santa Clara Convention Center.

Use Case: Fixing the Broken Build

November 4th, 2008

by Rob Mohr, AccuRev

In one of many travels and customer visits, I came across a very cool way that AccuRev was helping to improve the way development teams do their work. To be more specific, this group was using Change Packages integrated with the JIRA Issue Tracking system to manage changes across their various product releases. They also used CruiseControl for continuous integration that would kick off nightly builds and notify the team with the results of the build.

From what they told me, the success of builds has significantly improved since they started using AccuRev because of the ability for the developers to work in their own private workspaces where they can integrate and unit test before promoting their changes for the rest of the team. Although broken builds are, for the most part, a thing of the past, they will still occur once in a while and need to be fixed ASAP.

Here is how they do it with AccuRev

The stream structure below is a simpler view of their overall software development process, but will be sufficient to show the use case.

Promoting to the Integration Stream

To start, the 4 developers below have made changes in their workspaces that will be promoted and associated to 4 different issues.

As you can see below, the integration stream (EntSoft_Client_Int) is “aware” of which issues are active in the stream. These are the new “change packages” introduced in the stream to be included in the next nightly build.

Build Fails in the Integration Stream

The next morning, the team is notified that last nights build failed via an email notification from CruiseControl. They have also scripted CruiseControl to automatically enable a time based stream called the “Temp_Fix_Build” stream and assign the appropriate transaction number to rollback the change packages from last night.

Assign the Developer to Fix the Build

One of the developers creates a workspace on the Temp_Fix_Build and “change palettes” over each change package one at a time.  This gives them the ability to mix and match change packages together to determine which one of them is the problem.

Problem Solved

After the culprit is fixed, the repaired change package(s) are promoted back into the integration stream for all to share.

How AccuRev makes releasing Beta software easy

October 27th, 2008

I work for a large global enterprise software company with over 500 users, that has been using AccuRev for over 4 years.

We are currently managing a Beta release of our main product line.  The management of the source code is easy thanks to AccuRev, and here’s why:

1) Stream inheritance is just plain awesome!

We manage hundreds of streams to make up our suite.  We are able to promote code up through task streams, to integration streams, to qa streams where we ship.  For the Beta, we have child streams qa.beta, and our build machines use these streams.

2) Snapshots rule!

Snapshots by definition are immutable.  So you know what you built, when you built it.  Our build process creates snapshots automatically, and we fetch code based on snapshot.  We have 100% confidence in knowing what went into a build, and our snapshots are named to a pattern i.e. <parent stream>.yyyymmddhhmmss.build<build number>.  That gives one a lot of info about a build.

3) Timelocks let you stabilize code, even if set the next day!

We built our Beta release on snapshot, and the next day we retroactively set time locks on the qa.beta streams based on build time, and we access locked them down.  Now we can finalize the Beta, and still allow other integration and qa work to happen.

In summary, if we were using a traditional branch/merge SCM, this would be a lot of work and require dedicated personnel.  Instead, we did this part time and the StreamBrowser makes is so easy to see what code is where and why.  There is no confusion with views or other techniques used by other vendors.

Developing in streams, releasing from streams

September 25th, 2008

By Matthew Laudato

When I read the following blog, Developing in branches, releasing from trunk about a software configuration management development process that was constrained and tortured by branches, I had an ‘aha’ moment. Actually, I had three moments. They went something like this:

1. Branches are obsolete, so why does anyone still use them?

2. People use them because they are unaware of the alternatives – specifically streams.

3. I must act quickly to save my coding comrades from any further branch related pain.

If there is a cyberspace equivalent of shouting from the rooftops, then this blog post is it. My goal is to help the author to simplify his development process, and to free him and his team from the time-wasting, error-prone, outdated branch-induced hysteria that is clearly evident in his post. OK, so maybe that last sentence was a little hysterical, but you get the point.

In the post (which I encourage you to read first), the author outlines a branch-centric process that goes something like this:

- create issues in JIRA for feature (defect, etc) work

- create a Subversion branch for each JIRA issue, named by the JIRA issue ID (eg, SUP-3452). Developers write code on these feature branches.

- create a JIRA release (for example, R4.2) and target the issues created above to this release

- create a Subversion branch (off of the ‘trunk’) with the same name ‘R4.2′.

- Write a bunch of scripts that look at the JIRA release, find all the issues for that release, look at Subversion for the branches that are named for said issues, merge these branches to the release (R4.2) branch.

- Do some testing on the R4.2 branch

- Tag the release R4.2 branch

- Release software

- Merge the R4.2 tag back into the trunk

Rinse, repeat for the next release.

For some reason, I’m reminded at this point of a character in Tom Robbins’ “Even Cowgirls Get the Blues”. The character had a little business card that he would hand to people. One one side it read, ‘I believe in everything, nothing is sacred’. On the other side it read, ‘I believe in nothing, everything is sacred’. When I think about the author’s process, one side of my brain says ‘there is nothing wrong with this process’, and almost immediately, the other side screams ‘there is everything wrong with this process’. The truth is somewhere in the middle, and it all comes down to branches.

The problem with this process is that branches get in the way. The idea of isolating features in your SCM tool is a good one. Although I don’t personally agree with the idea of dedicating an SCM construct per-feature, there are plenty of people who do. (See Stream-per-task paradigm, for example by Dave Thomas). But that’s not the point. Whether you put one feature or ten in it, a branch constrains your ability to work efficiently. Once you create a branch, it loses all contact with the outside world, most importantly, with its own history and its relationship to other branches. The author even admits jokingly that a ‘release manager’ will likely be needed to handle the merges. Ya think?

Here’s an alternative process that uses AccuRev streams:

- Create issues in JIRA as before for the desired features and fixes and assign to developers. Use AccuBridge for JIRA to sync those issues into AccuRev

- Create a stream for R4.2

- Create (or reparent existing) developer workspaces off of this stream.

- When developers finish coding, they use ‘promote to issue’ to push the code to the R4.2 stream (automatically creating a change package that associates the code changes with the JIRA issue, and automatically updating the JIRA issue with an annotation about said code changes).

At this point, the R4.2 stream contains everything that you want to release. If you decide to back out a feature, you can use the revert by change package feature to remove it from the stream. The code changes are always available in the developer’s workspace if you need to revive them in the future. Once it is time to release, you can create a snapshot of the R4.2 stream to preserve it, and then you can go about working on the next release. If R4.2 and R5.0 (The Next Big Release) are happening in parallel, you could create an R5.0 stream as a child of R4.2, so that the R5.0 stream automatically inherits all of the R4.2 changes.

The advantages are many:

- Fewer and simpler merges

- Right-sized feature isolation and management

- Inherent parallelism through code inheritence

- Fewer process steps

- Built-in automation and thus fewer opportunities for manual errors.

I could go on, but I hear the footsteps in the hallway. There are people out there who don’t want you to know about streams. “We’ve always used branches,” they’ll say. “We’re too smart to use some vendor’s solution,” others will say. I say, don’t listen to them. Fight the power. Walk right up to them and say “Why branch when you can stream?” And then go ahead and watch your team spend less time fighting with branches and merges, and more time writing features and fixes for your customers.

Whither branches?

September 2nd, 2008

First, for all of you English majors out there, yes, the title is a deliberate play on words. My initial inclination was to use ‘wither’ instead of ‘whither’, to imply that SCM systems that still used branches were dried up and rotting legacies of a less enlightened past. But since this post is a response to a blog I read about the limits of branches, I though ‘whither’ was more appropriate, as I am really talking about the overall disposition of branches and how they should be used.

The author of “Is branching the answer?” poses several interesting and important observations about the limitations of branching as a means to push source code through a development process. My goal in this post is to show the reader how AccuRev addresses those issues.

To begin with, the author correctly notes that branches are often used to identify stages in a development process, and that merges are then used to ‘promote’ code through these branches as the code evolves. My first comment is on private branches, which AccuRev replaces with private developer workspaces. Private workspaces are essentially full SCM environments on a developer desktop, as opposed to the copies that private branches in, say, Subversion are. The practical difference is then when you promote from an AccuRev workspace, merges are required only for those changes that are in conflict with existing changes in the parent stream. The general case is that promotes are clean and do not require merges at all.

The author then lists four problems with the approach of using branches to ‘automate sharing of code changes within a team’. For most file and branch based systems, I tend to agree with the author, so my comments are on how AccuRev addresses these issues:

1. It encodes the team’s process into the file history of code making it difficult to quickly adjust the process as team conditions change.

With standard branches, that is true. With AccuRev, streams represent how code flows through the system. File histories reflect the individual ancestry of files. Since streams can be easily reparented, there is nothing difficult about changing the process.

2. Using a merge to promote changes introduces the possibiliy of an incomplete or bad merge that hides its effect, namely that the code isn’t promoted, or worse additional code is injected that doesn’t belong.

Promotion is a fundamental operation in AccuRev. Code changes are designed to be pushed up the stream hierarchy, from developers, to QA, all the way up to release. The only required merges are those that arise naturally from conflicts, and AccuRev’s merge tools make it straightforward to assure that merges when required are performed accurately. There are absolutely no ‘merges that are simply code promotions’. Also, it is impossible in AccuRev for anything ‘incomplete’ to get promoted. AccuRev uses atomic transactions for all operations, so success or failure is binary.

3. It clouds the history of file changes with multiple merges that make it difficult to understand if changes have correctly been merged and makes it hard to understand what is in a build of the software.

See previous response. One additional comment – there is no magic bullet for merges. AccuRev helps you identify when a merge is required, but to my knowledge there are no tools that help you decide whether changes have been correctly merged from a semantic point of view. Put another way, AccuRev can tell you that lines 3 and 4 are in conflict and require a merge, but only a human being can know how to properly resolve the conflict. That’s why they pay us the big bucks! As for what is in a given build, atomic transactions in AccuRev and the TimeSafe architecture assure that the contents of a given stream are unambiguous at any point in time.

4. It makes it hard to promote code to software teams that don’t have a close common branch. For example, you want to merge a fix from one project to another but the only common branch is the main line and merging there will hit production before you want to.

Streams make it easy to isolate lines of development from each other, especially for this common case. There is no need for a ‘common branch’ in order to do this. You simply place the changes of interest on the change palette and select the destination stream.

As to the author’s final point about Telelogic Synergy and change set routing, I will just say that AccuRev change packages provide a logical grouping of changes that can be promoted, reverted and routed in a similar fashion.

Overall, this was a very interesting post and I congratulate the author on identifying and commenting on the problems with branches in release management.

The Quest for Successful Parallel Development

May 1st, 2008

Read my earlier post: Fortune 500 Software Company Answers Your Questions on Using AccuRev

Up until fairly recently I had never worked anywhere that was able to do parallel development without it becoming a nightmare. Individual contributor or manager, we just seemed unable to find that secret magical incantation that would lead to a truly successful parallel development cycle.

Every time the team had a different plan, and the project’s champion would trumpet why this time ‘things would be different’. But in the end, we always failed in one of two areas (sometimes both):

  • The quality of the software suffered dramatically
  • We lost control of the schedule and we were forced to ship late

I actually worked on one project where the resulting release was so bad that a few weeks after release we had to pull the product off the shelves and work frantically to repair the damage.

Then a few years ago my team was looking for a new software configuration management tool to use. We started out wanting ClearCase, but after much deliberation, decided on Accurev. At first, we didn’t realize the power of the tool now in our control. But after a release or two, we started to realize what we had and adjusted our process accordingly and began to achieve very successful, high quality, and on time releases done in parallel with one another. We had found the holy grail of project management!

We started to understand not just what the best practices were with Accurev, but to understand what we were getting by following them. We made other changes to our processes that helped as well, but I give a lot of the credit to the tool-set for helping us in the right direction.

We started out performing a round-table series on our past projects, looking for common themes, and then choosing the top themes and seeing how we could use Accurev to solve them, and here are some of the things we found:

Problem: Often parallel projects require extensive code changes to the same files, leading to a need for developers to communicate what is being done to which files and when. Often this communication is not handled well.

Solution: Accurev’s work in progress and annotate commands, as well as the version browser allow developers to quickly view who else is making changes to the files that they care about, and what is being done to them. Now developers can pro-actively gather the information they need instead of depending on someone else remembering to tell them.

Problem: Often projects have late phase merge points which are both complicated and error prone. In addition since merging is generally done in the mainline, the entire release can be held when any two projects try to merge together.

Solution: By allowing stream definitions to be dynamic, Accurev allows developers to create common merge streams ad hoc at no cost, and to join the projects without effecting any other users. It also provides excellent trivial merge capabilities that save the users much of the error prone work, calling out only the non-obvious decision points which greatly simplifies the merge task.

Problem: Often bug fixes get made to one ongoing project, but developers forget to make the same fixes to other active lines of development.

Solution: Using change packages users can quickly make changes, move them to all appropriate locations, and quickly verify that the tasks were completed successfully. Using the stream inheritance feature minimizes the number of lines where a developer must make a fix for his coworkers to take advantage of it. This greatly lightens the load on developers and cuts down on the chance that something will be forgotten.

Problem: Ensuring 100% reproducibility of a build from pristine sources is a huge challenge. Changes to the build system environment, tools and code changes can all effect your ability to make a minor fix to released software.

Solution: By keeping your complete build system (environment, tools, sources) all in Accurev, and by using a snapshot you are able to guarantee 100% that what you try to build today is the same as what your built the day the snapshot was taken. Unlike a label, a snapshot is forever! This enables you to reduce the regression test matrix and focus on only the changes that developers make to the released software.

While I am sure that other SCM tools can offer similar results if you know where you are starting and know where you want to go. But what I marvel at is that using Accurev really lead us to the cup so that we could drink from it and enjoy the benefits of successful parallel development.

Have you seen the AccuRev 2-minute demo?

Always Act Like This is The Release

December 20th, 2007

by Damon Poole

I think of this practice as a major guiding principle of software development. If every practice is done as though it is part of the final act of releasing the product, then you will automatically have fewer practices. Fewer practices means less chance of problems falling through the cracks until the last minute and a higher level of maturity for things that have been done multiple times.

For instance, instead of having one process for developers to build during development and another build process for release, use exactly the same one. That way, when it comes time to release, the path is well worn and the chances are better that you will be able to completely reproduce the release if and when the time comes.

Stream Refactoring with Pass-throughs

September 20th, 2007

One of the greatest stories ever told is your software configuration management or CM process through the lense of a StreamBrowser. No other tool gives you the power to both visualize and manipulate your software development process in real-time. Over time, you’ll notice your stream hierarchy evolving into an elaborate tree effectively telling the story.

There are a number of techniques to help navigate the tree such as the GUI zoom feature and good naming conventions. But the unsung hero of stream organization is the pass-through stream. You probably already know about other types of streams such as dynamic, workspace, snapshot, and timebasis. Just like its siblings, the pass-through stream is also a configuration of source code but comes with a unique feature — all elements promoted to it are are automatically promoted to the pass-through’s parent stream, hence the name.

So what exactly are the benefits?

  • Group logically related child streams for organizational purposes.
  • A handle to reparent groups of streams without requiring an unnecessary promotion (to it).
  • Set include/exclude rules to be applied to all children streams without the unecessary promote step.
  • As an advanced technique, you can create a reftree on a pass-through to do software builds wherever the passthrough is currently parented — say last weeks build or last nights build — offering easy, visual, manual control over which configuration is built.

Lets put this knowledge to use. In the following diagram, you’ll see how a relatively sophisticated process model is cleanly organized Refactoring Streams with Pass-throughswith pass-through streams. There are a few pattern motifs used for organizing active development, nightly builds, system test builds, official release codelines, and customer one-offs. Notice how collapsing them all (left image) leaves the high-level process-centric streams nice and clean looking. Furthermore, notice how the names of the pass-through streams help tell the story of what activity is happening where.

Using pass-through streams is a passive refactoring technique that does not interfere with inheritance and can be introduced safely without disrupting development activity. In my experience, the more organized your stream structure, the happier the end users.

/happy refactoring/ – dave