Archive for the ‘Patterns’ category

Pattern for Floating Labels with Streams

September 1st, 2009

I’ve encountered a number of use cases in the field where people enjoy the use of  ‘floating’ labels.  Implemented correctly, floating labels are a good example of convention-over-configuration  usually eliminating repetitive, manual build system updates.   Implemented incorrectly, they can be a traceability curse.   In this writeup I’ll show two examples of how to implement a floating label pattern in AccuRev.

Deprecated Concept? If you’re used to using floating labels as ‘markers’ on a branch [in another CM system] for stable code or feature complete state, you’ll find that the AccuRev stream hierarchy and associated workflow is a much better solution – in some cases, eliminating the need for floating labels altogether.  In my experience, floating labels have been used because the traditional CM system didn’t support process – so you had to roll your own – ultimately leading to lots of scripting and a solution that doesn’t scale.

Stories of Misuse. Every now and then I hear stories about how an urgent quick fix or missing doc change -needed- to be included in the “release-4.0.1″ that just got labeled in source control a few days prior. In some traditional CM systems, you have the ability to commit new file changes, un-label the old version and re-label the new version. As easy as this sounds, this anti-pattern instantly breaks traceability [do your labels maintain history?] and can quickly lead to confusion as “release 4.0.1″ has multiple meanings depending on when the label was pulled, either before or after the re-label. Example. I once asked someone why they routinely re-label individual files and their answer was – “because creating a new full label takes about 30 minutes.”  Why so long? Because most CM systems label the individual files (or references) in a given configuration; the more files you have, the longer it will take to label the entire configuration.  I’ve even heard stories about creating labels (and branches!) that take upwards of an hour each run.  Unless you are using a time/txn-based CM system, labeling build events is required if you need to track and compare changes between builds over time.  If you are waiting 30-min per label, forget about the agile “10-minute” build – you haven’t even gotten to the compile step yet!

Valid Cases. In practice there are  cases where having a floating label makes sense, typically related to your build system.  Most cases have to do with marking transient build configurations.  For example, when doing nightly or continual integration builds, let the build script or CI server simply pull/poll a single label and let the developers or build engineer determine what content is bound to the label. This way, the build server can be “dumb” and just get whatever the developer or build engineer say it should have.  This eliminates the need to configure your build system with each-and-every new build label – especially painful for those build systems requiring manual edits of config xml files – ugh.

The Pattern.   Two examples are presented below showing how floating labels can be used to support continuous integration with streams.

Static Floating Label.  In this example, continuous builds are performed on the single stream App_Integration.  No need to change your CI configuration. New builds are kicked off whenever changes are  promoted [by team leads, etc] from the child project streams.

Static Floating Label
Static Floating Label

The CI/build tool is configured to monitor the single App_Integration stream.  As the configuration changes, the name of the build stream remains unchanged – hence, the App_Integration stream is the ‘floating label’.

When doing very-frequent builds (e.g. per-promote or even nightly) this pattern works well as long as you have the discipline to immediately fix any broken configurations.

Dynamic Floating Label.

In this example, individual build configurations for App_Integration are marked with snapshots.  New snapshots are created whenever changes are promoted [by team leads, etc] from child project  streams, or on schedule (e.g. Nightly).

Dynamic Floating Label
Dynamic Floating Label

The stream Int_Good_Build is dynamically reparented to either the latest snapshot (e.g. latest build) or reparented to an alternate (e.g. prior) snapshot representing the last known good build.  The ability to move this stream, on-demand and as needed, represents a dynamic ‘floating label’ pattern.  Simply reparent the stream, update the attached workspace and the files on disk will mirror the current release snapshot configuration.  The same pattern can be applied to production releases where internal snapshots capture detailed per-configuration changes, but the reparented floating label can represent the latest public release.

Summary. In all, I’m a big fan of creating snapshots for each build, and removing old[er] ones along the way.  There are two camps on the subject, and this will be left for another blog.  Thus, I like the visibility of knowning which snapshots have been created (implying a purpose), and having a ‘marker’ stream as a floating label (e.g. context) to identify the current build being tested and/or delivered to an Int or Test machine.   Lastly, having recent snapshots immediately available assist with debugging why the latest build is broken – diff by snapshot.  sweet.

/Happy Coding/ dave

1 comment »

Posted in Best Practices, Patterns

Tags:

Pattern for Versioning Generated Objects

May 5th, 2009

After building your software, do you check-in your generated binary  files? How about the output from test runs? If your software runs on multiple platforms or your test runs take hours/days to execute, you may want to consider storing the output — especially if binary reproducibility is critical.

Example. Consider shipping an application to a customer who 2 years later reports a defect. Can you reproduce their build “today”? Surely you have the exact versions of source files. But are you using the exact build file? Probably. How about the original version of the compiler? Maybe. But probably not. Don’t forget that your compilers get upgraded too — their optimization algorithms or bugfixes can change the binary execution format of your application. Thus, compiling source from 2 years ago may result in an equally functioning application at the user-level, but at the byte-level, things may have changed dramatically — and at a level where runtime defects (performance/memory) rear their ugly heads.

Myth #1: Committing generated  files results in longer checkout times. No developer wants to checkout source code and wait for or be inundated with megabytes of .o, .class, .jar, .war files that they are either never used or are going to be rebuilt anyway.  The AccuRev Truth: Use include/exclude rules on streams and workspaces to control which streams have access to generated objects and who will receive them during checkout.

Myth #2: Committing binary files slows down your CM system. Traditional SCM systems combine both meta data and content resulting in slower performance over time as the number of files increase (think labeling).  The AccuRev Truth: AccuRev stores meta-data separate from file contents and uses indexes to lookup and retrieve contents.  For example, transactions are labeled not files.  Using a card catalog (index lookup) to find your books is always faster than walking the isles (linear scan).

Myth #3: Storing generated artifacts will bloat the repository. Back in the day of wild-west coding, there was little rhyme or reason for where files were saved in the source tree.  The build system would simply compile the files it found, save the generated output right next to the source file, and as long as everything linked & compiled — it worked.  But in todays complex world of multi-layer software architectures, tiered deployments, mixed technologies, and sophisticated build tools, following a convention is almost a necessity (think  ruby on rails, maven, etc). The AccuRev Truth: Organizing the top-level source tree and configuring your build tool can make it very easy to carve out source vs. binary vs. tests vs. scripts, etc.  Using include/exclude rules, end-users can decide at the stream or workspace level what parts of the file tree need to be visible.

The Pattern. In this pattern for versioning generated artifacts, I’ll show how streams can be used to store generated files only in the appropriate stage of development and prevent unwanted exposure to developers.  Two options are present that can also be used in combination.

Option #1: sub-configurations

Option #1: sub-configurations

Option #1: Store and track generated artifacts as sub-configurations isolated from the mainline.   From a baseline snapshot such as a test build or release candidate, create a new child stream to store the generated artifacts.  Then create a second snapshot that represents both source code and generated artifacts. For a single “configuration” you now have two snapshots – one for source only and a second for source + binary.  Furthermore, you can diff these two snapshots to know exactly how the binary configuration is different from the source configuration.  You might also consider storing compiler files, debugging output, test output,the compilers themselves (!), etc.

Option #2: include/exclude rules

Option #2: include/exclude rules

Option #2: Store and track generated artifacts directly in mainline but exclude them from downstream access using stream-level exclude rules.   The top-most streams that need access to both source and binary will include the majority or entire filesystem footprint in their configurations.   The first stream that does not need access to generated objects will likely be the candidate to set an exclude rule on the folder(s) that contain those files.  The exclude rule is inherited to all children and grandchildren.

When using exclude rules, it is easiest to set a single rule on a top-level ‘./build’ or ‘./generated’ folder rather than creating a rule for each sub-folder in a large source tree.  Traditionally, make based build systems would generate the compiled files in-line with the source code.  Lately, ant based build systems would package all generated artifacts in a separate sub-tree off the root.  Regardless of your build tool, it’s best to have all generated artifacts in their own tree – it makes it easier to exclude as well as safer to clean!

In practice I see both patterns in use and both have equal merit depending simply on the situation at hand.  Option #1 is commonly used when generated artifacts are not to be included in the official release.  For example, transient or secondary artifacts such as test cases, debugging output, reports, etc.  These files are not promoted up to the release stream.  Option #2 is usually used when the generated artifacts are expected to be included in the official release snapshot.  Thus, they are promoted up through the test/build/release streams.  The build system for these types of ‘uber’ configurations may have multiple release targets creating different levels of release packages such as ‘minimal’, ‘app’ , ‘app-with-tests’ and ‘full’.  That is to say, the CM system may have all possible files but you can choose what actually gets deployed.  Ultimately, storing everything in the CM system may likely be the right choice for audit and reproducibility.

/Happy Coding/

No comments »

Posted in Best Practices, Patterns

Tags:

Pattern for Private Prototype Development

November 13th, 2008

by Dave Thomas

Your team has been coding a feature for a few days, across dozens of files, and everyone is excited with progress.  Deep into development, you find an interesting 3rd party library that may simplify your work and possibly take the team’s feature to the next level.  But it will require adding new files, moving some directories, and refactoring some critical code.  However, you’re not ready to commit your current changes because they are unfinished and will surely break everyone else.  Your changes need to be saved before prototyping in case of a rollback but also be integrated with prototype development.  And what if the new library is a bust?  If you start co-mingling the library integration with unfinished code, the potential revert process will be a complete nightmare and waste of time.

Private Versioning. With AccuRev’s private workspace, you can always commit your changes early, often, and safely without sharing amongst your peers.  But in this case you have two logical development efforts, the 2nd effort depends on the first and both need commits to preserve evolving changes.   How do you keep them cleanly separated and continue to work on both in parallel?

Stream Inheritance.  AccuRev streams have an intriguing and very powerful feature called inheritance.   Similar to how an OO subclass implicitly inherits methods from parent and grandparent classes, a child stream implicitly inherits versions of files & directories from parent and grandparent streams.   Taking advantage of inheritance, we can use streams to independently manage logical changes and cleanly maintain change dependencies without physically co-mingling files.

The Pattern.  Prototyping changes without co-mingling files can be done by simply creating a series of ‘personal’ or ‘private’ development streams (though, they are just regular dynamic streams). This pattern will create a “Feature”, “MyDevelopment”, and “MyPrototype” stream sub-hierarchy.  See Picture.

click to enlarge

click to enlarge

From your Integration stream (or equivalent), start by creating a “Feature” stream that will collect all changes from your entire team.  [See related blog, Stream-per-Task Pattern].  The majority of team members may have their workspaces directly from here.  Next, create a child stream called “MyDevelopment” that will track your personal ongoing development activity.  Finally, create a grandchild stream called “MyPrototype” to track changes that will be discarded or retained depending on the level of success.

The prototype development activity is committed, shareable, integrated, and yet cleanly segregated from both active feature and private development.

The “Feature” stream is the collection point of all in-progress development activities for the given feature from the entire team.  Developers will promote here frequently possibly kicking off an automated build with success/fail notification.   The “MyDevelopment” stream provides a collection point for all of your personal development changes.  This stream may be considered a “private” development stream simply because no other developers will likely use it – set a stream lock to be sure.  A lightweight Continuous Integration build (i.e. compilation only) may be performed on “MyDevelopment” for sanity sake or just compile and promote as a practice.  The “MyPrototype” stream is a collection point for all prototype changes. Even as new changes are promoted to “Feature” and “MyDevelopment”, the “MyPrototype” stream will automatically incorporate those changes (via inheritance) and the prototype developers will merge changes as necessary. The prototype development activity is committed, shareable, integrated, and yet cleanly segregated from both active feature and personal development. Also, by using a stream for prototype work, multiple developers can contribute and collaborate. If the prototyping work is deemed successful, the files can be promoted to “MyDevelopment”. If the prototyping efforts don’t work out – no problem – just remove the “MyPrototype” stream and re-purpose the workspaces.

The beauty of this pattern is that it isolates development activity by purpose without co-mingling physical file changes.   It lets the prototype developers go nuts and shoot from the hip while the regular feature developers (with the deadline!) work unimpeded and without fear of rampant changes — and everyone stays up-to-date.   And with no limit to stream depth, teams can perform prototyping efforts in parallel and/or perform prototyping based on existing prototypes by adding another child stream!  Furthermore, the pattern works for any size development activity or team, even for us Team-of-One developers with tons of ideas, fast fingers, and a few green screen x-terms (2 space, 80-char wrapping of course – <chuckle>).

This is a perfect example of how AccuRev empowers the developer to take control of their own development.  Creating streams is extremely easy and the stream browser provides unprecedented visibility into the entire development process.  With the right amount of security in place (Locks, ACLs, Triggers), the critical release streams (left side of the stream structure) can be locked down by the CM Admins, but the development related streams (right side) can be fair game for developers to create an environment that suits their purpose, such as prototyping.

Does anyone have a good story to tell about how this pattern (or equivalent) helped with a major refactoring effort or library upgrade?

/happy prototyping/ – dave

2 comments »

Posted in AccuRev, Patterns

Tags:

Pattern for Stable Development

March 27th, 2008

It’s Wednesday morning. 9:23am. With coffee in hand you execute a simple version control command: update.

Hey, blame yourself! You should have known better than to blindly accept all changes from every developer who quickly committed code the night before just minutes before darting out. What did you expect to happen? For teams that work on a single branch, broken updates can be quite common. For those that branch for isolation, their updates are comparably more stable but they are required to (painfully) merge more often to stay in sync.

Ok, lets get real. On some mornings, performing an update can yield minimal-hassle results and we all know that staying up-to-date is always a good thing. But often when you least expect it, often when it matters the most, often when time is critical, the simple execution of an update results in sheer and utter frustration. You know. The code doesn’t even compile. Forget passing unit tests. Forget working at runtime (these are the worst to deal with!). The last thing you want to do is spend the entire morning scrubbing logs, reverting out changes (complete WOT!), or walking the halls finding the culprit only to see that everyone you need is in a meeting in some cleverly named conference room.

But you had to! You know you need to update frequently in order to stay current with your peers. But back in your mind, you’re worried that their changes are going to negatively affect your changes. Nevertheless, all you can do is…. update. But what if you had control over the situation? In this pattern for stable development, I’ll describe an approach in AccuRev that provides incremental updates based on predefined good builds and minimizes merging (due to implicit merge tracking).

The Pattern. AccuRev’s stream-based architecture with built-in inheritance and time-safe snapshots can entirely eliminate the traditional problems of shared development (with branches). The first thing to acknowledge is that you need to work off of a stable codeline. period. AccuRev Pattern - Stable DevelopmentI know, I know. Some of you reading this think you need to always be working against bleeding edge in order to move as quickly as possible. Fine. Then you accept the top-half of this blog post and are willing to deal with the pain/time/sleep/sanity lost in the ether. For the rest of us who don’t live at the office, we need to work off of a stable codeline. With AccuRev, this means working from a snapshot (or time-based stream). In a previous blog, I wrote about a pattern for continuous integration (CI) where snapshots are created for each frequent “good build.” Extending this CI pattern, lets use each good build snapshot as the baseline for active development. The idea is simple: Reparent your workspace or project stream to subsequent “good build” snapshots and then update. The update will bring down (delta!) changes to your workspace since the last good build. In this way, you knowingly accept changes that are guaranteed to have passed some level of test criteria. In the meanwhile, changes in the mainline Integration stream can churn-n-burn with various levels of volatility without affecting your active development environments — you’re saved by the snapshot! After days or weeks of hopping build snapshots, when it’s time to deliver your changes to the rest of the world, simply reparent to mainline Integration, perform any last minute merges, test, then promote! For an extra level of stability, consider only working off of certified builds – snapshots that have passed rigorous tests.

Wait. Who does the reparenting? Good question. I say let the developers or team leads manage where and when project streams get promoted. Reparenting is easy and reversible! On the other hand, maybe your CM or Release Engineer takes on responsibility for managing builds and reparenting projects on particular days or by request. You can use stream locks to easily control who can reparent, by user or group!

What about setting a time rule on a stream? As you know, snapshots represent configurations at specific points in time. You may also know that you can optionally set a time rule on a dynamic stream to control (i.e. throttle) when changes are inherited from the parent streams. AccuRev - Setting a Time Rule on Dynamic Streams[StreamBrowser: Rclick Stream --> Change Stream --> set time]. Guess what! The effect of reparenting to snapshots can equally be obtained by simply setting a time rule on your parent stream! This works best if you have a stream-per-task paradigm so the time rule only affects your (team’s) active development. The idea is simple – anytime you want more changes, simply (re)set the time forward say an hour or day or week and any newer changes will be available to downstream streams and workspaces. Personally, I like setting time rules only for temporary use cases such as for quick testing or to wait a day for new changes. If I’m doing development that will take weeks to complete, I’d rather reparent to a snapshot because it lets me physically ’see’ where I’m working. But both options result in the same effect — controlled updates.

This pattern is ideal for letting teams stay up-to-date by reparenting to stable baselines and only updating changes that are known to have passed some level of testing. Ultimately, with clean(er) updates, the results should be increased productivity, increased quality, and happier developers! Happy developers == good.

/happy coding/ – dave

1 comment »

Posted in AccuRev, Patterns

Tags:

Pattern for Continuous Builds

March 5th, 2008

Continuous integration (CI) is all the rage these days because merging, building, and testing (shared) configurations early-and-often is a good thing. Actually, it’s a great thing! After all, finding problems sooner rather than later benefits everyone. For some, CI means simply testing compilation. (Phew… it works. Ship it! haha). For those investing time in a full test harness, CI may mean frequently executing a suite of tests at various levels (unit, functional, system) to validate functionality and identify regressions. I’ve even seen other levels of CI to include lab testing, flight testing, or even customer acceptance testing for even the smallest of changes. Regardless of how you ‘do’ CI, I’ll show how I use AccuRev for continuous integration. [Keep in mind that this is one interpretation of the subject matter]

The Pattern. The stream-based nature of AccuRev makes it very natural to define separate areas for development, integration, testing, and release. Managing CI Builds with AccuRevAs seen in my example stream structure, I have an Integration stream as the first point of merging between individual project streams. This Integration stream is a great place to hook up a CI tool [Cruise Control, CC.NET, FinalBuilder, QuickBuild] and perform nightly or per-promote builds. I prefer to create a snapshot before doing the build mainly because snapshot creation is atomic and their immutable configurations guarantee reproducibility. After creating the snapshot, I will pull the build from the snapshot name. You could build from the Integration stream directly (similar to the concept of a moving label), but creating snapshots makes it easy to visually identify with the build process and compare good builds from bad builds with simple stream diffs.  [Note: integrating any of the above mentioned CI tools is as simple as telling the build tool to pull code from a stream (by name) and then configure the build tool to execute at some frequency and notify people of the build status]

What about all those snapshots? At first, you may think, “Isn’t this going to create a gazillion snapshots? Won’t that take up a ton of (disk) space and totally clutter the stream browser view?” Well… No.

  • Snapshots are cheap. Snapshots are extremely cheap server-side entities consuming ~100bytes regardless of the number of elements they label… so go nuts! Snapshots mark transaction numbers, not elements! I say, always do what you need to solve important problems and answer tough questions even if that means creating a gazillion snapshots; just be sure to organize them.
  • Clean up as you go. Your CI build script (build.xml, Makefile, or doBuild.sh) can easily be instructed to remove a snapshot for every snapshot created. I’d recommend keeping around enough snapshots (say 3 to 10) to do valuable work such as comparing builds or serving as temporary baselines for developers who want to reparent. As you can see in the stream structure, AccuRev stores both active and inactive snapshots and it is easy to reactivate any snapshot if necessary (I’ve enabled the stream browser to show both; lower left corner option).
  • Group snapshots. I prefer to tuck logically related sets of snapshots behind a locked pass-through stream. The pass-through stream lets me collapse them all as a group and the lock prevents the pass-through stream from being accidentally being reparented.

Tip for very-long build/test cycles. Over the past few years I’ve encountered a few shops with single build/test cycles ranging from hours to days to complete. In this case, the concept of CI is slightly challenging because the notion of frequent builds is constrained. In this case, I’d recommend setting up two distinct test phases; quick and full. The “quick phase” is a quick pass sanity test only performing tasks such as compilation and unit testing — enough to let developers know they can continue on forward progress with little concern. The “full phase” is the full blown cycle, taking hours/days to complete, that completes all levels of testing such as compilation, unit testing, functional testing, system testing, etc. I would execute the quick phase early and often while the full phase may be once per week. As an additional step, I would mark the snapshot used for the full phase with a pass-through stream for the purpose of reporting configuration diffs or letting developers reparent their project streams/workspaces on the latest known good “certified” build.

Interested in continuous integration? Perhaps you’d also be interested in multistage continuous integration…

/happy building/ – dave

2 comments »

Posted in Patterns, Tips and Tricks

Tags:

Team of One Pattern

February 22nd, 2008

We all know that AccuRev is well suited for large enterprises with teams of developers spread across the globe. But what about the crack team of one developer? You know, those of use who go solo because we know we can do it faster, better, and leaner than any contrived dream team.

Can AccuRev really work for small teams including a team of one? … You bet your ASCII it does!

In fact, I use AccuRev for my own personal projects. They happen to be AccuRev integrations, but are software projects nonetheless [Vim Plugin, GNU Bash, etc]. At this point, critics may proclaim, “For one developer, you just need to commit to trunk and label for each release.” Well… that worked OK for the first and second release, but then came the need to maintain multiple versions in parallel with patch releases (due to wholescale refactoring per major release) as well as compatibility between corresponding versions of AccuRev (since these are plugins). The compatibility matrix completely obliterates any suggestion of linear branching/labeling… so fuh-get-abot-it. Time to graduate from the traditional branch-based tools.

Stream Structure for Team of One Development in AccuRev SCM

click to enlarge

The above picture shows my stream structure containing projects including vim-plugin and bash-completion. I’ll use the bash-completion project as a reference example to discuss my pattern of development. Even as a single developer, I found it critical to maintain a strict develop -> test -> release pattern simply because my development activities change day-to-day and typically turn into “It’s been a month since I looked at this code… time to roll up the sleeves and figure out what the heck I was thinking!” If I was forced to a single commit bucket (branch), I’d go nuts — trying to manage multiple patches, new development, updates to documentation, etc and then being forced to deliver it all because pulling out changes is about as fun as filling sandbags with a pitch fork… I’d rack my brains trying to keep it all straight especially since I have multiple projects going on concurrently!

My development process is as follows (annotated as steps 1-6 on the picture):

  1. Develop – After working in my private workspace on a unit of work for days, weeks, months, I promote to the “-test “stream. Then, continue working in the private workspace on the next set of work.
  2. Test – After unit testing and performing a clean-room functional test of all changes in my “-test” stream, I deem all changes “release ready” and promote to the top bash-completion stream.
  3. Release Candidate — The changes in the base stream represent a configuration that is good-enough to start a new codeline. I do NOT snapshot an official release X.Y (just yet). I first create a “dash-x” line to start the codeline (e.g. bash-completion-3.x for the 3.0, 3.1, 3.2, etc versions).
  4. Maintenance — In anticipation for even minor patch work, I proactively create a “-maint” stream to collect any upcoming maintenance changes based on the starting codeline. Initially, this stream will just be empty and identical in configuration to the parent ‘dash-x’ stream.
  5. Official Release — At this point, I’ve immediately created the”dash-x” and “-maint” streams in succession so they are all identical in contents – namely, all containing the next release. So NOW I create my first official “dot oh” release (e.g. bash-completion-3.0).
  6. Publish – With the official configuration under snapshot, I ‘pop’ the code, archive, and ship to my web server. La commedia e finite!

With the visual nature of the StreamBrowser and the ease of creating streams, managing multiple versions of multiple products with AccuRev is priceless. I use a simple, repeatable development pattern that lets me separate ongoing development work (workspaces) from upcoming changes being tested (-test stream) all separate from previous releases (dash-x) and patch development (-maint). And the best part about all of this is I can (and have, MANY times) come back to any project even months later and quickly ascertain the current state of the union — what’s in development, what’s in test, which releases are published, etc. Sweet.

Lastly, while I use this ‘dot oh’ pattern for my own projects, I even recommend it for large team development. It’s a great pattern and I hope you find it useful for your own stream management.

/happy releasing/ – dave

3 comments »

Posted in AccuRev, Best Practices, Patterns, version control

Tags:

Stream-per-Task Pattern – with AccuRev SCM

October 11th, 2007

Streams are the workhorse of an AccuRev depot. Without streams, everyone would commit code to a single “bucket.” As you know, streams are connected together in a hierarchy that defines a promotion-based workflow. For mainline development, a workflow can be as simple as:

Dev > Test > Prod

or, sophisticated like:

Dev > UnitTest > Int > LoadTest > FlightTest > Prod

The Pattern

While streams are always used for organizing the core workflow, they can also be extended to logically organize developer tasks using a “stream-per-task” paradigm. The concept is simple: create a new stream for every task that one or more developers plan to work on. Name the stream logically, as in “Feature X” or “BugFix Y”. The developers then create their workspaces from this stream. This pattern works exceptionally well with agile development groups where work is broken down into tasks.

The Advantages

In theory, the task stream is just another stream and promotion step on the way to delivering changes to the core workflow (see adjacent picture). Though, in practice, the advantages are numerous:Stream per Task

  • Organize development activity by feature or fix
  • Visually see all active projects (removed task streams are implicitly finished)
  • Visually see and manage who is working on any given project
  • Develop individual projects against latest mainline without forced committing to the core (relies on inheritance and significantly reduces rollback)
  • Control access to projects by user or group using stream locks
  • Retarget entire projects and teams with a single drag-n-drop

The Critics

No suggestion is without its critics. Let’s address a few of the common claims. Claim: adding a new stream creates an unnecessary promotion step. Response: use a pass-through stream instead! Claim: creating a stream-per-task will create 10’s, or 100’s of streams. Response: Yes, but each stream is a 120-byte (yes, byte) record of meta data on the server so space consumption is negligible; the GUI ‘zoom’ feature makes navigating thousands of streams manageable; remove each stream when the project is finished; and using a good naming convention goes a long way. Claim: Creating streams is yet-another-step for someone; who will do it? Response: it depends; Sometimes savvy project managers enter them in AccuRev; other times it’s the release manager(s) who take this on as a routine task; but I say, empower the developers! Why not have the dev leads or project-oriented developers do the task — it’s sooo simple and can be controlled with locks, ACLs, or basic triggers.

Workspace per TaskFor those critics still remaining who need proof, I’ve provided an illustrated stream hierarchy where everyone contributes to a single bucket. Compare this picture with the one above…. and I’m sure you’ll be convinced. In short, the drawbacks of everyone committing to the same bucket include:

  • Unable to collaborate in isolation
  • Requires reverting (unmerging) features who’s release has been postponed
  • Limited visibility to active projects; requires strict wspace naming; hard to enforce
  • Retargeting project work requires multiple workspace reparenting
  • Unable to control per-project delivery or access with locks or ACLs

Concrete Example

With the pros and cons out of the way, lets show a concrete example. For both discussions above, Stream per Task ExampleI intentionally created stream structures in abstract form using generic names and a layout that is comparable. But lets get real. Here is a stream structure that represents a more realistic environment and even includes elements from a previous blog post on using the pass-through stream. Isn’t it great to see how this stream-per-task paradigm helps methodically organize activity across the entire depot!

Are there any other pros or cons to this approach? Does anyone have examples of using this pattern in their own implementation of AccuRev?

/happy tasking/ – dave

9 comments »

Posted in Best Practices, Patterns

Tags:

Stream Refactoring with Pass-throughs

September 20th, 2007

One of the greatest stories ever told is your software configuration management or CM process through the lense of a StreamBrowser. No other tool gives you the power to both visualize and manipulate your software development process in real-time. Over time, you’ll notice your stream hierarchy evolving into an elaborate tree effectively telling the story.

There are a number of techniques to help navigate the tree such as the GUI zoom feature and good naming conventions. But the unsung hero of stream organization is the pass-through stream. You probably already know about other types of streams such as dynamic, workspace, snapshot, and timebasis. Just like its siblings, the pass-through stream is also a configuration of source code but comes with a unique feature — all elements promoted to it are are automatically promoted to the pass-through’s parent stream, hence the name.

So what exactly are the benefits?

  • Group logically related child streams for organizational purposes.
  • A handle to reparent groups of streams without requiring an unnecessary promotion (to it).
  • Set include/exclude rules to be applied to all children streams without the unecessary promote step.
  • As an advanced technique, you can create a reftree on a pass-through to do software builds wherever the passthrough is currently parented — say last weeks build or last nights build — offering easy, visual, manual control over which configuration is built.

Lets put this knowledge to use. In the following diagram, you’ll see how a relatively sophisticated process model is cleanly organized Refactoring Streams with Pass-throughswith pass-through streams. There are a few pattern motifs used for organizing active development, nightly builds, system test builds, official release codelines, and customer one-offs. Notice how collapsing them all (left image) leaves the high-level process-centric streams nice and clean looking. Furthermore, notice how the names of the pass-through streams help tell the story of what activity is happening where.

Using pass-through streams is a passive refactoring technique that does not interfere with inheritance and can be introduced safely without disrupting development activity. In my experience, the more organized your stream structure, the happier the end users.

/happy refactoring/ – dave

No comments »

Posted in Best Practices, Patterns

Tags: