If you’ve used reddit, you already know how cool it is. If not, you should check it out. reddit allows people to submit any web page as something worth checking out. You can also vote up or vote down things you like or dislike. Now there’s a reddit for AccuRev!
Archive for March, 2008
Surf For AccuRev Using reddit.com!
March 29th, 2008
Adrift in a Sea of Conflicting Priorities and Assignments? Here's a Life Preserver!
March 28th, 2008Do you ever feel like things are out of control on your project, that you are adrift in a sea of conflicting priorities and requests? Do you suddenly find out at the last minute that you are the bottleneck and everybody is breathing down your neck asking you what is taking you so long to create the moveStuff() method but you had no idea that anybody even cared about moveStuff() or that you owned it? Do you ever find yourself in the exact opposite position, wondering why Sue and Bob didn’t get their stuff done that you need and then your boss walks by while you are surfing the net waiting for Sue and Bob? And who is Bob anyway?
The solution is simple! All you need to do is get everybody to move to Project. Well, if you have somebody you can spare full-time to keep Project up to date of course. Oh and I almost forgot, you’ll need to start using a requirements tool. But that’s it really, other than integrating them all together over the weekend and of course that’s assuming you’ve already gotten a CRM tool for workflow.
There is a simpler solution. It isn’t perfect, and it doesn’t solve all problems, but it can definitely provide the following benefits:
- reduce the chaos
- increase your vision into where you are and what’s going on
- reduce the number of status and/or project management meetings
- reduce the need to provide the same information over and over again
- simplify collaboration both locally and for distributed teams
- provide a more Agile workflow
The answer is to reduce the amount of rework that you are already doing. Right now you are probably storing defects in a defect tracking system, enhancements (aka RFEs, requirements, etc) in a requirements management tool (usually Excel or Word but sometimes an actual RM tool), and if you are using a project management tool it is probably MS-Project. All three of these product areas evolved to provide different aspects of project management for different groups of people and as a result they have lots of overlap. Considering how hard it is to coordinate three different systems, why not consider standardizing on one system for most of the work? The only question is, which system?
If we are going to try to do most of our project management work in a single tool, we should first decide what the interesting activities are. I believe they are: recording enhancement requests and defects as they are gathered by marketing or reported by users, load balancing, estimated completion calculation, critical path determination, work assignment, workflow, and reporting.
First let’s consider how well suited Project is for doing most or all of these tasks. Project is good at taking a small static collection of large tasks and doing load balancing, estimated completion, and critical path determination. Thus, it is mostly used for the very narrow task of project management of enhancements.
Next let’s consider requirements management. For whatever reason, most people use Excel or Word as their requirements management tool instead of a “real” requirements management tool. Excel and Word are just not appropriate for project management.
Lastly, there is defect tracking. A defect tracking system covers the assignment, tracking, workflow and reporting of defects. There is usually a higher volume of defects than enhancements, and they are usually smaller in scope and have a more complicated and often more time critical workflow. If it works well for defects, it should work equally well for enhancements.
Based on this analysis, it makes sense to extend the project management that you are already doing with a defect tracking system to include enhancements. A generic name for something that is either a requirement, enhancement, or defect is “work item.” By using work items to track all work, it is easy to see where you are and what remains to be done. Now you can use a similar workflow for enhancements as you do for defects, for instance from newly entered, to triaging, to assignment, to test development, to completion, to test, to integration, to delivery. You can easily run a query to see which work items have their code written but do not yet have any tests. Similarly, you can see which work items are done from a coding perspective and have tests but have not yet been verified as done by QA. This will give you a much more complete view of your overall project status and progress.
Whatever you are currently using for defect tracking it will be straightforward to start getting the benefits of managing defects and enhancements together. Just add a field that indicates if a work item is a defect or an enhancement. You may need to make a few more changes to accommodate a slightly different workflow for enhancements than you have for defects, but I think you’ll find it is worth the effort. For one example of how this can work, you can take a look at how AccuRev does it using AccuWorkflow.
Pattern for Stable Development
March 27th, 2008It’s Wednesday morning. 9:23am. With coffee in hand you execute a simple version control command: update.
Hey, blame yourself! You should have known better than to blindly accept all changes from every developer who quickly committed code the night before just minutes before darting out. What did you expect to happen? For teams that work on a single branch, broken updates can be quite common. For those that branch for isolation, their updates are comparably more stable but they are required to (painfully) merge more often to stay in sync.
Ok, lets get real. On some mornings, performing an update can yield minimal-hassle results and we all know that staying up-to-date is always a good thing. But often when you least expect it, often when it matters the most, often when time is critical, the simple execution of an update results in sheer and utter frustration. You know. The code doesn’t even compile. Forget passing unit tests. Forget working at runtime (these are the worst to deal with!). The last thing you want to do is spend the entire morning scrubbing logs, reverting out changes (complete WOT!), or walking the halls finding the culprit only to see that everyone you need is in a meeting in some cleverly named conference room.
But you had to! You know you need to update frequently in order to stay current with your peers. But back in your mind, you’re worried that their changes are going to negatively affect your changes. Nevertheless, all you can do is…. update. But what if you had control over the situation? In this pattern for stable development, I’ll describe an approach in AccuRev that provides incremental updates based on predefined good builds and minimizes merging (due to implicit merge tracking).
The Pattern. AccuRev’s stream-based architecture with built-in inheritance and time-safe snapshots can entirely eliminate the traditional problems of shared development (with branches). The first thing to acknowledge is that you need to work off of a stable codeline. period. 
I know, I know. Some of you reading this think you need to always be working against bleeding edge in order to move as quickly as possible. Fine. Then you accept the top-half of this blog post and are willing to deal with the pain/time/sleep/sanity lost in the ether. For the rest of us who don’t live at the office, we need to work off of a stable codeline. With AccuRev, this means working from a snapshot (or time-based stream). In a previous blog, I wrote about a pattern for continuous integration (CI) where snapshots are created for each frequent “good build.” Extending this CI pattern, lets use each good build snapshot as the baseline for active development. The idea is simple: Reparent your workspace or project stream to subsequent “good build” snapshots and then update. The update will bring down (delta!) changes to your workspace since the last good build. In this way, you knowingly accept changes that are guaranteed to have passed some level of test criteria. In the meanwhile, changes in the mainline Integration stream can churn-n-burn with various levels of volatility without affecting your active development environments — you’re saved by the snapshot! After days or weeks of hopping build snapshots, when it’s time to deliver your changes to the rest of the world, simply reparent to mainline Integration, perform any last minute merges, test, then promote! For an extra level of stability, consider only working off of certified builds – snapshots that have passed rigorous tests.
Wait. Who does the reparenting? Good question. I say let the developers or team leads manage where and when project streams get promoted. Reparenting is easy and reversible! On the other hand, maybe your CM or Release Engineer takes on responsibility for managing builds and reparenting projects on particular days or by request. You can use stream locks to easily control who can reparent, by user or group!
What about setting a time rule on a stream? As you know, snapshots represent configurations at specific points in time. You may also know that you can optionally set a time rule on a dynamic stream to control (i.e. throttle) when changes are inherited from the parent streams. 
[StreamBrowser: Rclick Stream --> Change Stream --> set time]. Guess what! The effect of reparenting to snapshots can equally be obtained by simply setting a time rule on your parent stream! This works best if you have a stream-per-task paradigm so the time rule only affects your (team’s) active development. The idea is simple – anytime you want more changes, simply (re)set the time forward say an hour or day or week and any newer changes will be available to downstream streams and workspaces. Personally, I like setting time rules only for temporary use cases such as for quick testing or to wait a day for new changes. If I’m doing development that will take weeks to complete, I’d rather reparent to a snapshot because it lets me physically ’see’ where I’m working. But both options result in the same effect — controlled updates.
This pattern is ideal for letting teams stay up-to-date by reparenting to stable baselines and only updating changes that are known to have passed some level of testing. Ultimately, with clean(er) updates, the results should be increased productivity, increased quality, and happier developers! Happy developers == good.
/happy coding/ – dave
Fortune 500 Software Company Answers Your Questions on Using AccuRev
March 26th, 2008As one of the world’s largest software companies, we often field a number of questions from people thinking about making the switch to Accurev for software configuration management. I thought I might post some of the common Accurev questions and our answers to them proactively to save a little time 
How many users do they have and how big is your repository?
We have 125 users, spanning 3 product offerings, and the repository is about 20Gb.
What kind of storage are you using on the backend?
Nothing too fancy. Windows box with a 100Gb RAID array – mostly done for data redundancy. I think we could have gotten away with less, but we erred on the side of caution.
How fast has your Accurev repository grown since day one?
We have averaged a growth rate of roughly 200-250Mb per year over the past 6 years.
What is the performance of Accurev like?
I can’t say we’ve had many complaints. We have a setup where the primary server is in colocated with the largest US-based team, and we have AccuReplica servers at our other US sites and another at our site in India . My experience is that it is much faster over VPN than Perforce or ClearCase were.
How long did it take for users to become familiar with the streams concept?
Not long at all – hours at most. Our developers caught on right away and loved it. We ran some education sessions to help get things jump started, but most developers understood immediately – “Hey, I don’t have to make the same fix 4 different places anymore!”, and Accurev merging is so good that merge points are no longer a pain for developers.
How many admins do you have?
Zero. We setup some simple policies for behavior – some best practices that we tell everyone, and then my release engineer is in charge of the ‘mainline’ streams that will turn into product releases. It takes him less than an hour a week to do the admin type tasks in the system (update triggers, lock streams on occasion, create user accounts), but by and large the things a ClearCase or CVS admin would do are done by the individual developers. Our developers love it because it empowers them to do what they need to do.
What kind of response speed do you get from Accurev support?
Accurev’s email-based support is excellent. I can’t recall ever waiting more than an hour or two for someone to answer, and usually it takes only 1-2 emails to resolve the problem – the support team knows that they are doing. In 6 years I can’t recall EVER having to pickup the phone and call the support line because the email is answered so quickly.
How painful are server and client upgrades?
They are a piece of cake. The regular installer is smart enough to perform the upgrade in a wizard fashion and there are no skill testing questions! When we went to v4.6 it took less than an hour to perform the upgrade, push new clients and verify that everything was working, including our build system. If you have provisioning software you could deploy the new clients in moments. We have upgraded our servers all the way from v3.0 and have never experienced any issues at all with the upgrades.
What do you like least about Accurev?
Directory scanning can be more time consuming than I’d like. It would be nice if it were done as a background task all the time when the UI is running instead of only when you make a request. Other than that, I’m hard pressed to think of anything I don’t like.
What is the best thing about Accurev?
Developer Empowerment – far and away this tool lets my developers do what they need to do exactly when they need to do it.
Parrallel development – never have I worked on projects that required so little codeline coordination as we’ve enjoyed since taking on Accurev. Merging code is so easy that merge points are done in a fraction of the time. And Accurev makes it possible to merge your code so often that merges are less painful when they happen.
Visualization of change – with the dynamic process version browser and the annotation tool visualizing how your code has changed over time is a snap.
Continuous Integration: Building the Pyramid
March 13th, 2008I just saw the movie 10,000 B.C. (No, not the movie about when ClearCase was created). Certainly a diversion from everyday tasks, but I found myself focused on a couple of aspects of the movie. My first reaction seeing the workers moving the stones to build the pyramids (never mind the accuracy, just stay with me) was: how long does it take to move a single stone up that ramp? What if it took the group of workers pulling it a whole day? And what happens if they pull it all the way to the top and it cracks? Or is the wrong size? Now you’ve got to get rid of the stone and you lost a day.
Now take a look at the developers around you. You’ve labored hard all day, you feel pretty good about your accomplishments, take a rest and review it tomorrow. But you come in the next day and the build failed. You hauled that code to right place, only to find it cracked and the wrong size. What have you got now? You lost a day, your workers are all pointing the finger at each other and management wants to know why there’s a delay. You’re now spending the second day figuring out what was wrong on the first day, delaying the work you should have done.
Sounds pretty bad right? What you need is a way to continuously check what is going on. So you ask each developer to run a build, maybe even run some tests before promoting their code. Each developer seems to be on track, but now those integration points keep burning you. Bob only thinks in Windows, he keeps making references that work on his machine. When Julie, your UNIX guru, gets them they never build. Now you need some integration checking. But if Bob performs the integration on Windows he doesn’t have a problem (today) and Julie is sick and tired of managing Bob’s work. The build guru you hired would mutiny if she had to run builds all the time. You need some automation to relieve the pain.
And along comes Continuous Integration (CI). Martin Fowler wrote a great summary of CI (you can find the materials here). Besides this article there are many tools and even some books that are worthwhile for in depth knowledge of CI (take a look at Continuous Integration: Improving Software Quality and Reducing Risk – a 2008 Jolt Award winner). I’ll do them all a disservice and sum it up: frequently integrate autonomously.
I find the keyword here is autonomously. I don’t understand why developers spend so much time writing software for others to improve their processes, and yet they are willing to manually and repetitively do their own work. And to top it off they will rewrite software over and over. Just as you shouldn’t be spending your time writing a file i/o subsystem for the umpteenth time, you shouldn’t be running your build processes manually for the umpteenth time. If you can spend 2% of your time integrating builds instead of 15% of your time, isn’t that better?
Some time ago we had the problem I described above with Bob and Julie. We had Windows developers whose code would prevent builds on occasion with another OS. We have six primary OS that we ship, with many other variants. You can imagine the pain when nearing a release date, if you were to come into the office and find out you have no build. Once this happens it becomes a midday manual process to run and test the build. What drudgery. CruiseControl can solve this problem, and with a little work, can run with your custom build scripts. CruiseControl can greatly reduce the loss of nightly builds, as a matter of fact I’m hard pressed to think of the last time a nightly build failed for this reason, although we’re looking into commercial offerings that provide even more value. For now, having developers speak up (Hey Joe, that checkin you just did failed on Linux) can reduce your own personal stress level. You’ll have less hesitation to check in code just before you go home, with less need for connecting the dots because CI has your back. On occasion driving home my phone still buzzes, and when it does I know the build failed. Not the nightly build, CI alerted us soon enough so I still have time to fix it before that.
No worries. We’ll get that pyramid done on time.
AccuRev admin tricks – SCM law enforcement
March 12th, 2008Here’s an AccuRev Tech Tip you might not have been aware of to help with your software configuration management law enforcement. As you know, the AccuRev TimeSafe architecture guarantees that every single transaction ever executed is going to be fully traceable, a blessing to those who care about auditing, perhaps regulatory compliance, or even just wanting to be able to see who has been up to what.
But what if you want to know what has *not* been done? Huh? What the heck am I talking about? I’m actually referring to commands that are not executed because you’ve put security in place to prevent them. So even though the offending users don’t get their way, perhaps someone might want to know who is trying to upset the apple cart or merely have the ability to apprise the user of organizational best practices.
One way to do this is through email. Let’s say you’re implementing the default set of commands that AccuRev limits to Administrative users, and someone unauthorized tries to create a new user. Put the following into the proper place in the server_admin_trig trigger:
if ( `$::AccuRev ismember $principal "$admingrp"` == 0 ) {
print TIO "Execution of '$command' disallowed:\n";
print TIO "You are not in the $admingrp group.\n";
system("simple_email", $arg1, $arg2); # insert this line #
close TIO;
exit(1);
}
Now, whenever someone who isn’t allowed tries to create a user, you can get a nice email that says:
To: anyone_who_cares
Subject: AccuRev alert
Body: User “jtalbott” tried to run the “mkuser” command. That’s 10 demerits. Put him on double-secret probation!
…or something similarly amusing. What you’re basically doing is sending your desired parameters into the simple_email script. You can write your simple_email script using whatever language you want, as long as it’s accessible in the local path, and naturally it would be reusable. Now, for virtually any operation in AccuRev that you want to know about, whether you want to know if it succeeds or fails, you can get notified about it. You’re certainly not limited to email either. simple_email could send text messages, page someone, basically any form of communication you prefer that can be coded.
It really is that easy. I highlighted a certain example, but there are many practical applications. How do you think you would be able to take advantage of this in your organization? Or if you’re already doing it, what kind of things are you communicating?
Editor Learning Curves
March 7th, 2008The importance of software tools can never be understated. For a developer, choosing the right text editor is a very important choice. Do you use notepad? pico? ed? nano? emacs? vim?
If you’re looking to swap editors or looking for an upgrade, maybe this picture can help. It’s one of my favorites.
<click image to enlarge>
If these don’t suit your need, maybe you should try editing source code with butterflies.
<click image to enlarge>
/happy editing/ – dave
Pattern for Continuous Builds
March 5th, 2008Continuous integration (CI) is all the rage these days because merging, building, and testing (shared) configurations early-and-often is a good thing. Actually, it’s a great thing! After all, finding problems sooner rather than later benefits everyone. For some, CI means simply testing compilation. (Phew… it works. Ship it! haha). For those investing time in a full test harness, CI may mean frequently executing a suite of tests at various levels (unit, functional, system) to validate functionality and identify regressions. I’ve even seen other levels of CI to include lab testing, flight testing, or even customer acceptance testing for even the smallest of changes. Regardless of how you ‘do’ CI, I’ll show how I use AccuRev for continuous integration. [Keep in mind that this is one interpretation of the subject matter]
The Pattern. The stream-based nature of AccuRev makes it very natural to define separate areas for development, integration, testing, and release. 
As seen in my example stream structure, I have an Integration stream as the first point of merging between individual project streams. This Integration stream is a great place to hook up a CI tool [Cruise Control, CC.NET, FinalBuilder, QuickBuild] and perform nightly or per-promote builds. I prefer to create a snapshot before doing the build mainly because snapshot creation is atomic and their immutable configurations guarantee reproducibility. After creating the snapshot, I will pull the build from the snapshot name. You could build from the Integration stream directly (similar to the concept of a moving label), but creating snapshots makes it easy to visually identify with the build process and compare good builds from bad builds with simple stream diffs. [Note: integrating any of the above mentioned CI tools is as simple as telling the build tool to pull code from a stream (by name) and then configure the build tool to execute at some frequency and notify people of the build status]
What about all those snapshots? At first, you may think, “Isn’t this going to create a gazillion snapshots? Won’t that take up a ton of (disk) space and totally clutter the stream browser view?” Well… No.
- Snapshots are cheap. Snapshots are extremely cheap server-side entities consuming ~100bytes regardless of the number of elements they label… so go nuts! Snapshots mark transaction numbers, not elements! I say, always do what you need to solve important problems and answer tough questions even if that means creating a gazillion snapshots; just be sure to organize them.
- Clean up as you go. Your CI build script (build.xml, Makefile, or doBuild.sh) can easily be instructed to remove a snapshot for every snapshot created. I’d recommend keeping around enough snapshots (say 3 to 10) to do valuable work such as comparing builds or serving as temporary baselines for developers who want to reparent. As you can see in the stream structure, AccuRev stores both active and inactive snapshots and it is easy to reactivate any snapshot if necessary (I’ve enabled the stream browser to show both; lower left corner option).
- Group snapshots. I prefer to tuck logically related sets of snapshots behind a locked pass-through stream. The pass-through stream lets me collapse them all as a group and the lock prevents the pass-through stream from being accidentally being reparented.
Tip for very-long build/test cycles. Over the past few years I’ve encountered a few shops with single build/test cycles ranging from hours to days to complete. In this case, the concept of CI is slightly challenging because the notion of frequent builds is constrained. In this case, I’d recommend setting up two distinct test phases; quick and full. The “quick phase” is a quick pass sanity test only performing tasks such as compilation and unit testing — enough to let developers know they can continue on forward progress with little concern. The “full phase” is the full blown cycle, taking hours/days to complete, that completes all levels of testing such as compilation, unit testing, functional testing, system testing, etc. I would execute the quick phase early and often while the full phase may be once per week. As an additional step, I would mark the snapshot used for the full phase with a pass-through stream for the purpose of reporting configuration diffs or letting developers reparent their project streams/workspaces on the latest known good “certified” build.
Interested in continuous integration? Perhaps you’d also be interested in multistage continuous integration…
/happy building/ – dave