Archive for October, 2007

Three Absolute Requirements for Successful Offshore Application Development, Part 2

October 31st, 2007

 

Requirement 2: Business Interests Must be Aligned

There’s an old saying about the game of poker: if you look around the table and can’t figure out who’s the fish, it’s you.

Let’s face it: they’re doing it for the money. Just like you. But you’d better understand how they make money in the business relationship or you’re going to feel like you’re the only one not making out.

There are lots of companies that reward late projects and punish early ones. Not intentionally, of course. Maybe you know can recognize who this is:

When projects were late they would respond by creating special incentives, adding people to the project, and increasing the project visibility. When the project was doing well, they’d pull people off to fight other fires, take the best people away from the project leader to work on other projects, and management would stop attending the project review meetings.

Next project comes along, and the offshore project leader comes in with an optimistic schedule. You do the math.

Hey, if you’re reading this blog, you’re a smart person. You know you’re not going to change their culture, so maybe you should spend some time to learn to deal with it. Sure, people are people, and application developers in Moscow, in my experience, have more in common with application developers in New York than they do with factory workers in Minsk, but when people get in groups their behavior changes. That “group dynamic” differs depending on their cultural background.

In the US and northern Europe, senior application developers can follow a technical career ladder and get increasing career respect and satisfaction from being architects and CTOs. At least that’s the theory. In much of Asia, career status is all based on the number of people that report to you in the hierarchy. It’s an up-or-out philosophy that makes it hard for an Indian engineer to explain to his mother why after five years with the company he’s still not a manager.

When working with an outsourcing company, if you don’t figure out their model and how they get paid, it’s like playing poker blindfolded. Many companies build a model based on rotating junior people through their company on standard projects. If you need junior engineers to implement an e-commerce Web site, it’s probably something they will do many times and can do with the talent on hand. If you need people to figure out how to do something unique, your outsourcer might have all the brains in the world, but it just isn’t going to make business sense for them to put their technical leads on the project, when they could be running other e-commerce Web site apps. It makes as much sense as buying a French car or a German perfume.

One of the hardest problems in assessing professional job productivity is identifying the impact of quality. If the reward system is around requirements trace-ability and meeting schedules, quality is going to suffer. If you try to make quality a hurdle to be overcome, you’re going to spend such a long time in requirements specification you are never going to get a payback from offshoring the application development.

There are two things smart companies do to deal with this alignment dilemma:

  1. Short iterations, e.g. a two-week Agile process
  2. Remember Requirement #1, and bring the remote team lead back to corporate frequently to maximize alignment with the leader.

In Part 3, Don’t Annoy the Pig.

SCM Team Collaboration

October 26th, 2007

It’s amazing how many times I’m asked about how you can collaborate on sub-projects or individual features with other developers without affecting the rest of the development team.  As a matter of fact, this came up the other day, which is why I thought it would make a good post.

There are several ways that teams can collaborate in AccuRev, and depending upon what the situation is, would choose different methods.  I describe two simple scenarios where 2 developers are working on a few files together and another where several developers are teamed together to work on a sub-project.

Scenario 1: Two developers are working on the same foo.c file and must complete the work before it can be merged with the rest of the project (ie. promoted).

The great thing about private workspaces is that developers have a place where they can commit their code without impacting the rest of the team.  The code *is* placed on the server, but still remains private to the developer.  Meaning, other developers who update their workspaces will not propagate those changes into their private workspaces.

However, since the code *is* on the server, another developer can explicitly pull that version into their workspace using the “send to workspace” command (aka. “co”).  The developer can make changes and keep the file to place the new version on the server, again stored in the private workspace.  This exchange continues until the work is completed and the file can be promoted to be shared with the rest of the team.

This is typically a good method to use for collaborating on just a few files, but what if you had an entire sub-project to share with many developers.

Scenario 2: Many developers working on entire sub-project which must be completed with testing before merging with the rest of the project.

This scenario is the “bread and butter” of what AccuRev and the stream based architecture is all about.

To set this up, you simply create a stream for the sub-project from the main project stream.  The sub-project development team creates workspaces off the sub-project stream while the rest of the development group is based upon the main project stream.

With inheritance, any changes made by the main project team is automatically inherited by the sub-project team so they [sub-project team] are continuously integrating with the main development effort.  The main project team is completely unaffected by the changes made by the sub-project team.  The sub-project team can also control what changes are inherited into their stream by setting a time basis on their stream.  The time basis will only allow the changes to flow into their stream up to the time set.

Eventhough I described this scenario as a method for larger groups, it can also be used for the simple case of 2 developers working on the same file.   Streams are so architectually light weight, that they can be created and removed in a matter of seconds.

 I hope this little tip helped for those looking to collaborate.

Three Absolute Requirements for Successful Offshore Application Development – Part 1

October 25th, 2007

 

Failure teaches a man many things: don’t mix beer and tequila, dates don’t want to hear about your ex, and good software isn’t developed offshore. But success can be a better teacher, and a little failure combined with some success is best of all. After fifteen years doing projects in India, England, Israel, Canada, Moscow and the Ukraine, I’ve got the scars to prove I’ve learned a few basic requirements.

Requirement Number 1:

Leadership is everything. In case you missed it, Leadership is EVERYTHING. And I don’t mean sitting on the phone in the corporate office. I mean the person who represents everything you want to achieve and sits in that remote office having to drill the corporate mission into people who have little respect for corporate, and strongly suspect that the feeling is mutual.

The Roman Empire lasted for a thousand years, which might be longer than your project is going to last. The job with the most difficult requirements in the Roman Empire, tougher than Senator, tougher than General, was Governor of a remote province. Romans knew that the Governor had the toughest job. He had to convert his provincials into people who wanted what Rome wanted, people who would profitably grow Rome’s Empire, not people who would revolt and suck up Roman resources.

There are few application development jobs tougher than managing a remote development group. And there are precious few people that can do it. It takes great leadership, as well as management ability, which is hard to find.

Next, in Part 2, if you can’t identify the fish at the table, the fish is you.

Lorne Cooper

AccuRev TimeSafe makes retrieving code from "anywhen" easy

October 24th, 2007

One of the problems frequently faced in traditional branch-and-label version control systems is how to retrieve a particular configuration of code from the past if it wasn’t tagged or labeled at the time. You forgot to label the release-candidate-6 branch from last month before building it, and now you have no way to accurately pull the correct files.

AccuRev, through a combination of its TimeSafe architecture and the power of its streams, makes this easy. Say you have a development process where code follows a promotion model from individual contributors to an Integration area, to a QA area, to Production. The release candidates are based on the QA stream. The QA team built and released RC5 on Friday, June 7th at 11:00 pm, but unfortunately neglected to snapshot it. In the following weeks, additional changes were promoted to the QA stream.

To get guaranteed access to the code configuration as it existed for RC5, you could do the following:
* create a stream off of QA and assign it a time basis of June 7th, 11:00 pm
* run “accurev pop” to retrieve the contents to disk

anywhen_stream.jpg

It’s as easy as that! Furthermore, if you want to automate this process, how about creating a stream for all your source pulls, call it something like <depot>_source_pull (or equivalent, doesn’t really matter). Then, whenever you need source files, use the “accurev chstream” command to first move it to whatever parent stream you want the files from and second to change the time-basis on it to the correct time or transaction. Lastly, you pop from that time-basis stream to place the files in whatever location you want:

“accurev chstream -s <depot>_source_pull -b <depot>_target_stream”
“accurev chstream -s <depot>_source_pull -t 356″
“accurev pop -R -O -v <depot>_source_pull -L c:\build_dir .”

There are other ways to skin this cat in AccuRev, as there usually are, but this is a quick way to get guaranteed results.

Force Testing

October 19th, 2007

There are a variety of testing methods: White Box. TDD. Mocks. Unit. Stress. Functional. Taste. Flight. Load. and more…

It’s Friday… Here’s my favorite test courtesy of Boeing… check out the 777 Wing Force Test.

/happy testing/ – dave

New AccuRev 4.5 product review on CMCrossroads

October 18th, 2007

I just noticed this great new AccuRev product review on the CMCrossroads Website for AccuRev 4.5 by Bob Aiello, an SCM expert from a large financial institution in NY. To answer the first commenter, take a look here at the comparison of how AccuRev does reparenting, or retargets work to a future release vs. Perforce.

Agile Programmable Completion – AccuRev + GNU Bash

October 18th, 2007

When at the command line (CLI), productivity means keeping your hands on the keyboard. But once your fingers have memorized all the commands, flags, static arguments, and common usage patterns — can you still get faster?

Yes.

Programmable completion is a shell facility that allows for customizing the command line in real-time as it is typed. Also referred to as “TAB Completion”, many shells in both Linux and Windows have a default implementation that support completion on filenames and directories. If you’re lucky, you’ll even get environment variables and functions.

Let’s move to software configuration management (SCM). Various branch and label-based SCM systems like CVS have basic tab completion for commands and flags. Thats a good start. But an agile user needs a context-sensitive, custom-data completion facility. What you ~really~ want is completion on your own data — branch names, labels, usernames, etc.

Users of stream-based AccuRev are in luck.

Do you use AccuRev on Linux? If so, download the latest GNU Bash (2.05+) completion for AccuRev 4.5.x. Here is the README. You’ll never have to memorize flags or type stream names again.

Coming in Part 2 — Support for Windows users.

/happy tabbing/ – dave

Put your source code on a "Build Artifact" diet with AccuRev

October 12th, 2007

I was working with a prospect recently and showing them how easy it was to import their source code into AccuRev from their existing version control tool. We were about to load the external files and did a quick check on the disk space for the tree to get a rough estimate on how long it would take. Almost 2 Gigabytes! I looked a bit closer and found that each of the numerous directories we were importing had a “UnicodeDebug” sub-dir.

Now, I’m not a Visual Studio expert, but in this case storing build artifacts in AccuRev is really an unnecessary step. Because AccuRev guarantees total reproducibility of any code configuration, it would be redundant to store those directories and files on the server since they can be recreated at any time. Of course, the developers might want to keep them on their local drives though. It was a simple matter to set a local environment variable, ACCUREV_IGNORE_ELEMS=*UnicodeDebug* After that, even though the code remains on local workspace disk, AccuRev doesn’t “see” them as external (neither from the GUI nor from the command-line) which removes the possibility of accidentally adding them to source control.

There’s a much longer debate that can be had concerning versioning build artifacts in general, but that’s a much wider scope than this post.

Oh, and by the way, the source tree had dropped from 2 GB to 18 MB. Loaded in about 15 seconds!

Stream-per-Task Pattern – with AccuRev SCM

October 11th, 2007

Streams are the workhorse of an AccuRev depot. Without streams, everyone would commit code to a single “bucket.” As you know, streams are connected together in a hierarchy that defines a promotion-based workflow. For mainline development, a workflow can be as simple as:

Dev > Test > Prod

or, sophisticated like:

Dev > UnitTest > Int > LoadTest > FlightTest > Prod

The Pattern

While streams are always used for organizing the core workflow, they can also be extended to logically organize developer tasks using a “stream-per-task” paradigm. The concept is simple: create a new stream for every task that one or more developers plan to work on. Name the stream logically, as in “Feature X” or “BugFix Y”. The developers then create their workspaces from this stream. This pattern works exceptionally well with agile development groups where work is broken down into tasks.

The Advantages

In theory, the task stream is just another stream and promotion step on the way to delivering changes to the core workflow (see adjacent picture). Though, in practice, the advantages are numerous:Stream per Task

  • Organize development activity by feature or fix
  • Visually see all active projects (removed task streams are implicitly finished)
  • Visually see and manage who is working on any given project
  • Develop individual projects against latest mainline without forced committing to the core (relies on inheritance and significantly reduces rollback)
  • Control access to projects by user or group using stream locks
  • Retarget entire projects and teams with a single drag-n-drop

The Critics

No suggestion is without its critics. Let’s address a few of the common claims. Claim: adding a new stream creates an unnecessary promotion step. Response: use a pass-through stream instead! Claim: creating a stream-per-task will create 10’s, or 100’s of streams. Response: Yes, but each stream is a 120-byte (yes, byte) record of meta data on the server so space consumption is negligible; the GUI ‘zoom’ feature makes navigating thousands of streams manageable; remove each stream when the project is finished; and using a good naming convention goes a long way. Claim: Creating streams is yet-another-step for someone; who will do it? Response: it depends; Sometimes savvy project managers enter them in AccuRev; other times it’s the release manager(s) who take this on as a routine task; but I say, empower the developers! Why not have the dev leads or project-oriented developers do the task — it’s sooo simple and can be controlled with locks, ACLs, or basic triggers.

Workspace per TaskFor those critics still remaining who need proof, I’ve provided an illustrated stream hierarchy where everyone contributes to a single bucket. Compare this picture with the one above…. and I’m sure you’ll be convinced. In short, the drawbacks of everyone committing to the same bucket include:

  • Unable to collaborate in isolation
  • Requires reverting (unmerging) features who’s release has been postponed
  • Limited visibility to active projects; requires strict wspace naming; hard to enforce
  • Retargeting project work requires multiple workspace reparenting
  • Unable to control per-project delivery or access with locks or ACLs

Concrete Example

With the pros and cons out of the way, lets show a concrete example. For both discussions above, Stream per Task ExampleI intentionally created stream structures in abstract form using generic names and a layout that is comparable. But lets get real. Here is a stream structure that represents a more realistic environment and even includes elements from a previous blog post on using the pass-through stream. Isn’t it great to see how this stream-per-task paradigm helps methodically organize activity across the entire depot!

Are there any other pros or cons to this approach? Does anyone have examples of using this pattern in their own implementation of AccuRev?

/happy tasking/ – dave