Feeding The Right Sources To Your Builds

I was recently asked the best approach for how to control the input to one's builds, and while there is no single solution to this that fits all projects, here are two ways you might want to go.

Freeze Then Branch

The current approach for JBoss Tools development involves continuous nightly builds from trunk until such time as the project is deemed frozen in prep for an upcoming milestone or release candidate. At that time, the trunk is copied to branches/JBossTools-3.1.0.M4, and very little is done to that branch - only urgent tweaks and fixes. A new Hudson job is cloned from the current one, and adjusted to point at the branched sources instead of trunk. The maps are also updated to point at the new source location in SVN.

This allows more nightly builds towards an upcoming stable development milestone, while new development can continue in parallel in trunk. When the milestone is released, the unneeded job is disabled or removed.

The only issue with this approach is that all plugins, built from the head of a branch (or trunk) are re-versioned w/ the latest timestamp every time they're compiled. Thus upgrading from one 80M update to the next requires another 80M download. To mitigate this, milestones are released only once every month or two.

Tag And Release

The current approach for numerous Eclipse.org projects, such as GEF, is to develop continuously in HEAD and when a number of changes are ready to be released, a tool such as the org.eclipse.releng.tools plugin can be used to, in a single operation, both tag the sources and release those tags into the map file.

This permits a granular approach to plugin versioning wherein only the plugins which have actually changed are renumbered, and thus incremental updates between releases is possible, and if only a single plugin changes from week to week, only that plugin will be upgraded.

This approach also allows your adopters to get more stable, weekly updates rather than hourly or ad hoc nightlies which may include incomplete changes. Sure, you can only watch for good builds in Hudson, but a more predictable schedule makes inter-build communication easier.

The only issue with this approach is that it introduces extra overhead, unless the tag&release process can be automated and run on a (weekly?) schedule. For CVS sources, there is the org.eclipse.releng.tools.tagandrelease crontab script available; for SVN, no such script (yet) exists. Want to help? See bug 264713.

Hybrid Approach

With the Athena Common Build, you can set up a Hudson job to run from tagged maps, but also override those tags on demand to force a build from head (or trunk) instead of from the specific repo tags.

To do so, pass in the following flags to EXTRAFLAGS via your build's job configuration / parameters. To build from the workspace's sources instead of fetching anything new from the repo (bug 252774):

-localSourceCheckoutDir /opt/users/hudsonbuild/.hudson/jobs/${JOB_NAME}/workspace/

To fetch from the repo, but ignore tags (bug 251926):

-forceContextQualifier -fetchTag HEAD

Or, like many projects on build.eclipse.org have done, set up two jobs: one for nightly from trunk, and one for weekly integration from maps. Then instead of doing CI builds when you remember to, they will run automatically when the repo changes so you'll have immediate feedback when something has broken. Add email, IRC, or Twitter notification to the job, and everyone else will know too.

Delicious Wordle

My del.icio.us Wordle

The Group Of Seven

Seven things about me, as inspired by Roy's tag:

1. In just under 3 weeks I've become utterly obsessed with my Blackberry. My wife, who at first was apprehensive of hers (along with all new technology), has been using it non-stop as well. It's been a good text'mas. :)

2. My wife and I live with two American Pointers, two horses*, three laptops, two smartphones, and many, many usb peripherals. The office is the warmest room in the house.

3. My day job consists of maintaining and improving the build systems for JBoss Tools and JBoss Developer Studio. My night job involves numerous Eclipse projects, including Modeling and Dash. Basically, I hibernate in the winter in front of the computer.

4. My parents - divorced - are of Jewish/American (dad) and German/Canadian (mom) decent. They've now been separated longer than they've been together. With the subprime mortgage crisis in the US, he recently moved back to Canada to seek more gainful employment. Unfortunately, he chose Calgary which isn't faring much better, now that gas prices are finally coming back down. My mom now lives in Scaborough near us, having returned from her self-imposed escape to Vancouver, and is more or less retired.

5. I graduated from the University of Waterloo back in '99 with a BASc in Chemical Engineering, having spent the balance of my last two years learning about programming, workflow design, process control, and optimization. In retrospect, it seems I was always headed for release engineering.

6. Much like Andi Gutmans, I too hate shaving... but as I work from home, I can get away with not doing so for days at stretch. I love my 30-second commute to work, which offsets my 8-to-10-hour workdays quite nicely. Why commute 2-plus-hours a day when you can use that time to get real work (or at least email) done? Thankfully, more and more companies are realizing this and are letting us work remotely.

7. I try to find time - when not working or hacking - to walk my dogs, hike, cycle, kayak & swim: sometimes several in series. One favourite route involves cycling from my house to the beach at maximum speed, swimming in the lake to cool off, then portaging the bike up the side of Bellamy ravine and home. This past season I rode over 750KM and kayaked over 75KM, mostly in and around Toronto, Scarborough and Pickering.

So, then, here are 7 more in this chain, in no particular order...

1. Opal Aristizabal, tech writer and Insight driver
2. Jamie Callingham, one of the few teacher-types I know who actually has time to maintain a blog
3. Alain Cloutier, graphic designer with a slightly insane blog, two great kids, and horse-addicted wife. He and I share a similar pain -- horsefolk are a special (wonderful) breed of crazy. :)
4. Kenn Hussey, IBM ex-pat and blogger with great frequency, especially this past Blaugust.
5. Wassim Melhem, overworked - but always well-dressed - project manager
6. Julia Raphael Morgan, opera singer sans blog at the moment...
7. Andrew Overholt, Linux evangelist and fellow cyclist

And here are the rules I'm supposed to pass on to the above bloggers:

• Link your original tagger(s), and list these rules on your blog.
• Share seven facts about yourself in the post - some random, some weird.
• Tag seven people at the end of your post by leaving their names and the links to their blogs.
• Let them know they've been tagged by leaving a comment on their blogs and/or Twitter.

* - The horses don't live with us, thankfully -- it's someone else's job to feed and muck them.