PDE Best Practices : A Noob's Guide To Plugin & Build Health

I've drafted a collection of tips for keeping your plugins healthy and your builds green.

Feel free to add to, change, or comment on the list here:

http://wiki.eclipse.org/Common_Build_Infrastructure/Getting_Started/FAQ/Plugin_Health_And_Best_Practices

Dash Athena: Eclipse Common Build System / Running Tests On Your System

Bjorn recently kvetched that Eclipse projects met two or three of those goals, but fell down on the "common build system" and "tests run on your system" [1].

While it's true I've seen a number of projects who don't have, don't run, or don't publish their tests, I'm a little disappointed to see Bjorn's no longer committed to the common build solution we've been working on since September 2006 (in earnest since June 2008). We do have a project to solve both those concerns, but like all things at Eclipse, it's powered by YOU. You want it to happen, you have to help. I'm looking for a few good contributors and committers for the Dash Athena project to supplement the great people we already have. Or, if you don't have time to contribute code, you can help by using the system, testing it, opening bugs, enhancing documentation, and blogging about it.

So, what is Dash Athena?

Well, it's a common build system using Hudson and PDE which can also be run commandline on Linux, Windows or Mac, or in Eclipse. It can produce zips of plugins, features, examples, tests, then run those tests. It can also produce update sites with p2 metadata, which can then be published to eclipse.org (or sourceforge.net, for that matter) so everyone can get your bits via Update.

Tests will currently only run on Linux - if you'd like to help us port to Mac OS X and Windows, please step up. The system works with CVS, SVN, and probably Git/Bzr/Hg too, since it supports building from locally checked-out sources and will copy your features/plugins so they're in the format that PDE requires. It supports source input via map files (soon Project Set Files (*.psf), too!) and binary inputs via zips and p2 repos / update sites.

If you aren't sure how to get started w/ an Athena build, please don't hesitate to ask. If you feel the docs are insufficient, incomplete, or inaccurate, let me know - or better - fix them! Want your own Hudson job to run your build? Just open a bug and we'll set you up.

Oh, and incidentally, the irony is not lost on me that I'm using American iconography above even though 5 of the 6 committers on the project are Canucks. :)

No More Blue Balls

Just a quick note to let people know that I've updated the Eclipse Hudson instance (and you can too, if you're in the Hudson admin group!)

As part of this update, I've installed a plugin to make Hudson look more consistent with other CI tools:

I know it's generally not advisable to do potentially breaking changes like this on a Friday evening, but after a number of ups and downs this week, the Galileo BuckyBuilder is actually green (blue) ...

... and it's my last weekend as a 2⁵ year old, so out with a bang we go.

If I broke anyone's Hudson job w/ this update, let me know - I'll be checking mail over the weekend, just in case.

Git 'Er Done

The discussions in bug 257706: Host a git repository on Eclipse Foundation servers, support git as the repository of Eclipse projects rages on, wind blowing in both directions.

Let's look at the objections to Git @ Eclipse:

Implementing a common build infrastructure would also be complicated by additional code repositories as well.

Not true; the Athena system already supports CVS and SVN, plus a "build from local sources" mode which works w/ a cvs/svn tree dump, a workspace w/ checked out projects, or (TBD, we haven't tested this yet) with a git repo. And we have an open bug to make repo tree structure irrelevant to the local checkout mode. Party on. There is even a Git plugin for Hudson so you can use Hudson to watch your repo for changes, like it does with CVS and SVN.

Can't use unapproved or non-EPL code at Eclipse.org

Not true; from discussions w/ legal@eclipse.org, I've been told at least twice that as long as you're not SHIPPING code that falls under a non-EPL or non-approved-CQ you're entirely fine to USE that code as server-based infrastructure. Rock on.

Cannot include tooling in a release train or host its project at Eclipse.org

Not true; since eGit is EPL and jGit is BSD, I don't see a problem with distributing the tooling that would connect to a Git repo hosted at Eclipse.org. We worked around the license woes for SVN tooling support. We can do it again. (Of course IANAL, TINLA.)

Conclusion:

No legal concerns with use of Git as hosted server infrastructure. Dash Athena (Common Builder) will support Git. Tooling is safe for inclusion in release trains (either fully like CVS is or partially like SVN is).

Only remaining issue is therefore to get it installed and allocate resources to support/manage it. With all the erosion going on lately, this need should not be trivialized.

Before the thaw this spring, this tree was on top of the bluff. With nothing to support it, it was dropped like an unchampioned feature request.

However, in the spirit of open source, several people on the above bug have offered to help w/ setup, testing, support, etc. So the burden here will be shared, like many things at Eclipse (eg., Babel, Hudson). Erosion continues, but we can all help to shore up the loss.

As most people prolly already know, Sourceforge supports the whole spectrum of VCS and DVCS options. If we don't want people to host projects there, Eclipse has to at least offer something from the DVCS world to encourage participation here. Keep the barrier to entry high, and people will go elsewhere. Lower the barrier, and people will come here to party instead.

With everyone feeling the economic- and time-pinch these days, can we really afford to discourage contributions at Eclipse simply because, as the silverbacks say, "why, back in my day, we only had CVS, vi, and notepad, and dangit, that was good enough!" ?

After all, the new world is inevitable.

All I really want is builds

With April 20 just around the corner, it's time to get ill... but first, something completely different.

Builds, all I really want is builds
And in the morning it's builds
'Cause in the evening it's builds

I like the way that they run
And when they're green they're so much fun
And I can always make 'em smile
Fetch all their sources and compile

Back in the day
There was this project 'round the way
It had no build system per se
But they kept codin', come what may

I asked them, "Please?" they said, "You may."
Their code was tight and that's ok
If they would code I'd be R.E.
And now it runs on build-dot-E

I hope you'll say, "Hey, me and you should try Athe-
na, more to come, but under way!"
We really need a website, eh?
For now slides, code, and the wi-kay

We'll move to Git when it's read-ay
But that won't be before this May
So get yourself a build today
Without the ol' P-D-E dismay

Builds - to do the fetchin'
Builds - do the compilin'
Builds - to do the testin'
Builds - and zipped-up filin'
Builds, that's all I really want is builds
Early adopters they want builds
In next-gen Hudson they want builds
Signed, packed, p2 repos...
Builds, builds, builds, builds, builds!

NOTE: Normally the artist name below links to the video on YouTube, but there's no official one, and everything else I found (school projects, Lego animation, stick figures, slideshows) were either incomplete, incorrect, or potentially NSFW. YMMV.

Beastie Boys - Girls

EclipseCon '09: Dash Athena Common Builder Workshop: Post Mortem

On behalf of myself, Andrew Overholt and Andrew Niefer, thanks to everyone who attended our workshop on the new Dash Athena common builder.

Looks like the feedback was favourable...

(more stats here)

However, +1 doesn't tell us anything about how we can improve, what you liked, or what could have been done differently. If you have specific feedback, we'd love to hear it. Please post it on the EclipseCon site, or here.

If you missed the talk but want to try this at home, the updated slides and exercises are posted here.

And if you're a project at Eclipse.org and want to start using the common builder, there are three ways to do so:

On your local machine / vserver

This scenario is covered in the workshop - try the exercises for setting up GEF and Linux Tools, then create your own .releng project and try your own build. Additional information on vserver setup can be seen here: Fedora, WinXP, MacOSX.

On build.eclipse.org headlessly

ssh to build.eclipse.org using your committerid (same password as for CVS access). If you can't log in, open a bug and ask for a full login shell. Once there, you can run a build using /opt/public/cbi/build/scripts/start.sh. Run the script without options to learn how it works, and see the examples. If anyone has a problem starting up from scratch, let me know and I'll flesh out some docs in the wiki.

On build.eclipse.org via Hudson

For the moment, you need to be added to the callisto-dev group before you can use Hudson. Open a bug and ask for access. Any other questions, comments, suggestions... file a bug, post a comment, or find us on IRC.