p2 Repository Association And The Fine Art Of Forceably Enabling Disabled Sites 2: Tycho Edition

Back in February, I blogged about how to add associate sites to an update site generated w/ the Buckminster or B3 aggregator.

Since we've moved our build infrastructure to use Tycho + Maven, I had to port the script over to work there for our update site aggregation.

Here are the moving pieces:

• pom.xml that builds the site, listing input sites as <repository> entries, and a shout-out to build.xml, below, to do extra stuff during the install phase
• build.xml ant script called by pom.xml, above, to do additional work after the creation of the site_assembly.zip update site zip
• aggregateSite.jbosstools.properties properties file listing the aggregate sites to add

Update Site Aggregation: B3 vs. Tycho

Last year, I used to aggregate update sites with the Buckminster Aggregator, but since that won't install into Eclipse 3.6 (Helios), I had to migrate to the B3 Aggregator. This new version of the Aggregator is greatly expanded and worked fine until recently, when it has begun to suffer from a rather nasty p2 problem: instead of the Director just installing the aggregator into an Eclipse instance prior to then running the aggregation, I get "The copies of profile SDKProfile are not in sync," and the whole process dies. http://www.blogger.com/img/blank.gif

So, in order to find a workaround, I went back to Tycho, and discovered you can merge update sites with little more than two simple files:

• a site.xml, which lists the features to aggregate and how to categorize them, and
• a pom.xml to list the source sites and drive the aggregation.

Unfortunately the Tycho solution doesn't include the ability to add associate sites to the metadata after generation, but that can simply be done as a downstream step (Tycho can call Ant using the maven-antrun-plugin).

We Don't Need Another Repo

Re Wayne's blog about an Eclipse.org Maven Repo:

EMF has had an undocumented/unmarketed Maven2 repo for about 4 years now.

All you need is to take an update site zip, unpack it, rearrange the folder structure, and rename the jars. Then you create little XML files called .poms to describe the jars in the tree, and Maven-aware tools can read the tree. It's fairly trivial. http://download.eclipse.org/modeling/emf/emf/maven2/ is the URL, IIRC. About once every 2 years someone asks about our providing such a repo, and I give out the URL. Clearly not a huge demand for it.

One might argue that creation of such a folder structure is in the purview of Athena's publishing scripts, which today eases the process of copying your bits to download.eclipse.org, then unpacks your Update site so it can be scanned by p2 rather than downloaded as a single archive. It too is fairly trivial. I would not be adverse to converting my existing shell script for the EMF repo creation into a generic Ant script for use by Athena users.

Frankly though, I think it would be more valuable if the m2eclipse folks added support for reading/converting p2 repos. Publishing yet another file format would require another release-train-like workflow (we already have two: EPP and buckybuilder for Galileo) and more people maintain it. Even if every project published their own maven repo, we'd want for the sake of ease of use to aggregate them into a central place for easier navigation and discovery by maven tools. So, like with Ganymede, we'd have each project's bits copied to two places on disk for each build. (Galileo used composite repos to POINT at project repos rather than copying them saving tons of disk space and CPU cycles. AFAIK, Maven does not support this concept, but I could be wrong.)

There's also another benefit to having tooling to support converting from p2 repo to maven2 repo: the aggregate repo could be housed at apache.org and suck THEIR bandwidth and support resources instead. Thus just as Eclipse.org is upstream from Fedora's Eclipse project .rpms (which are upstream from Debian/Ubuntu's .debs), Eclipse.org p2 repos could be upstream from Apache's Maven repo(s). After all, Apache already collects maven artifacts for non-apache.org projects to facilitate the use and adoption of maven, so this is entirely in line with their standard operating procedures.

Thoughts?

Posted from Blackberry using Opera Mini, by the side of Shawnigan Lake, Vancouver Island, BC

No More Blue Balls

Just a quick note to let people know that I've updated the Eclipse Hudson instance (and you can too, if you're in the Hudson admin group!)

As part of this update, I've installed a plugin to make Hudson look more consistent with other CI tools:

I know it's generally not advisable to do potentially breaking changes like this on a Friday evening, but after a number of ups and downs this week, the Galileo BuckyBuilder is actually green (blue) ...

... and it's my last weekend as a 2⁵ year old, so out with a bang we go.

If I broke anyone's Hudson job w/ this update, let me know - I'll be checking mail over the weekend, just in case.