Much ado about scripting, Linux & Eclipse: card subject to change

Showing posts with label site. Show all posts
Showing posts with label site. Show all posts

2011-02-13

Simplifying The p2 Process, Part 3: Associate Sites

In Part 1 of this series, I looked at use of composite repos to provide a way of combining update sites into a single URL for ease of use and a single point of entry from which to do updates.

In Part 2, I discussed why we switched from using a collection of SDKs against which to build - using the now-deprecated brute-force "just unzip into eclipse root folder or dropins" approach - to using a single target platform update site so as to simplify maintenance and provide a reusable artifact for both build and workspace provisioning.


Now, let's look at the no-brainer that says that "less is more" when it comes to telling p2 from where to get updates, and that less effort for your user when installing is always a win.

If this sounds familar, I did blog about this briefly back in August. Since then, we've also added a quick XSLT script to remove the "Uncategorized" category that Tycho automatically adds for features which are listed in your site.xml but are not associated with a category. While this isn't strictly related to associate sites, it is about ease of use; while I applaud the desire to have everything belong to a category bucket (perhaps because the model Tycho's using requires it), the reason we'd rather hide these is to declutter the install view and not confuse people by suggesting features that won't work on their OS (eg., for which there's no XulRunner port).

But I digress...

Associate Sites

In the old days of yore, you could situate an associateSites.xml next to your site.xml in your "classic" update site, and Eclipse Update Manager would happily read that file and add those extra sites to your list of available update sites.

Then came p2, and while the old way still worked, it was no longer ideal. So, the new approach was to insert these associate sites directly into the p2 metadata for the site, content.xml and artifacts.xml (or content.jar and artifacts.jar).

This could be accomplished via a somewhat hacky appraoch - unpacking the existing metadata (content.jar) and shoehorning in the information at the bottom of the content.xml file, using an ant script (see "add.associate.sites" target, below) and a list of sites to be added:

<target name="add.associate.sites" if="associate.sites">
        <if>
                <and>
                        <!-- Defined in aggregateSite.properties -->
                        <isset property="associate.sites" />
                        <not>
                                <equals arg1="${associate.sites}" arg2="" />
                        </not>
                </and>
                <then>
                        <if>
                                <available file="${update.site.source.dir}/content.jar" type="file" />
                                <then>
                                        <unzip src="${update.site.source.dir}/content.jar" dest="${update.site.source.dir}" />
                                        <delete file="${update.site.source.dir}/content.jar" />
                                </then>
                        </if>
                        <!-- counter variable -->
                        <var name="associate.sites.0" value="" />
                        <for param="associate.site" list="${associate.sites}" delimiter=", 
">
                                <sequential>
                                        <var name="associate.sites.0" value="${associate.sites.0}00" />
                                </sequential>
                        </for>
                        <length property="associate.sites.length" string="${associate.sites.0}" />

                        <loadfile srcfile="${update.site.source.dir}/content.xml" property="content.xml">
                                <filterchain>
                                        <tailfilter lines="-1" skip="1" />
                                </filterchain>
                        </loadfile>
                        <echo file="${update.site.source.dir}/content.xml" message="${content.xml}" />
                        <echo file="${update.site.source.dir}/content.xml" append="true">  <references size='${associate.sites.length}'>
</echo>
                        <for param="associate.site" list="${associate.sites}" delimiter=", 
">
                                <sequential>
                                        <!-- insert into content.xml -->
                                        <echo file="${update.site.source.dir}/content.xml" append="true">    <repository uri='@{associate.site}' url='@{associate.site}' type='0' options='1'/>
<repository uri='@{associate.site}' url='@{associate.site}' type='1' options='1'/>
</echo>
                                </sequential>
                        </for>
                        <echo file="${update.site.source.dir}/content.xml" append="true">  </references>
</repository>
</echo>
          <!--  
    workaround for Tycho bug: uncategorized features in site.xml are put into
    "Uncategorized" category, rather than just being uncategorized (hidden) 
   -->
                 <copy file="${update.site.source.dir}/content.xml" tofile="${update.site.source.dir}/content.old.xml" overwrite="true" />
                        <xslt style="remove-uncategorized.xsl" in="${update.site.source.dir}/content.old.xml" out="${update.site.source.dir}/content.xml" />
                        <zip destfile="${update.site.source.dir}/content.jar" basedir="${update.site.source.dir}" includes="content.xml" />
                        <delete file="${update.site.source.dir}/content.xml" />
                        <delete file="${update.site.source.dir}/content.old.xml" />
                </then>
        </if>
</target>

So, now, instead of telling people to add multiple update sites to resolve missing potentially dependencies when installing, we can cause those extra sites to be automatically added at the same time they add the single URL for JBoss Tools. Now the additional sites need only be listed for reference, but no additional effort is required by the user.

BONUS HACK: to force a site that may already be listed (but disabled) to be added again, and this time definitely be enabled, you can add an extra slash into URL. Thus http://download.eclipse.org/birt/update-site/2.6 becomes http://download.eclipse.org//birt/update-site/2.6/, and as p2 sees a new site, it adds the new site (instead of ignoring it because it's already present but disabled. Again, a win.

Alternatively, you could cast an arcane spell using a p2.inf file in your feature's root folder or plugin's META-INF/ folder to add these additional, required sites... or do whatever processing you might need. I'm not sure if Tycho supports this yet, or how fully PDE supports reading this information. Got sample code? Send it to me as a comment below or via twitter to @nickboldt. Thanks!


In part 4, I'll talk a little about how to prevent your product build from getting updates from unofficial sources, and preload your product with the official sites from which to get updates. Because it's important to balance ease of use with prevention of unsupported features. SPOILER ALERT: may contain p2.inf instructions.

2010-08-11

p2 Repository Association And The Fine Art Of Forceably Enabling Disabled Sites 2: Tycho Edition

Back in February, I blogged about how to add associate sites to an update site generated w/ the Buckminster or B3 aggregator.

Since we've moved our build infrastructure to use Tycho + Maven, I had to port the script over to work there for our update site aggregation.

Here are the moving pieces:

  • pom.xml that builds the site, listing input sites as <repository> entries, and a shout-out to build.xml, below, to do extra stuff during the install phase
  • build.xml ant script called by pom.xml, above, to do additional work after the creation of the site_assembly.zip update site zip
  • aggregateSite.jbosstools.properties properties file listing the aggregate sites to add

2010-06-30

Update Site Aggregation: B3 vs. Tycho

Last year, I used to aggregate update sites with the Buckminster Aggregator, but since that won't install into Eclipse 3.6 (Helios), I had to migrate to the B3 Aggregator. This new version of the Aggregator is greatly expanded and worked fine until recently, when it has begun to suffer from a rather nasty p2 problem: instead of the Director just installing the aggregator into an Eclipse instance prior to then running the aggregation, I get "The copies of profile SDKProfile are not in sync," and the whole process dies. http://www.blogger.com/img/blank.gif

So, in order to find a workaround, I went back to Tycho, and discovered you can merge update sites with little more than two simple files:

  • a site.xml, which lists the features to aggregate and how to categorize them, and
  • a pom.xml to list the source sites and drive the aggregation.

Unfortunately the Tycho solution doesn't include the ability to add associate sites to the metadata after generation, but that can simply be done as a downstream step (Tycho can call Ant using the maven-antrun-plugin).

2010-03-04

Athena: p2 repos w/ associate sites

Want to insert associate update sites into your user's Eclipse when they go to install your features from an update site/zip? With an Athena build this is now a trivial exercise. For example... say you're building a plugin which supports integration with Subclipse or Subversive. In a traditional install from an SDK or runtime zip, you'd unpack into eclipse/, and wonder why the new plugins didn't work. In the new p2 install world, you point Eclipse at the update site or zip, and are told that you cannot install the SVN integration because missing requirements cannot be found. So you hunt down the required sites, add them to Eclipse, and try again; this time you are allowed to install the new features. To improve the ease of use for the end user, Athena now allows you to define associate.sites in your build.properties. If this property is set, URLs will be added into your p2 repo's metadata and Eclipse will automatically add those sites when it scans your zip or site. No more manual copy+paste from documentation needed!

associate.sites=\
http://subclipse.tigris.org/update_1.6.x,\
http://download.eclipse.org/technology/subversive/0.7/update-site/,\
http://community.polarion.com/projects/subversive/download/eclipse/2.0/update-site/
And, of course, you can reuse these sites when resolving dependencies against which to build or test:
repositoryURLs=\
http://download.eclipse.org/releases/galileo/,\
http://download.eclipse.org/athena/repos/eclipse-Update-3.5.2-201002111343.zip,\
http://download.eclipse.org/eclipse/updates/3.5/,\
${associate.sites}

IUsToInstall=org.eclipse.sdk.feature.group+org.eclipse.sdk.ide+\
org.eclipse.team.svn+org.eclipse.team.svn.core+org.eclipse.team.svn.ui+\
org.tigris.subversion.subclipse.core+org.tigris.subversion.subclipse.ui