tag:blogger.com,1999:blog-178239792024-03-12T20:50:45.239-04:00DivByZero.comMuch ado about scripting, Linux & Eclipse: card subject to changenickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.comBlogger209125tag:blogger.com,1999:blog-17823979.post-50467916847764032092011-12-11T14:12:00.003-05:002011-12-11T14:15:23.127-05:00Build Nomenclature Conventions: What's in a name?<i>The following post is inspired by Mickael Istria's recent blog, <a href="http://mickaelistria.wordpress.com/2011/12/07/call-a-spade-a-spade-and-a-nightly-a-snapshot/">Call a spade a spade, and a Nightly a Snapshot</a>.</i>
<p>
When I was doing builds for the Eclipse Modeling Project, I-builds were weekly published nightlies -- same level of stability as a SNAPSHOT (to use Maven parlance) or nightly, but published on a weekly schedule to bridge the gap between nightly/daily/SNAPSHOT/CI builds and the every-6-weeks milestone releases. The goal was to provide something stable enough for early adopters to grab once a week, but without the non-stop flux of nightlies. Regardless of the label on the build, the process was the same: tag CVS, then build using that tag.
<p>
The Final/GA/Release ("R") builds were done as simple renames of the last good milestone or release candidate build, so as to ensure binary-compatibility w/ the last-tested milestone/RC. The same was true for "M" and "S" builds -- they were just renamed "I" builds, and the letter was there simply to differentiate between a maintenance build (M), a stable milestone (S), or release (R).
<p>
Branching only happened when a release was done and it was time to produce the maintenance stream vs. the ongoing next-year-release. Sometimes branching would happen AFTER the x.y.1 maintenance because it saved duplication of commits in the x.y+1.0 and x.y.1 streams.
<p>
--
<p>
Now at JBoss, we publish "nightly" builds, which are keyed to SVN changes and therefore could be as often as hourly or as infrequent as weekly, depending on what's happening in the repo.
<p>
We also do milestone builds about once ever 6-8 weeks (similar to the Eclipse.org release train schedules), which is more carefully vetted, tested, and QE'd. It is produced using the same *process* as the nightlies, but are named differently and pulled from a freshly-created stable branch in the repo (so its degree of change/churn is less). (Branching happens right before every milestone or release candidate so that hardening/stabilization/documentation can happen in the branch while trunk stays open for new development.)
<p>
--
<p>
Bottom line -- I've only ever needed three types of builds, regardless of nomenclature or labelling differences. And of these 3, the last 2 are the same thing but renamed to underline the build quality/stability:
<p>
* nightly/CI/integration/weekly/SNAPSHOT build (unstable, for bleeding edge adopters)
<p>
* development milestone (probably a re-christened nightly; stable, early adopters)
<p>
* stable release / Final / GA (probably a re-christened milestone; release quality)
<p>
--
<p>
So... does it matter if it's called nightly, integration or SNAPSHOT? or Stable, Milestone, Maintenance, Final, GA or Release? As long as it's easily reproducible (yeah, Tycho!), what's in a name?nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com1tag:blogger.com,1999:blog-17823979.post-40308052476548710732011-10-26T20:48:00.003-04:002011-10-26T20:51:51.851-04:00HOWTO: Use Maven, Ant, and XSLT to scrub unwanted p2 metadata from an update site<p>Some time ago, <a href="http://divby0.blogspot.com/2011/02/simplifying-p2-process-part-4-using.html">I wrote about Using p2.inf to add/remove update sites</a>. Tonight I found a simpler way to remove references in p2 metadata to external 3rd party sites.
<p>For example, say you're repackaging some 3rd party features onto your own site, but don't want those features to provide references to the vendor's own update sites because you want to ensure that your product's site will only result in your sanctioned version being installed.
<p>When you generate an update site, p2 pulls the information in the included features and will result in a section of references in the site's metadata that looks like this:
<p><pre class="brush:xml">
<references size="6">
<repository uri="http://download.eclipse.org/egit/updates" url="http://download.eclipse.org/egit/updates" type="0" options="0"/>
<repository uri="http://subclipse.tigris.org/update_1.6.x" url="http://subclipse.tigris.org/update_1.6.x" type="1" options="0"/>
<repository uri="http://download.eclipse.org/egit/updates" url="http://download.eclipse.org/egit/updates" type="1" options="0"/>
<repository uri="http://subclipse.tigris.org/update_1.6.x" url="http://subclipse.tigris.org/update_1.6.x" type="0" options="0"/>
<repository uri="http://eclipse.svnkit.com/1.3.x/" url="http://eclipse.svnkit.com/1.3.x/" type="0" options="0"/>
<repository uri="http://eclipse.svnkit.com/1.3.x/" url="http://eclipse.svnkit.com/1.3.x/" type="1" options="0"/>
</references>
</pre>
To remove that, you can play with p2.inf directives, or you can simply perform an XSL transformation on the generated content.xml (inside content.jar, if your metadata is compressed) to remove the <tt><references/></tt> node:
<p><pre class="brush:xml">
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0">
<xsl:template match="/">
<xsl:apply-templates select="*"/>
</xsl:template>
<xsl:template match="*">
<xsl:copy >
<xsl:for-each select="@*">
<xsl:copy />
</xsl:for-each>
<xsl:apply-templates />
</xsl:copy>
</xsl:template>
<xsl:template match="references" />
</xsl:stylesheet>
</pre>
If you're generating your update site w/ Tycho, this transform can be called via a simple Ant script:
<p><pre class="brush:xml">
<target name="remove.references">
<!-- requires ant-contrib only if you like using if-then-else structures -->
<if>
<available file="${update.site.source.dir}/content.jar" type="file" />
<then>
<unzip src="${update.site.source.dir}/content.jar" dest="${update.site.source.dir}" />
<delete file="${update.site.source.dir}/content.jar" />
</then>
</if>
<copy file="${update.site.source.dir}/content.xml" tofile="${update.site.source.dir}/content.old.xml" overwrite="true" />
<xslt style="remove-references.xsl" in="${update.site.source.dir}/content.old.xml" out="${update.site.source.dir}/content.xml" />
<zip destfile="${update.site.source.dir}/content.jar" basedir="${update.site.source.dir}" includes="content.xml" />
<delete file="${update.site.source.dir}/content.xml" />
<delete file="${update.site.source.dir}/content.old.xml" />
</target>
</pre>
Then, in your site's pom.xml, to call the Ant script, do this:
<p><pre class="brush:xml">
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<!-- make sure this variable is defined, eg., set to 1.3 -->
<version>${maven.antrun.plugin.version}</version>
<executions>
<execution>
<id>install</id>
<phase>install</phase>
<configuration>
<quiet>true</quiet>
<tasks>
<!-- called AFTER generating update site + zip to tweak content -->
<ant antfile="build.xml">
<property name="SOME_ANT_VARIABLE" value="${SOME_MAVEN_VARIABLE}" />
</ant>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
<dependencies>
<!-- some dependencies your ant script might need -->
<dependency>
<groupId>commons-net</groupId>
<artifactId>commons-net</artifactId>
<version>1.4.1</version>
</dependency>
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant</artifactId>
<version>1.7.1</version>
</dependency>
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant-nodeps</artifactId>
<version>1.7.1</version>
</dependency>
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant-trax</artifactId>
<version>1.7.1</version>
</dependency>
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant-commons-net</artifactId>
<version>1.7.1</version>
</dependency>
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant-apache-regexp</artifactId>
<version>1.7.1</version>
</dependency>
<dependency>
<groupId>ant-contrib</groupId>
<artifactId>ant-contrib</artifactId>
<version>1.0b3</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
</pre>
<p>I suppose there's probably a way to call a transform directly from Maven w/o the Ant wrapper, but this allows unpacking and repacking of the content.jar to get at the content.xml file.nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com0tag:blogger.com,1999:blog-17823979.post-81891973835754145372011-07-27T11:48:00.003-04:002011-07-27T12:46:36.116-04:00MANIFEST.MF and feature.xml versioning rulesI'm forever forgetting what the rules are for dependency declarations in MANIFEST.MF and feature.xml for <a href="#plugin">osgi plugins</a> and <a href="#feature">features</a>. And Googling often results in frustration rather than an answer. So, because today I actually found a concise list of the rules, I thought I'd repost them here, with some minor edits to help clarify.
<blockquote>
<a name="plugin"></a><h4>OSGi Plugin Version Ranges</h4><p>Dependencies on bundles and packages have an associated <em>version range</em> which is specified using an interval notation: a square bracket
“[” or “]” denotes
an <em>inclusive</em> end of the range and a round bracket
“(” or “)” denotes
an <em>exclusive</em> end of the range. Where one end of the range is to be included and the other excluded, it is permitted to
pair a round bracket with a square bracket.
The examples below make this clear.</p><p>If a single version number is used where a version <em>range</em> is
required this does <em>not</em> indicate a single version, but the range <em>starting</em> from that version and
including all higher versions.</p><p>There are four common cases:
<ul type="disc">
<li><p>A “strict” version range, such as [1.2.3,1.2.3], which
denotes that version and only that version.</p></li>
<li><p>A “half-open” range, such as [1.2.3,2.0.0), which has an inclusive lower limit
and an exclusive upper limit, denoting version 1.2.3 and any version after this, up
to, <em>but not including</em>, version 2.0.0.
</p></li>
<li><p>An “unbounded” version range, such as 1.2.3, which
denotes version 1.2.3 and <em>all</em> later versions.</p></li>
<li><p>No version range, which denotes any version will be acceptable. <span style="font-weight:bold;"><span style="font-style:italic;">NOT RECOMMENDED.</span></span></p></li>
</ul>
</blockquote>
<a href="http://divbyzero.com/eclipse/osgi-concepts-versioning.html">The complete text of the above snippet can be seen here</a> (<a href="http://divbyzero.com/eclipse/osgi-concepts-versioning.pdf">or here as PDF</a>).
<p>Example:
<pre class="brush:shell">
Require-Bundle: org.eclipse.core.runtime;bundle-version="[3.4.0,4.0.0)",
org.eclipse.core.resources;bundle-version="[3.4.0,4.0.0)",
org.eclipse.ui.ide;bundle-version="[3.4.0,4.0.0)",
org.eclipse.ui.navigator;bundle-version="3.5.100",
com.ibm.icu</pre>
<p>See also:
<ul>
<li><a href="http://help.eclipse.org/indigo/topic/org.eclipse.platform.doc.isv/reference/misc/plugin_manifest.html">plugin manifest</a> (plugin.xml)</li>
<li><a href="http://help.eclipse.org/indigo/topic/org.eclipse.platform.doc.isv/reference/misc/bundle_manifest.html">osgi bundle manifest</a> (MANIFEST.MF)</li>
</ul>
<hr/>
<a name="feature"></a>In terms of feature manifest (feature.xml) rules, <a href="http://help.eclipse.org">help.eclipse.org</a> has pretty good documentation, but the most important thing to remember - and what I often have to look up - is how to state the matching rules for required upstream features & plugins.
Experience says it's always better to state things explicitly so there's no downstream guesswork needed and anyone reading your manifest knows EXACTLY what version(s) are required for or compatible with your feature. Plus, while YOU might be using PDE UI to build, someone else might be using Tycho and Maven, and every tool can interpret missing metadata their own way.
<p><b><i>When in doubt, spell it out.</i></b>
<blockquote>Valid values and processing are as follows:
<ul><li>if version attribute is not specified, the match attribute (if specified) is ignored.
<li><span style="font-style:italic;"><span style="font-weight:bold;">perfect</span></span> - dependent plug-in version must match exactly the specified version. If "patch" is "true", "perfect" is assumed and other values cannot be set. <span style="font-style:italic;">[1.2.3,1.2.3]</span>
<li><span style="font-style:italic;"><span style="font-weight:bold;">equivalent</span></span> - dependent plug-in version must be at least at the version specified, or at a higher service level (major and minor version levels must equal the specified version). <span style="font-style:italic;">[1.2.3,1.3)</span>
<li><span style="font-style:italic;"><span style="font-weight:bold;">compatible</span></span> - dependent plug-in version must be at least at the version specified, or at a higher service level or minor level (major version level must equal the specified version). <span style="font-style:italic;">[1.2.3,2.0)</span>
<li><span style="font-style:italic;"><span style="font-weight:bold;">greaterOrEqual</span></span> - dependent plug-in version must be at least at the version specified, or at a higher service, minor or major level. <span style="font-style:italic;">1.2.3</span></ul>
</blockquote>
<a href="http://help.eclipse.org/indigo/topic/org.eclipse.platform.doc.isv/reference/misc/feature_manifest.html">The complete text of the above snippet can be seen here</a>.
<p>Example:
<pre class="brush:xml">
<requires>
<import feature="org.eclipse.m2e.feature" version="1.0.0" match="compatible"/>
<import feature="org.maven.ide.eclipse.wtp.feature" version="0.13.0" match="greaterOrEqual"/>
<plugin id="ch.qos.logback.classic" version="0.9.27.v20110224-1110" match="greaterOrEqual"/>
<plugin id="ch.qos.logback.core" version="0.9.27.v20110224-1110" match="greaterOrEqual"/>
<plugin id="ch.qos.logback.slf4j" version="0.9.27.v20110224-1110" match="greaterOrEqual"/>
<plugin id="org.slf4j.api" version="1.6.1.v20100831-0715" match="compatible"/>
<plugin id="com.ning.async-http-client" version="1.6.3.201106061504" match="equivalent"/>
<plugin id="org.jboss.netty" version="3.2.4.Final-201106061504" match="perfect"/>
<plugin id="org.hamcrest.core" version="1.1.0.v20090501071000" match="equivalent"/>
</requires></pre>nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com0tag:blogger.com,1999:blog-17823979.post-32321242653393612862011-02-16T17:57:00.005-05:002011-03-03T10:24:15.044-05:00Simplifying the p2 Process, Part 4: Using p2.inf to add/remove update sites<p>In <a href="http://divby0.blogspot.com/2011/01/simplifying-p2-process-part-1-p2.html">Part 1</a> of this series, I looked at use of composite repos to provide a way of combining update sites into a single URL for ease of use and a single point of entry from which to do updates.
<p>In <a href="http://divby0.blogspot.com/2011/01/simplifying-p2-process-part-2-target.html">Part 2</a>, I discussed why we switched from using a collection of SDKs against which to build - using the now-deprecated brute-force "just unzip into eclipse root folder or dropins" approach - to using a single target platform update site so as to simplify maintenance and provide a reusable artifact for both build and workspace provisioning.
<p>In <a href="http://divby0.blogspot.com/2011/02/simplifying-p2-process-part-3-associate.html">Part 3</a>, I looked at the idea of associating your repo with its upstream requirement sites, so that end-users need only use a single URL, rather than a half-dozen.
<hr/>
Finally, let's look at how you can use a p2.inf file to remove sites you don't support and add sites you do.
<p>In JBDS 4, we include only two update sites - one for core features, and one for certified third-party extras, so that users will only get official updates from us, rather than from Spring, Eclipse, or anywhere else. Sure, they can <b>manually</b> add other URLs themselves, but that's a bit like pulling off the 'do not remove this tag' tag on a mattress or removing the 'warranty void if removed' sticker on your laptop.
<p>So, first, we remove all the update site and discovery site URLs from our upstream features' feature.xml files, so they don't trickle down into the product.
<p>Next, we use a p2.inf file:
<p><pre class="brush:shell"># To explicitly remove a site, use instructions.unconfigure
instructions.configure=\
org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(type:0,location:https${#58}//www.your.server.com/,name:Core Product Updates);\
org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(type:1,location:https${#58}//www.your.server.com/,name:Core Product Updates);\
org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(type:0,location:https${#58}//www.your.server.com/extras/,name:Extra Product Updates);\
org.eclipse.equinox.p2.touchpoint.eclipse.addRepository(type:1,location:https${#58}//www.your.server.com/extras/,name:Extra Product Updates);\
</pre></p>
<p>Then, to generate a site using that p2.inf instruction, here's a bit of Ant code:
<p><pre class="brush:xml"><echo>Run p2.publisher.UpdateSitePublisher using launcherjar = @{launcherjar}</echo>
<java jar="@{launcherjar}"
fork="true" timeout="10800000" jvm="${java.home}/bin/java" failonerror="true"
maxmemory="256m" taskname="p2"
>
<classpath>
<fileset dir="${builder.build.path}/plugins"
includes="org.eclipse.equinox.launcher_*.jar, org.eclipse.equinox.p2.publisher_*.jar, org.eclipse.equinox.p2.updatesite_*.jar"
/>
<fileset dir="${clean.eclipse.home}/plugins"
includes="org.eclipse.equinox.launcher_*.jar, org.eclipse.equinox.p2.publisher_*.jar, org.eclipse.equinox.p2.updatesite_*.jar"
/>
<pathelement location="${builder.build.path}/plugins" />
<pathelement location="${clean.eclipse.home}/plugins" />
</classpath>
<arg line=" org.eclipse.equinox.launcher.Main -consolelog -application org.eclipse.equinox.p2.publisher.UpdateSitePublisher"
/>
<arg line=" -metadataRepository file:${updateSiteJarDir}/ -metadataRepositoryName "${update.site.product.name} ${update.site.description} Update Site""
/>
<arg line=" -artifactRepository file:${updateSiteJarDir}/ -artifactRepositoryName "${update.site.product.name} ${update.site.description} Artifacts""
/>
<arg line=" -source ${updateSiteJarDir}/" />
<arg line=" -compress -publishArtifacts -reusePack200Files -configs *,*,*" />
</java></pre></p>
<p>Or, put your p2.inf file in the same directory as your build.properties ...
<p><pre class="brush:shell">product=${builderDirectory}/jbds-all.product
runPackager=true
p2.gathering=true
p2.category.site=file:${builderDirectory}/site.xml
# locations. Don't need a baseLocation, the transformedRepoLocation will have what we need
buildDirectory=${product.build.directory}/jbds-all-package
transformedRepoLocation=${product.build.directory}/jbds-all-package/transformed
repoBaseLocation=${product.build.directory}/jbds-all-package/toTransform
# The prefix that will be used in the generated archive.
archivePrefix=studio
# The location underwhich all of the build output will be collected.
collectingFolder=${archivePrefix}
# The list of {os, ws, arch} configurations to build.
configs = linux,gtk,x86 & win32,win32,x86 & linux,gtk,x86_64 & macosx,cocoa,x86 & macosx,cocoa,x86_64
buildId=${product.name}-product-${versionTag}
buildLabel=${buildId}
skipBase=true
skipMaps=true
skipFetch=true</pre></p>
<p> ... and your .product file ...
<p><pre class="brush:xml">
<?xml version="1.0" encoding="UTF-8"?>
<?pde version="3.5"?>
<product name="JBoss Developer Studio for Web and SOA Development" uid="com.jboss.jbds.all" id="com.jboss.jbds.product.product" application="org.eclipse.ui.ide.workbench" version="4.0.0.qualifier" useFeatures="true" includeLaunchers="true">
<configIni use="default">
</configIni>
<launcherArgs>
<programArgs>--launcher.XXMaxPermSize
256m
--launcher.defaultAction
openFile</programArgs>
<vmArgs>-Xms512m
-Xmx1024m
-Dosgi.bundles=reference:file:org.eclipse.equinox.simpleconfigurator_1.0.200.v20100503.jar@1:start
-Dosgi.instance.area.default=@user.home/workspace</vmArgs>
<vmArgsMac>-XstartOnFirstThread -Dorg.eclipse.swt.internal.carbon.smallFonts
-Xdock:icon=../Resources/JBDevStudio.icns</vmArgsMac>
</launcherArgs>
<windowImages/>
<splash
location="com.jboss.jbds.product" />
<launcher name="jbdevstudio">
<solaris/>
<win useIco="true">
<ico path="jbds.ico"/>
<bmp/>
</win>
</launcher>
<vm>
</vm>
<plugins>
</plugins>
<features>
<feature id="com.jboss.jbds.product.feature" version="4.0.0.qualifier"/>
</features>
</product></pre></p>
<p> ... and when generating a product using PDE, that file and its instructions should be read at the correct time.
<p>Hope this series has been helpful! If you have any examples of what you've done with .product or p2.inf files, please feel free to send me a link to your post or the file in your cvs, svn, or git repo. I'd love to see what else you can do with p2 and product builds.
<p>See also:
<ul><li><a href="http://help.eclipse.org/helios/index.jsp?topic=/org.eclipse.platform.doc.isv/guide/p2_customizing_metadata.html">help.eclipse.org - Customizing p2 metadata</a>
<li><a href="http://dev.eclipse.org/viewcvs/viewvc.cgi/org.eclipse.releng.eclipsebuilder/eclipse/buildConfigs/sdk/builder/p2.inf?view=co">p2.inf for building Eclipse SDK</a>
</ul>nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com1tag:blogger.com,1999:blog-17823979.post-33110767911754783682011-02-13T09:00:00.000-05:002011-02-13T09:00:01.129-05:00Simplifying The p2 Process, Part 3: Associate Sites<p>In <a href="http://divby0.blogspot.com/2011/01/simplifying-p2-process-part-1-p2.html">Part 1</a> of this series, I looked at use of composite repos to provide a way of combining update sites into a single URL for ease of use and a single point of entry from which to do updates.
<p>In <a href="http://divby0.blogspot.com/2011/01/simplifying-p2-process-part-2-target.html">Part 2</a>, I discussed why we switched from using a collection of SDKs against which to build - using the now-deprecated brute-force "just unzip into eclipse root folder or dropins" approach - to using a single target platform update site so as to simplify maintenance and provide a reusable artifact for both build and workspace provisioning.
<hr/>
<p>Now, let's look at the no-brainer that says that "less is more" when it comes to telling p2 from where to get updates, and that less effort for your user when installing is always a win.
<p><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSg8M8WH4grbkP1sfqT0yFujtthsXiIlh9cIDt1O46AXycpiR8kb4oA_fpwpuPZt_NuucTx5CuPjDlEllJ_-Hk0rGFXguNxdnIB0pvEwTT4z27ZxIkGyCayqAoI3SOzN_oq75p_g/s1600/win.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 275px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSg8M8WH4grbkP1sfqT0yFujtthsXiIlh9cIDt1O46AXycpiR8kb4oA_fpwpuPZt_NuucTx5CuPjDlEllJ_-Hk0rGFXguNxdnIB0pvEwTT4z27ZxIkGyCayqAoI3SOzN_oq75p_g/s320/win.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5571852112415434946" /></a>
<p>If this sounds familar, I did blog about this briefly <a href="http://divby0.blogspot.com/2010/08/p2-repository-association-and-fine-art.html">back in August</a>. Since then, we've also added a <a href="http://anonsvn.jboss.org/repos/jbosstools/branches/jbosstools-3.2.x/build/aggregate/site/remove-uncategorized.xsl">quick XSLT script</a> to remove the "Uncategorized" category that Tycho automatically adds for features which are listed in your site.xml but are not associated with a category. While this isn't strictly related to associate sites, it is about ease of use; while I applaud the desire to have everything belong to a category bucket (perhaps because the model Tycho's using requires it), the reason we'd rather hide these is to declutter the install view and not confuse people by suggesting features that won't work on their OS (eg., for which there's no XulRunner port).
<p>But I digress...
<h3>Associate Sites</h3>
In the old days of yore, you could situate an <a href="http://download.jboss.org/jbosstools/updates/stable/ganymede/associateSites.xml">associateSites.xml</a> next to your site.xml in your "classic" update site, and Eclipse Update Manager would happily read that file and add those extra sites to your list of available update sites.
<p><a href="http://divby0.blogspot.com/2008/05/ganymede-poster-contest.html">Then came p2</a>, and while the old way still worked, it was no longer ideal. So, the new approach was to insert these associate sites directly into the p2 metadata for the site, content.xml and artifacts.xml (or content.jar and artifacts.jar).
<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://4.bp.blogspot.com/-pJLvuww9JMw/TVM1nZP1J-I/AAAAAAAAGQY/DIJZflknkfU/s1600/p2_Came_From_Beneath_Equinox.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 249px;" src="http://4.bp.blogspot.com/-pJLvuww9JMw/TVM1nZP1J-I/AAAAAAAAGQY/DIJZflknkfU/s320/p2_Came_From_Beneath_Equinox.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5571856114895890402" /></a>
<p>This could be accomplished via a somewhat hacky appraoch - unpacking the existing metadata (content.jar) and shoehorning in the information at the bottom of the content.xml file, using <a href="http://anonsvn.jboss.org/repos/jbosstools/branches/jbosstools-3.2.x/build/aggregate/site/build.xml">an ant script</a> (see "add.associate.sites" target, below) and a <a href="http://anonsvn.jboss.org/repos/jbosstools/branches/jbosstools-3.2.x/build/aggregate/site/aggregateSite.jbosstools.properties">list of sites to be added</a>:
<p><pre class="brush:xml"><target name="add.associate.sites" if="associate.sites">
<if>
<and>
<!-- Defined in aggregateSite.properties -->
<isset property="associate.sites" />
<not>
<equals arg1="${associate.sites}" arg2="" />
</not>
</and>
<then>
<if>
<available file="${update.site.source.dir}/content.jar" type="file" />
<then>
<unzip src="${update.site.source.dir}/content.jar" dest="${update.site.source.dir}" />
<delete file="${update.site.source.dir}/content.jar" />
</then>
</if>
<!-- counter variable -->
<var name="associate.sites.0" value="" />
<for param="associate.site" list="${associate.sites}" delimiter=",
">
<sequential>
<var name="associate.sites.0" value="${associate.sites.0}00" />
</sequential>
</for>
<length property="associate.sites.length" string="${associate.sites.0}" />
<loadfile srcfile="${update.site.source.dir}/content.xml" property="content.xml">
<filterchain>
<tailfilter lines="-1" skip="1" />
</filterchain>
</loadfile>
<echo file="${update.site.source.dir}/content.xml" message="${content.xml}" />
<echo file="${update.site.source.dir}/content.xml" append="true"> <references size='${associate.sites.length}'>
</echo>
<for param="associate.site" list="${associate.sites}" delimiter=",
">
<sequential>
<!-- insert into content.xml -->
<echo file="${update.site.source.dir}/content.xml" append="true"> <repository uri='@{associate.site}' url='@{associate.site}' type='0' options='1'/>
<repository uri='@{associate.site}' url='@{associate.site}' type='1' options='1'/>
</echo>
</sequential>
</for>
<echo file="${update.site.source.dir}/content.xml" append="true"> </references>
</repository>
</echo>
<!--
workaround for Tycho bug: uncategorized features in site.xml are put into
"Uncategorized" category, rather than just being uncategorized (hidden)
-->
<copy file="${update.site.source.dir}/content.xml" tofile="${update.site.source.dir}/content.old.xml" overwrite="true" />
<xslt style="remove-uncategorized.xsl" in="${update.site.source.dir}/content.old.xml" out="${update.site.source.dir}/content.xml" />
<zip destfile="${update.site.source.dir}/content.jar" basedir="${update.site.source.dir}" includes="content.xml" />
<delete file="${update.site.source.dir}/content.xml" />
<delete file="${update.site.source.dir}/content.old.xml" />
</then>
</if>
</target></pre>
<p>So, now, instead of <a href="http://download.jboss.org/jbosstools/updates/JBossTools-3.1.1.GA/">telling people to add multiple update sites</a> to resolve missing potentially dependencies when installing, we can cause those extra sites to be automatically added at the same time they add the single URL for JBoss Tools. Now the <a href="http://www.jboss.org/tools/download/dev#noteBirt">additional sites need only be listed for reference</a>, but no additional effort is required by the user.
<p><B>BONUS HACK</b>: to force a site that may already be listed (but disabled) to be added again, and this time definitely be enabled, you can add an extra slash into URL. Thus <a href="http://download.eclipse.org/birt/update-site/2.6">http://download.eclipse.org/birt/update-site/2.6</a> becomes <a href="http://download.eclipse.org//birt/update-site/2.6/">http://download.eclipse.org//birt/update-site/2.6/</a>, and as p2 sees a new site, it adds the new site (instead of ignoring it because it's already present but disabled. Again, a win.
<p><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://4.bp.blogspot.com/-NdFiEbj5a7w/TVM0cQOVfBI/AAAAAAAAGQM/6DMPnlf40Sg/s1600/let%2Bthe%2Bwookie%2Bwin.jpg"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 242px;" src="http://4.bp.blogspot.com/-NdFiEbj5a7w/TVM0cQOVfBI/AAAAAAAAGQM/6DMPnlf40Sg/s320/let%2Bthe%2Bwookie%2Bwin.jpg" border="0" alt=""id="BLOGGER_PHOTO_ID_5571854823983512594" /></a>
<p>Alternatively, you could cast an <a href="http://en.wikipedia.org/wiki/Spellcasting_101">arcane spell</a> using a <a href="http://wiki.eclipse.org/Equinox/p2/Customizing_Metadata">p2.inf file</a> in your <a href="http://eclipsesource.com/blogs/2009/05/14/pimp-your-p2-profile/">feature's root folder or plugin's META-INF/ folder</a> to add these additional, required sites... or do whatever <a href="http://help.eclipse.org/helios/index.jsp?topic=/org.eclipse.platform.doc.isv/guide/p2_actions_touchpoints.html">processing</a> you might need. <i>I'm not sure if Tycho supports this yet, or how fully PDE supports reading this information. Got sample code? Send it to me as a comment below or via twitter to <a href="http://twitter.com/#!/nickboldt">@nickboldt</a>. Thanks!</i>
<p><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://2.bp.blogspot.com/-63Uq4NtMf0A/TVMuIuUWRfI/AAAAAAAAGP0/VpknjNCghvI/s1600/Spellcasting_101_interface.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 175px;" src="http://2.bp.blogspot.com/-63Uq4NtMf0A/TVMuIuUWRfI/AAAAAAAAGP0/VpknjNCghvI/s320/Spellcasting_101_interface.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5571847891394643442" /></a>
<hr/>
<p>In part 4, I'll talk a little about how to prevent your product build from getting updates from unofficial sources, and preload your product with the official sites from which to get updates. Because it's important to balance ease of use with prevention of unsupported features. <b><i>SPOILER ALERT</i></b>: may contain p2.inf instructions.nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com0tag:blogger.com,1999:blog-17823979.post-21905761174831381682011-02-09T18:25:00.001-05:002011-02-09T18:39:13.121-05:00Simplifying The p2 Process, Part 2: Target Platform ReposIn <a href="http://divby0.blogspot.com/2011/01/simplifying-p2-process-part-1-p2.html">Part 1</a> of this series, I looked at use of composite repos to provide a way of combining update sites into a single URL for ease of use and a single point of entry from which to do updates.
<h3>Defining a Target</h3>
<p>Now, I'd like to talk about how to escape the proliferation of zips needed to establish a target platform. For those unfamiliar with the term "target platform", it's either the installed base against which you're compiling your code, or it's the collection of things you have to install first before you can install something on top of that.
<p>For the JBoss Tools case, we have at least 8 prereqs for installation. Here's what you had to install prior to JBoss Tools 3.1.1:
<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/_i21-98vOfTA/TVC3Vmu2ecI/AAAAAAAAGPk/cpa-TZR1X0A/s1600/Screenshot-target-platform-as-SDKs-and-runtimes.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 234px; height: 205px;" src="http://3.bp.blogspot.com/_i21-98vOfTA/TVC3Vmu2ecI/AAAAAAAAGPk/cpa-TZR1X0A/s320/Screenshot-target-platform-as-SDKs-and-runtimes.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5571154320859494850" /></a>
<p>Now, admittedly, because there is also the <a href="http://download.eclipse.org/releases/ganymede/">Ganymede update site</a>, you don't necessarily need to download and unpack all these zips in order to install JBoss Tools - instead, <a href="https://www.jboss.org/tools/download/installation/update_3_1.html">you need only enable the Ganymede site</a>. (Same story for <a href="http://download.eclipse.org/releases/helios/">Helios</a> and <a href="https://www.jboss.org/tools/download/installation/update_3_2.html">JBoss Tools 3.2</a>.)
<p>However, to do a reproduceable PDE-based build, you still need to create this base install. Traditionally, PDE's approach was to download and unpack these zips into the root of the Eclipse install running the build. <a href="http://wiki.eclipse.org/Athena_Common_Build">Athena</a> attempted to improve on this situation by allowing you to <a href="http://wiki.eclipse.org/Common_Build_Infrastructure/Athena_Progress_Report#2009-09-01">define a list of update sites and IUs</a> (features and/or plugins) which were needed to define the platform. But it was far from portable, and hardly reusable.
<p>Buckminster (later b3) also approached this problem by creating its own markup for defining what sites and what IUs to install, backed by an EMF model. But rather than dealing with a UI to create the model and populate it, I found it more useful to simply generate an instance of the <a href="http://wiki.eclipse.org/Eclipse_b3/aggregator/manual">aggregator</a> model and then use the aggregator to fetch & install IUs. But as the aggregator is simply a wrapper for the underlying p2.mirror and p2.director tasks, you can use those directly too.
<p>But as they say... "<a href="http://en.wikipedia.org/wiki/Don%27t_Bore_Us,_Get_to_the_Chorus!">Don't bore us, get to the chorus!</a>" So, here's some sample code for the various solutions for build-time provisioning.
<ol><li><a href="http://dev.eclipse.org/viewcvs/viewvc.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.dash.common.releng/tools/scripts/aggregateRepos.xml?view=markup&root=Technology_Project">Using the buckminster aggregator</a> (<a href="http://dev.eclipse.org/viewcvs/viewvc.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.dash.common.releng/tools/scripts/aggregateRepos.m2eclipse.properties?view=markup&root=Technology_Project">properties file</a>) - stopped working for us w/ Eclipse 3.6, so we switched to b3
<p>
<li><a href="http://dev.eclipse.org/viewcvs/viewvc.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.dash.common.releng/tools/scripts/aggregateRepos.b3.xml?view=markup&root=Technology_Project">Using the b3 aggregator</a> (<a href="http://dev.eclipse.org/viewcvs/viewvc.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.dash.common.releng/tools/scripts/aggregateRepos.b3.m2eclipse.properties?view=markup&root=Technology_Project">properties file</a>) - stopped worked consistently due to network timeouts resolving deps & fetching IUs.
<p>
<li><a href="http://dev.eclipse.org/viewcvs/viewvc.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.dash.common.releng/tools/scripts/partialMirrorFromRepo.xml?view=markup&root=Technology_Project">Using p2.mirror</a> - underlying p2 ant task for mirroring from one or more repos to local disk
<p>
<li><a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/installation/installJBossTools.xml">Using p2.director</a> - underlying p2 ant task for installing IUs (from local or remote repo) into some target Eclipse
</ol>
<p>So, with these tools, you could create a p2 repo from other repos - mirroring and installing IUs as needed - and even script an installation. But was there a better way?
<h3>Target Platform Definition File</h3>
<p>Enter the target platform definition file (.target). This file contains a list of IUs and the p2 repos from which to provision them. So, it's like a b3 aggregator model, or an <a href="http://dev.eclipse.org/viewcvs/viewvc.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.gef.releng/build.properties?view=markup&root=Technology_Project">Athena build.properties file</a>, but abstracted away from the concept of a build, because it can be used for building but ALSO for provisioning a user's installed Eclipse base.
<p>Unfortunately, the Target Platform Definition File editor in Eclipse 3.6 is less than optimal for large targets, or when your internet connection is suboptimal. So, after fighting with it for a while, filing bugs, and ultimately giving up, I went back to my handy-dandy XML editor (often just vim) to maintain it more simply. So rather than having Eclipse automatically install things based on a .target file, I revert to a workflow that actually works: installing by hand from an update site.
<p>While Buckminster does support .target files (or <a href="http://www.ralfebert.de/blog/eclipsercp/rcp_builds/">so I've read</a>), I didn't want to be dependent on it any more, preferring a more "pure" solution.
<p>So, based on code from Peter Nehrer (<a href="http://twitter.com/#!/pnehrer">@pnehrer</a>), I then wrote <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/target-platform/target2p2mirrorXml.xsl">an XSL transform</a> to <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/target-platform/target2p2mirror.xml">create a p2.mirror script</a> from a .target file, wrapped with <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/target-platform/build.xml">another Ant script</a> (and optionally, a <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/target-platform/pom.xml">Maven pom.xml script</a>).
<p>And why might you care? Well, this .target file can be used to:
<ul><li>Provision a developer's Eclipse, using the Target Platform Definition Editor and a few clicks (when it <a href="http://divby0.blogspot.com/2010/07/troubleshooting-eclipseorg-mirrors.html">doesn't time out</a>)
<li>Provision a developer's Eclipse via script for offline or multiple users (getting the team up to speed)
</ul>
<P>And yes, much (or all) of the above can be done w/ Buckminster and/or b3, if you like that approach.
<p>But I prefer to create the .target as input to a build process, rather than being explicitly tied to one. So, as I noted above, if you have a .target file, you can easily generate a p2 repo, and use that repo to run downstream builds. Now, instead of having a half-dozen zips to download and unpack with every build (using the deprecated and unsupported "<a href="http://wiki.eclipse.org/Equinox_p2_Getting_Started#Dropins">dropins</a>" method) you can use a fully-p2-friendly repo site which contains everything you need to do your builds - whether you're a Hudson server or a developer working at home or offline.
<h3>Benefits</h3>
<ul><li>
Unlike "a collection of zips" this single-source-site <b>can be versioned</b> with each release.
<p><li>It only contains <b>WHAT YOU ACTUALLY NEED</b> rather than extraneous sources and doc and tangential plugins/features you don't. It's a bit like making muffins by first grinding your own flour, but at least you know there's nothing evil in that muffin mix, and you will be able to consistently reproduce the recipe every time, regardless of where you might be on teh interwebz.
<p><li>f you're a keener / beta tester who likes to build against the latest milestone (or even a weekly integration build) of Eclipse 3.next or 4.future, you can use the script above to self-update. So, while the TP itself is a contained snapshot listing the explicit versions of feature groups needed, it can also be run in "<b>get the latest available</b>" mode in order to keep your TP current against some HEAD or trunk development / releases.
<p><li>By splitting the TP out of the build, you can <b>build it upstream</b>. So, where in the past we had one "uberbuild" and an implied TP therein, now we have a TP build job, and it is then shared by the 34 downstream jobs which depend on it for their dependencies.
</ul>
<h3>Shut up and show me the code!</h3>
<p><pre class="brush:shell">
# for the "foo.target" file, build a local target platform repo, fetching the latest versions and updating the .target file
$ ant -f build.xml -DtargetFile=foo.target -DuseLatest=true
# for the "bar.target" file, build a local target platform repo, but fetch only the stated versions of IUs
$ ant -f build.xml -DtargetFile=bar.target -DuseLatest=false
</pre>
<p>That's it. I also wrap the <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/target-platform/build.xml">build.xml ant script</a> w/ <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/target-platform/pom.xml">a pom</a> which allows it to be called from an upstream Maven/Tycho process, but that's nothing more than just calling the script using the antrun plugin (and a few ant dependencies), like this:
<p>
<pre class="brush:xml"><build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<phase>validate</phase>
<configuration>
<tasks>
<ant antfile="build.xml">
<property name="targetFile" value="multiple.target" />
<!-- <property name="repoDir" value="/path/to/where/to/provision/repo"/> -->
</ant>
</tasks>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>commons-net</groupId>
<artifactId>commons-net</artifactId>
<version>1.4.1</version>
</dependency>
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant-commons-net</artifactId>
<version>1.7.1</version>
</dependency>
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant-trax</artifactId>
<version>1.7.1</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build></pre>
<p><b><a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/target-platform/"/>The rest of the code is here</a>.</b>
<hr/>
<p>In part 3, I'll look back at the success we've had using associate sites instead of asking people to manually add 3rd party URLs when installing JBoss Tools. <b><i>SPOILER ALERT</i></b>: one URL is easier for people to use than 6.
<p>In part 4, I'll talk a little about how to prevent your product build from getting updates from unofficial sources, and preload your product with the official sites from which to get updates. Because it's important to balance ease of use with prevention of unsupported features. <b><i>SPOILER ALERT</i></b>: may contain p2.inf instructions.nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com3tag:blogger.com,1999:blog-17823979.post-51541722857026057812011-02-02T16:48:00.005-05:002011-02-02T17:21:22.658-05:00Visualizing OSGi Dependencies<a href="http://divby0.blogspot.com/2011/02/howto-find-osgi-dependencies-in.html">Yesterday</a> I blogged about how to find dependencies in features on plugins or features using a shell script to rip through feature jars.
<p>But maybe you're less commandline, and more visual? Well, it may be over three years old, but there's a way to visualize plugin interdependencies using <a href="http://blog.ianbull.com/2007/08/pde-dependency-view-soc.html">Ian Bull's PDE Dependency View</a>. Frankly, I'm amazed this isn't already a core feature in PDE (and correct me if it is).
<p>To use this tool, simply install it from:
<blockquote><a href="http://download.eclipse.org/eclipse/pde/incubator/visualization/site">http://download.eclipse.org/eclipse/pde/incubator/visualization/site</a></blockquote>
<p>After installing and restarting, hit <b><code>CTRL-3</code></b> and type "Graph" to find the "Graph Plug-In Dependencies View". <i>(It's also available from <b><code>Window > Show View > Other...</code></b> (<b><code>ALT-SHIFT-Q,Q</code></b>) under Plug-in Development, if you prefer to kick it old-school.)</i>
<p>Next, right-click in the view or hit the "Focus on" button in the view, and select the plugin on which you want to focus.
<p>Now you can browse up or down through plugins to explore dependencies.
<p>For example, to see what plugins depend on a given plugin, such as org.eclipse.tm.terminal, click the "Show Callers" button in the view.
<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/_i21-98vOfTA/TUnV-RWyjII/AAAAAAAAGPM/fHg17E_Mi7Q/s1600/Screenshot%2B-%2BPDE%2Bviz%2Bgraph%2Bof%2Bdependencies%2Bon%2Btm.terminal.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 103px;" src="http://3.bp.blogspot.com/_i21-98vOfTA/TUnV-RWyjII/AAAAAAAAGPM/fHg17E_Mi7Q/s320/Screenshot%2B-%2BPDE%2Bviz%2Bgraph%2Bof%2Bdependencies%2Bon%2Btm.terminal.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5569217680007924866" /></a>
<p>Or, to see on which plugins org.jboss.ide.eclipe.as.rse.ui depends, click the "Show Callees" button in the view. You can shift-click on nodes to highlight them for emphasis, or click and drag them around.
<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://1.bp.blogspot.com/_i21-98vOfTA/TUnTg-DW6lI/AAAAAAAAGPE/48v8GIF0WEg/s1600/Screenshot%2B-%2BPDE%2Bviz%2Bgraph%2Bof%2Bdeps%2Bof%2Borg.jboss.ide.eclipse.as.rse.ui.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 240px;" src="http://1.bp.blogspot.com/_i21-98vOfTA/TUnTg-DW6lI/AAAAAAAAGPE/48v8GIF0WEg/s320/Screenshot%2B-%2BPDE%2Bviz%2Bgraph%2Bof%2Bdeps%2Bof%2Borg.jboss.ide.eclipse.as.rse.ui.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5569214977586686546" /></a>nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com1tag:blogger.com,1999:blog-17823979.post-15061366297672971412011-02-01T01:17:00.011-05:002011-02-01T01:37:40.076-05:00HOWTO: Find osgi dependencies in featuresSay you're trying to build something with Tycho & Maven 3 and while resolving dependencies before compilation, you're told:
<blockquote><pre class="brush:shell">[INFO] [Software being installed:
org.eclipse.tm.terminal.local.feature.group 0.1.0.v201006041240-10-7w312117152433,
Missing requirement:
org.eclipse.tm.terminal.local 0.1.0.v201006041322
requires
'bundle org.eclipse.cdt.core 5.2.0'
but it could not be found,
Cannot satisfy dependency:
org.eclipse.tm.terminal.local.feature.group 0.1.0.v201006041240-10-7w312117152433
depends on:
org.eclipse.tm.terminal.local [0.1.0.v201006041322]]
</pre></blockquote>
<p>To quickly verify where this dependency is coming from, you can go look into the feature.xml for the org.eclipse.tm.terminal.local feature jar... but if you don't have it installed, this is somewhat more cumbersome; besides, you then have to unpack the jar before you can look inside it.
<p>And maybe that feature contains a number of OTHER dependencies that you'll also need to resolve in your target platform when building. Sure, there are UI tools to do this within Eclipse, but when you're working on remote servers sometimes UI isn't available.
<p>Workaround? Assuming you have <a href="http://wiki.eclipse.org/Equinox/p2/Ant_Tasks/Partial_Mirroring/Example">a mirror of the update site(s)</a> from which you're trying to resolve the dependency (eg., Helios) or can ssh to dev.eclipse.org, you can simply run a quick shell script to do the investigative work for you:
<blockquote><pre class="brush:shell">$ cd ~/downloads/releases/helios/201009240900/aggregate/; ~/bin/findDepInFeature "*tm*" cdt
./features/org.eclipse.tm.terminal.local_0.1.0.v201006041240-10-7w312117152433.jar
<import feature="org.eclipse.cdt.platform" version="7.0.0" match="greaterOrEqual"/>
<import plugin="org.eclipse.cdt.core" version="5.2.0" match="compatible"/>
<import plugin="org.eclipse.core.runtime"/>
</pre></blockquote>
Where the script looks like this:
<blockquote><pre class="brush:shell">#!/bin/bash
# find plugins/feature deps by searching in some folder for feature jars, and searching through their feature.xml files for dependencies
# 1 - featurePattern - pattern of features to search (eg., "org.eclipse.tptp" or "\*" for all features)
# 2 - dependencyPattern - pattern of plugins/feature deps for which to search (eg., "org.eclipse.tptp.platform.instrumentation.ui")
# 3 - location - directory in which to search, if not "."
if [[ ! $1 ]]; then
echo "Usage: $0 <featurePattern> <dependencyPattern> <location>"
echo ""
echo "Example: $0 tm.terminal cdt"
exit 1
fi
# if no location, look in current dir (.)
if [[ $3 ]]; then location="$3"; else location="."; fi
# if no featurePattern, search all features for dependencyPattern
if [[ ! $2 ]]; then featurePattern="*"; dependencyPattern="$1"; else dependencyPattern="$2"; featurePattern="$1"; fi
rm -fr /tmp/findinfeature/; mkdir -p /tmp/findinfeature/features/
for f in $(find "$location" -type f -name "*${featurePattern}*" | egrep -v "pack.gz|source" | grep features | egrep "${featurePattern}"); do
#echo "$f [$featurePattern, $dependencyPattern]"
unzip -q $f -d /tmp/findinfeature/ feature.xml
# <import feature="org.eclipse.cdt.platform" version="7.0.0" match="greaterOrEqual"/>
# <import plugin="org.eclipse.cdt.core" version="5.2.0" match="compatible"/>
if [[ ! $(cat /tmp/findinfeature/feature.xml | egrep "<import" -A3 | egrep "plugin=|feature=" -A1 -B1 | egrep "\".*${dependencyPattern}[^\"]*\"" -A1 -B1) ]]; then
rm -fr /tmp/findinfeature/feature.xml
else
mv /tmp/findinfeature/feature.xml /tmp/findinfeature/${f}_feature.xml
echo "${f}"
cat /tmp/findinfeature/${f}_feature.xml | egrep "<import" -A3 | egrep "plugin=|feature=" -A1 -B1 | egrep "\".*${dependencyPattern}[^\"]*\"" -A1 -B1
echo ""
fi
rm -fr /tmp/findinfeature/feature.xml
done</pre></blockquote>nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com1tag:blogger.com,1999:blog-17823979.post-71947900889161907692011-01-29T16:40:00.008-05:002011-01-29T18:24:48.695-05:00Simplifying The p2 Process, Part 1: p2 Composite Repos<p>
With the release of <a href="http://community.jboss.org/en/tools/blog/2011/01/27/jboss-tools-32-cr1--killer-of-bugs">JBoss Tools 3.2</a> and <a href="http://devstudio.jboss.com/earlyaccess/">JBoss Developer Studio 4.0</a> just around the corner, you may be thinking to yourself, "Self, how many update sites and SDK zips and runtimes will I need to download THIS time?"
<p>
Or maybe you're thinking, "Self, why is this so damn complicated?"
<p>
Well, folks, we heard your <a href="http://www.kvetch.com/">kvetching</a> and we did something about it.
<h3>Composite Repos</h3>
<p>While this is not a new concept to many, we embraced <a href="http://www.vogella.de/blog/2010/04/06/eclipse-p2-composite-repository/">the composite update site</a> this past year and it's made life a lot easier for iterative, agile development cycles. Last year, JBoss Tools 3.1 was built as a single Hudson job, with a second one for JBoss Developer Studio. This meant that any change in any of the components would cause a build to be launched, and 4-6hrs later, we'd have fresh bits. Yeah, far from ideal.
<p align="center"><a href="http://www.jinx.com/youth/shirts/geek/fresh_data.html"><img src="http://www.jinx.com/Content/Product/1178p_42c_zoomb.jpg" width="465"/></a>
<p>This year, we split up the monolith (and added a few new components!) so that now we have 34 update sites to compose into a single one against which builds can then be built. This composite update site looks like this:
<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://1.bp.blogspot.com/_i21-98vOfTA/TUSL3emByyI/AAAAAAAAGOo/rU88XjI4dIg/s1600/Screenshot-3.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 141px;" src="http://1.bp.blogspot.com/_i21-98vOfTA/TUSL3emByyI/AAAAAAAAGOo/rU88XjI4dIg/s320/Screenshot-3.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5567728824558406434" /></a>
<p><b>compositeArtifacts.xml</b></p>
<p><pre class="brush:xml">
<?xml version='1.0' encoding='UTF-8'?>
<?compositeArtifactRepository version='1.0.0'?>
<repository name='JBoss Tools Staging Repository'
type='org.eclipse.equinox.internal.p2.artifact.repository.CompositeArtifactRepository'
version='1.0.0'>
<properties size='2'>
<property name='p2.compressed' value='true'/>
<!-- get new time w/ `date +%s000` -->
<property name='p2.timestamp' value='1294205433000'/>
</properties>
<children size='34'>
<child location='http://download.jboss.org/jbosstools/builds/staging/jbosstools-3.2_trunk.component--archives/all/repo/'/>
...
<child location='http://download.jboss.org/jbosstools/builds/staging/jbosstools-3.2_trunk.component--ws/all/repo/'/>
<child location='http://download.jboss.org/jbosstools/builds/staging/jbosstools-pi4soa-3.1_trunk/all/repo/'/>
<child location='http://download.jboss.org/jbosstools/builds/staging/jbosstools-teiid-designer-7.1_trunk/all/repo/'/>
<child location='http://download.jboss.org/jbosstools/builds/staging/jbosstools-drools-5.2_trunk/all/repo/'/>
<child location='http://download.jboss.org/jbosstools/builds/staging/jbosstools-savara-1.1_trunk/tools/'/>
<child location='http://download.jboss.org/jbosstools/builds/staging/xulrunner-1.9.1.2/all/repo/'/>
</children>
</repository></pre>
<p><b>compositeContent.xml</b></p>
<p><pre class="brush:xml">
<?xml version='1.0' encoding='UTF-8'?>
<?compositeMetadataRepository version='1.0.0'?>
<repository name='JBoss Tools Staging Repository'
type='org.eclipse.equinox.internal.p2.metadata.repository.CompositeMetadataRepository'
version='1.0.0'>
<properties size='2'>
<property name='p2.compressed' value='true'/>
<!-- get new time w/ `date +%s000` -->
<property name='p2.timestamp' value='1294205433000'/>
</properties>
<children size='34'>
<child location='http://download.jboss.org/jbosstools/builds/staging/jbosstools-3.2_trunk.component--archives/all/repo/'/>
...
<child location='http://download.jboss.org/jbosstools/builds/staging/jbosstools-3.2_trunk.component--ws/all/repo/'/>
<child location='http://download.jboss.org/jbosstools/builds/staging/jbosstools-pi4soa-3.1_trunk/all/repo/'/>
<child location='http://download.jboss.org/jbosstools/builds/staging/jbosstools-teiid-designer-7.1_trunk/all/repo/'/>
<child location='http://download.jboss.org/jbosstools/builds/staging/jbosstools-drools-5.2_trunk/all/repo/'/>
<child location='http://download.jboss.org/jbosstools/builds/staging/jbosstools-savara-1.1_trunk/tools/'/>
<child location='http://download.jboss.org/jbosstools/builds/staging/xulrunner-1.9.1.2/all/repo/'/>
</children>
</repository></pre>
<p>So, now that JBoss Tools is built in 34 pieces, the bits that haven't changed aren't rebuilt over and over and builds are faster. If that sounds insanely obvious to you, well, we used to have a lot of inter-component cyclic dependencies. We eliminated those early in the development cycle for JBoss Tools 3.2, and have been able to build smarter and faster ever since.
<p>Added benefits to this composite site are:
<ul><li>Newly built and published bits are instantly available from the composite site - sure, the same was true under last year's PDE "uberbuild" regime, but that's because everything was built fresh every time, which was slow and near-impossible to get people to run at home.
<p>
<li>Developers can use this site to install latest updates to components they're interested in testing - again, this was true before; but now using the same site and searching for updates, developers and beta testers can get incremental updates to the components that have actually changed, rather than having to pull down 160M every day to get a few K of changes.
<p>
<li>Tycho can be pointed at this site (see below) in order to resolve binary p2 dependencies, so building a component deep in the dependency chain can be done w/o having to first build its upstream dependencies - this wasn't a concern before because everything was built from source every time, so by definition everything was already on disk. But now, if a developer only cares about a single component, like ModeShape or GWT, they need only have that source (and some bootstrapping code) on disk. Smaller, faster, more agile. And way more likely to be built locally before checking in code than before, making the painful "who broke what and when?" process much less painful. Fewer moving pieces and local dev builds at home mean - in theory - fewer incomplete or breaking commits.</ul>
<p>When we first moved to Tycho, we needed to build a series of components locally in order to just get to a deep component. For example, the Struts component needs VPE, which needs JST and <a href="http://divby0.blogspot.com/2010/04/build-xulrunner-1912-update-site-with.html">XulRunner</a>. JST also needs the Common component, which in turn needs the Tests component.
<p>So, to build Struts locally, 5 other components would have to be built locally first. This worked, but was still a fairly large barrier to entry for most developers (much less contributors!)
<p>But with this new composite site, building Struts can be done without this lengthy bootstrapping; instead we just point Tycho at this composite site, and it pulls down the 5 upstream components' jars from this p2 repo - because the upstream deps are already built in Hudson.
<p>Here's what we added to our <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/parent/pom.xml">parent pom.xml</a> to have the builds find the binaries:
<p><pre class="brush:xml">
<repository>
<id>jbosstools-nightly-staging-composite-trunk</id>
<url>http://path.to.the.site/staging/_composite_/trunk/ </url>
<layout>p2</layout>
<snapshots>
<enabled>true</enabled>
</snapshots>
<releases>
<enabled>true</enabled>
</releases>
</repository></pre>
<p>So, using this composite update site, we can use Maven 3 with Tycho 0.10 to <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/aggregate/site/">generate a single update site</a> (<a href="http://download.jboss.org/jbosstools/builds/staging/jbosstools-3.2_trunk.aggregate/">staged here</a>, then ultimately <a href="https://www.jboss.org/tools/download/dev">published here</a>).
<hr/>
<p>In part 2, I'll look at why we switched from using a collection of SDKs (Eclipse, EMF, DTP, GEF, M2E, RSE, TPTP, UMl2, WTP, XSD and more) against which to build - using the now-deprecated brute-force "just unzip into eclipse root folder or dropins" approach - to using a single target platform update site. <i><b>SPOILER ALERT</b></i>: Easier to update and maintain.
<p>In part 3, I'll look back at the success we've had using associate sites instead of asking people to manually add 3rd party URLs when installing JBoss Tools. <i><b>SPOILER ALERT</b></i>: one URL is easier for people to use than 6.
<p>In part 4, I'll talk a little about how to prevent your product build from getting updates from unofficial sources, and preload your product with the official sites from which to get updates. Because it's important to balance ease of use with prevention of unsupported features. <i><b>SPOILER ALERT</b></i>: may contain p2.inf instructions.
<p>By the way, <a href="http://community.jboss.org/en/tools/blog/2011/01/27/jboss-tools-32-cr1--killer-of-bugs">JBoss Tools 3.2.0.CR1</a> and <a href="http://devstudio.jboss.com/earlyaccess/">JBoss Developer Studio 4.0.0.CR1</a> are available. Get 'em while they're hot (and <a href="http://sourceforge.net/apps/wordpress/sourceforge/2011/01/27/service-downtime/">sourceforge is not</a>).nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com0tag:blogger.com,1999:blog-17823979.post-23188948370417475872010-12-30T23:14:00.015-05:002010-12-31T00:18:37.855-05:00Using Git like CVS<p>
Tonight I moved the <a href="http://rainbowcinemas.ca">http://rainbowcinemas.ca</a> sources from phpeclipse and CVS to PDT and Git.
<p>Below are some gotchas and tips for initial repo creation, how to keep the central remote copy up to date, and how to work around complaints about updating master directly from remote. I'm sure there's a better way to do this w/o the need for the workaround, but this is what I found worked.
<table><tr valign="top"><td colspan="2">
<h4 style="color:#994444">Initial setup</h4>
<p>To crawl a directory and create a git project for each subfolder:
<blockquote><pre>for d in $(find . -maxdepth 1 -type d -not -name "." | \
egrep -v ".ssh|logs|OLD|download|upload"); do cd $d; \
<b>git init; git add .; git commit -m "initial commit" .</b>; \
cd ..; \
done</pre></blockquote>
<p>See also <a href="http://www.kernel.org/pub/software/scm/git/docs/user-manual.html#setting-up-a-shared-repository">Setting up a shared repository</a>.
</td>
</tr><tr valign="top"><td>
<h4 style="color:#994444">Create local clone via commandline</h4>
<p><b>git clone ssh://servername/path/to/.git folderToCreateLocally</b>
</td><td>
<h4 style="color:green">Create local clone w/ eGit</h4>
<p>Once your repo is created, you can clone a copy from the remote server onto your local box, and import it into Eclipse (with eGit installed) using <b>File > Import > Git > Projects from Git > Clone... </b>.
</td></tr>
<tr valign="top"><td>
<h4 style="color:#994444">Commit local changes via commandline</h4>
<p><a href="http://divby0.blogspot.com/2010/11/git-vs-svn-basic-commandline-syntax.html">As outlined before</a>, you can <span style="font-weight:bold;">git pull</span>, <span style="font-weight:bold;">git checkout</span>, <span style="font-weight:bold;">git commit</span>, and finally <span style="font-weight:bold;">git push</span> your changes.
<p>If you encounter an error trying to commit changes back to the repo, see the section below, "Allow a ref update to the currently checked out branch of a non-bare repository".
</td>
<td>
<h4 style="color:green">Commit local changes w/ eGit</h4>
<p>With eGit, you can pull, push, checkout, commit, merge, etc. using the context menu on a Git project or with the Synchronize view. I don't recommend using any of the change sets / models except the Git Change Set, since the others will tend to show more than is actually needed (like local changes which Git doesn't track).
</td></tr>
<tr valign="top"><td colspan="2">
<h4 style="color:#994444">Allow a ref update to the currently checked out branch of a non-bare repository</h4>
Update the <a href="http://www.kernel.org/pub/software/scm/git/docs/git-config.html"><b>~/.gitconfig</b></a> file on the remote server to look something like this:
<blockquote><pre>[user]
name = Your Name
email = your@email.address
[color]
branch = auto
diff = auto
interactive = auto
status = auto
[core]
editor = vim
[merge]
tool = vimdiff
[receive]
<b>denyCurrentBranch = warn</b></pre></blockquote>
<h4 style="color:#994444">Retrieve changes into remote repo</h4>
<p>Because I'm using the remote server to both host http-accessible files AND host the git repo, it's important that changes checked into the git repo be then checked out into the local filesystem so that the local workspace is in synch with the repo's metadata.
<p>To pull changes, I use <span style="font-weight:bold;">git status</span> (to review changes), <span style="font-weight:bold;">git reset HEAD <filename></span> (to reset specified file, or omit filename to reset all changes) and finally <b>git checkout</b> to retrieve the changed file from the repo into the working directory.
<h4 style="color:#994444">Access the server w/o a password prompt</h4>
<p>To skip being prompted for a password when you connect over ssh, scp, or fish, add your public SSH key to the <a href="http://www.eng.cam.ac.uk/help/jpmg/ssh/authorized_keys_howto.html"><b>~/.ssh/authorized_keys</b></a> file on the remote server.
<h4 style="color:#994444">Access the server via an alias</h4>
<p>Instead of having to reference the server as username@server when connecting, you can add an entry to your <a href="http://linux.die.net/man/5/ssh_config"><b>~/.ssh/config</b></a> file that looks like this:
<blockquote><pre>Host shortName
Hostname fully.qualified.domain.name.or.IP.address
User yourUsername
Port 22</pre></blockquote>
</td></tr></table>nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com4tag:blogger.com,1999:blog-17823979.post-10885331858873789812010-11-15T11:46:00.006-05:002010-11-15T12:59:35.367-05:00HOWTO: Contributing to JBoss Tools using Git<p>If you'd like to use Git instead of SVN as your SCM tool of choice, here's how you can connect to the JBoss Tools SVN repo, pull down all the sources, work on them locally, then either commit changes back into the SVN repo (or submit a patch, if you're not already a committer).
<p><i>The instructions below assume you have either Linux, Mac OSX, or Windows w/ <a href="http://www.cygwin.com/">cygwin</a>. If you have none of those, YMMV.</i>
<h3>Fetch sources from SVN</h3>
<p>First, fetch sources from SVN using <a href="http://www.kernel.org/pub/software/scm/git/docs/git-svn.html">git-svn</a>. If you don't want to check out all the components, use a subset of the components listed below. <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/">The complete list is here</a>.
<blockquote><pre>
# create a directory into which to check out the JBoss Tools projects
mkdir ~/trunk; cd ~/trunk;
# fetch projects - this will take quite some time
# Committers, use http://svn.jboss.org/repos/jbosstools/trunk/
# Contributors, use http://anonsvn.jboss.org/repos/jbosstools/trunk/
for d in \
archives as birt bpel bpmn build cdi common \
deltacloud documentation download.jboss.org drools \
esb examples flow freemarker gwt hibernatetools \
jbpm jmx jsf jst maven modeshape portlet profiler \
requirements runtime seam site smooks struts \
tests tptp usage vpe ws xulrunner; do \
git svn clone http://anonsvn.jboss.org/repos/jbosstools/trunk/${d};
done</pre></blockquote>
<h3> Configure Eclipse </h3>
<p>Next, fire up <a href="http://www.eclipse.org/downloads/packages/eclipse-ide-java-ee-developers/heliossr1">Eclipse Helios 3.6 for Java EE Developers</a>.
<p>Install the latest eGit from <a href="http://download.eclipse.org/egit/updates">http://download.eclipse.org/egit/updates</a>.
<p>Install the latest m2eclipse from <a href="http://m2eclipse.sonatype.org/sites/m2e/">http://m2eclipse.sonatype.org/sites/m2e/</a> and optionally, <a href="http://m2eclipse.sonatype.org/sites/m2e-extras/">http://m2eclipse.sonatype.org/sites/m2e-extras/</a>.
<p> Restart when prompted.
<h3> Import Git projects into Eclipse</h3>
<p>Now, import Git projects into Eclipse using:
<blockquote><pre>File > Import
Git > Projects from Git
Click 'Add' then browse for ~/trunk/
Enable [x] Look for nested repositories
Click 'Search', then click 'OK' when done
</pre></blockquote>
<p>
<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://2.bp.blogspot.com/_i21-98vOfTA/TOFtgGW-giI/AAAAAAAAGG4/vSdBC9rLcW8/s1600/Screenshot.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 309px;" src="http://2.bp.blogspot.com/_i21-98vOfTA/TOFtgGW-giI/AAAAAAAAGG4/vSdBC9rLcW8/s320/Screenshot.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5539829414872646178" /></a>
<blockquote><pre> Select a local repo from the list, click 'Next'
(*) Import Existing Projects
(*) Try to share automatically
Click 'Next'
Click 'Select All', then click 'Finish'
</pre></blockquote>
<p>
<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjA99guHOOgJ-WLUr-Tk9xE3yoPj4KPQAEfcCX9rtXqAIonLWdCREI5r5-fBAarx07DKpa7EXUl3KRUnokV9Ft7WaWbcKt3v9CXubYLKBVy8S3mgcTfgl5MDylYoY85jPp-Gkd66A/s1600/Screenshot-1.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 256px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjA99guHOOgJ-WLUr-Tk9xE3yoPj4KPQAEfcCX9rtXqAIonLWdCREI5r5-fBAarx07DKpa7EXUl3KRUnokV9Ft7WaWbcKt3v9CXubYLKBVy8S3mgcTfgl5MDylYoY85jPp-Gkd66A/s320/Screenshot-1.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5539829421534876210" /></a>
<p>Repeat for other components you want to import. You can add each component to a working set to keep your workspace sorted by component.
<h3>Resolve missing dependencies</h3>
<p>While the <a href="http://www.eclipse.org/downloads/packages/eclipse-ide-java-ee-developers/heliossr1">Eclipse Helios 3.6 for Java EE Developers</a> contains most of the dependencies against which JBoss Tools must be compiled, it does not contain everything. For that, you need to install extra dependencies. There are two places to go:
<p>
<ol><li><a href="http://download.jboss.org/jbosstools/updates/target-platform/latest/">JBoss Tools Target Platform p2 Repo</a> (also <a href="http://download.jboss.org/jbosstools/updates/target-platform/e361-wtp322.target.zip">available as an archived update site zip</a> for offline use) - contains all the Eclipse.org, google.com, and sonatype.org features needed to compile / install all of JBoss Tools. You can install everything, or just the pieces you need.
<li><a href="http://download.jboss.org/jbosstools/updates/nightly/trunk/">JBoss Tools Nightly Repo</a> (Update Site) - if you don't have all the source projects in your workspace, you can resolve dependencies against this site and install them from here. Once again, you can install everything, or just the pieces you need.
</ol>
<h3>Build & run tests</h3>
<p>With m2eclipse installed, you can simply right-click a project and select '<b><code>Run As > Maven Build (ALT-SHIFT-X, M)</code></b>', which will prompt you complete a run configuration dialog. Here are the simplest options you need to set:
<blockquote><pre>
Goals: clean install
[x] Resolve Workspace artifacts</pre></blockquote>
<p><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://1.bp.blogspot.com/_i21-98vOfTA/TOF0QpxXRyI/AAAAAAAAGHM/61jA14YLrhw/s1600/Screenshot-2.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 277px; height: 320px;" src="http://1.bp.blogspot.com/_i21-98vOfTA/TOF0QpxXRyI/AAAAAAAAGHM/61jA14YLrhw/s320/Screenshot-2.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5539836846082049826" /></a>
<p>You can also run <a href="http://community.jboss.org/wiki/HowtoBuildJBossToolswithMaven3">Maven to build your projects outside Eclipse</a>, if you prefer.
<p>If running outside Eclipse, <a href="https://www.jboss.org//tools/docs/testing#remote">you can run tests which are still tied to the Eclipse debugger</a>.
<h3>Commit changes to master repo</h3>
<p>Because <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=315264">there's no support yet</a> for '<b><code>git svn rebase</code></b>' or '<b><code>git svn dcommmit</code></b>' you're stuck pushing changes to the master repo using the commandline. However, you can shorten the amount of typing needed using an .alias file. See below.
<h3>Use an .alias file</h3>
<p>To avoid having to type the same git commands over and over, I use these <a href="http://en.wikipedia.org/wiki/Alias_%28command%29">shortcuts</a> in my ~/.alias file:
<blockquote><pre>
# update local git-svn repo from master SVN repo
alias gitup='for d in $(find . -maxdepth 1 -type d); do cd $d; echo $d; if [[ -d .git ]]; then git svn rebase; fi; cd -; done'
# Push local changes to master SVN repo
alias gp='git svn rebase; git svn dcommit'
# commit local changes to local git-svn repo
alias ci='git commit -m'
# check status of local git-svn repo
alias stat='git status'</pre></blockquote>
<p>So, after committing changes (with eGit or via commandline) I can push those to the master SVN repo w/ a simple '<b><code>gp</code></b>'. If your shell doesn't read the .alias file, make sure your .bashrc loads the file using one of these commands:
<blockquote><pre>source /home/yourUserName/.alias
. /home/yourUserName/.alias
</pre></blockquote>
Or, put them directly in your <a href="http://tldp.org/LDP/abs/html/sample-bashrc.html">.bashrc</a> file.nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com0tag:blogger.com,1999:blog-17823979.post-32881893137404504662010-11-03T07:54:00.007-04:002010-11-05T16:34:36.604-04:00My First Maven Plugin<p>Having been working with Maven and Tycho for about the last 8 months, it was high time I got around to writing my first Maven plugin, because I need to break myself from my tendency to revert to ant and bash whenever I need to do something outside the normal Maven flow.
<p>So, first I imported the git sources for Tycho 0.11.0-SNAPSHOT from <a href="https://github.com/sonatype/sonatype-tycho">https://github.com/sonatype/sonatype-tycho</a> so I'd have something from which to learn. Sure, Hello World samples are nice, but a working example is always more fruitful.
<blockquote><pre>git clone http://github.com/sonatype/sonatype-tycho.git
</pre></blockquote>
<p><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://2.bp.blogspot.com/_i21-98vOfTA/TNFRUsb4oYI/AAAAAAAAGGM/bEEUzedJ020/s1600/Screenshot-1.png"><img style="float:right; margin:0 0 10px 10px;cursor:pointer; cursor:hand;width: 120px; height: 200px;" src="http://2.bp.blogspot.com/_i21-98vOfTA/TNFRUsb4oYI/AAAAAAAAGGM/bEEUzedJ020/s200/Screenshot-1.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5535294832982401410" /></a>
<p>Next, I imported a few Tycho projects (tycho-p2-facade, tycho-p2-plugin, tycho-p2-publisher-plugin) from the git clone folder into Eclipse using <a href="http://m2eclipse.sonatype.org/">m2eclipse</a> 0.10.2. Because these are Maven projects <strike>and don't need Eclipse project clutter, there are no .project files and therefore they couldn't be imported as pre-existing projects.
<p>So, I had to wobble through the New > Maven Project wizard a little until I figured out that I could check the 'Create a simple project (skip archetype selection)' box to create a new .project file in an existing project folder. Only wrinkle here is that if there's an existing pom.xml in a project, the New Maven Project wizard complains. The obvious workaround is to rename the existing pom.xml to pom.xml_, create the new Maven Project in w/ m2eclipse in the existing folder (as above), then replace the pom.xml that m2eclipse creates with the actual (renamed) one, pom.xml_ and refresh the project in Eclipse. Voila! I also updated the project settings to use JDK5 instead of the default JDK1.4.</strike> You need only do <b>File > Import > Maven > Existing Maven Projects</b> and browse for the pom files. So much easier than what I did above. <i>Thanks to everyone for setting me straight!</i>
<p><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://1.bp.blogspot.com/_i21-98vOfTA/TNFTTIr9vrI/AAAAAAAAGGY/4dvdmm7f9Mw/s1600/Screenshot-2.png"><img style="float:right; margin:0 0 10px 10px;cursor:pointer; cursor:hand;width: 126px; height: 200px;" src="http://1.bp.blogspot.com/_i21-98vOfTA/TNFTTIr9vrI/AAAAAAAAGGY/4dvdmm7f9Mw/s200/Screenshot-2.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5535297005229555378" /></a>
<p>So, with sample code to clone from it was time to create my own plugin project, then a second project on which that plugin depended in order to test the dependency chain thru Maven.
<p>The first uses the "maven-archetype-plugin" archetype v1.1; the second, the "maven-archetype-quickstart" archetype v1.1. Here are the two projects:
<p><ol><li><a href="https://github.com/nickboldt/sample-maven-plugin/tree/master/sample-maven-plugin/">sample-plugin</a>
<li><a href="https://github.com/nickboldt/sample-maven-plugin/tree/master/sample-maven-project-depends-on-plugins/">sample-project</a>
</ol>
<p>The glue between these projects is in the sample-project's pom.xml:
<p><blockquote><pre><dependency>
<groupId>org.jboss.maven.plugin</groupId>
<artifactId>sample-plugin</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency></pre></blockquote>
And that's seemingly it. Next... adding actual useful functionality to a plugin, then using that functionality in our <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/parent/pom.xml">JBoss Tools build's parent pom.xml</a>.nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com4tag:blogger.com,1999:blog-17823979.post-891365397752289872010-10-08T00:05:00.003-04:002010-10-08T00:28:00.577-04:00HOWTO: Find the feature that contains a plugin<p>Tycho is awesome.
<p>However, like all build systems, it has its limitations.
<p>One such limitation is that when you're building against a target platform, and something's missing, you get errors such as these:
<p><blockquote><pre>[INFO] Cannot complete the request. Generating details.
{org.osgi.framework.executionenvironment=OSGi/Minimum-1.0,OSGi/Minimum-1.1, osgi.ws=cocoa, osgi.arch=x86, osgi.os=macosx, org.eclipse.update.install.features=true, org.osgi.framework.system.packages=}
[Software being installed: org.jboss.tools.tptp.feature.feature.group 1.2.0.qualifier, Missing requirement: org.eclipse.tptp.platform.instrumentation.ui 4.4.1.v201009092123 requires 'bundle org.eclipse.hyades.probekit [4.2.0,5.0.0)' but it could not be found, Cannot satisfy dependency: org.eclipse.tptp.platform.instrumentation.ui.feature.group 4.3.1.v201009092123-797908s73533D4H6D56 depends on: org.eclipse.tptp.platform.instrumentation.ui [4.4.1.v201009092123], Cannot satisfy dependency: org.jboss.tools.tptp.feature.feature.group 1.2.0.qualifier depends on: org.eclipse.tptp.platform.instrumentation.ui.feature.group 4.3.0]
[ERROR] Internal error: java.lang.RuntimeException: org.eclipse.equinox.p2.core.ProvisionException: No solution found because the problem is unsatisfiable. -> [Help 1]
org.apache.maven.InternalErrorException: Internal error: java.lang.RuntimeException: org.eclipse.equinox.p2.core.ProvisionException: No solution found because the problem is unsatisfiable.
</pre></blockquote>
<p>The important part of that error message is as follows:
<p><blockquote><pre>org.jboss.tools.tptp.feature.feature.group 1.2.0.qualifier
requirement: org.eclipse.tptp.platform.instrumentation.ui 4.4.1.v201009092123
<b>requires 'bundle org.eclipse.hyades.probekit [4.2.0,5.0.0)'
but it could not be found</b>
org.eclipse.tptp.platform.instrumentation.ui.feature.group 4.3.1.v201009092123-797908s73533D4H6D56
depends on: org.eclipse.tptp.platform.instrumentation.ui [4.4.1.v201009092123]
dependency: org.jboss.tools.tptp.feature.feature.group 1.2.0.qualifier
depends on: org.eclipse.tptp.platform.instrumentation.ui.feature.group 4.3.0]
</pre></blockquote>
<p>So, how do you find which feature contains that plugin, so that you can add it to your target platform?
<p>First, you need access to the repository. If you have direct server access to the repository from which the plugin comes (eg., the TPTP Helios update site), you can run <a href="http://divbyzero.com/linux/findInFeature.sh.txt">this script</a> in the root of that repository.
<p>If you don't have server access (eg., you can't ssh to dev.eclipse.org and look in ~/downloads/tptp/updates/helios), then you can pull down a zip of the site (or use a <a href="http://wiki.eclipse.org/Equinox/p2/Ant_Tasks/Partial_Mirroring/Example">p2.mirror script</a> to fetch a copy of the site to your local machine)... and then run <a href="http://divbyzero.com/linux/findInFeature.sh.txt">this script</a> in the root of that repository.
<p>Essentially the script finds matching feature jar files, unpacks them to extract the feature.xml files therein, and then greps those files for lines which suggest an included plugin matching the pattern for which you're searching:
<blockquote><pre>
$ findInFeature platform.probekit
./features/org.eclipse.tptp.platform.probekit_4.5.1.v201009092123-7H7BF8PAkF7B77ZARCNEK.jar
<plugin
id="org.eclipse.tptp.platform.probekit"
./features/org.eclipse.tptp.platform.trace_4.5.1.v201009092123-7L7O8bBgJ9E99jAfGWEM.jar
<plugin
id="org.eclipse.tptp.platform.probekit.launch"
</pre></blockquote>
From there, it's a trivial exercise to add another line item into your <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/target-platform/e361-wtp322.target">target platform</a> file. First, paste in the feature jar:
<blockquote><pre>
./features/org.eclipse.tptp.platform.probekit_4.5.1.v201009092123-7H7BF8PAkF7B77ZARCNEK.jar
</pre></blockquote>
<p>Then use vim to pattern-replace that string:
<blockquote><pre>
:%s/.\+\/\(org.\+\)_\(\d\+.\+\)\.jar/\t\t\t<unit version="\2" id="\1.feature.group"\/>/g
</pre></blockquote>
<p>And you'll end up with a new .feature.group added to the target:
<blockquote><pre>
<unit version="4.5.1.v201009092123-7H7BF8PAkF7B77ZARCNEK" id="org.eclipse.tptp.platform.probekit.feature.group"/>
</pre></blockquote>nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com3tag:blogger.com,1999:blog-17823979.post-51550632576071999842010-10-01T22:43:00.008-04:002010-10-01T23:28:41.193-04:00JBoss Tools: making it easier to build against a complex target platform<p>So you want to be a JBoss Tools developer? Awesome. Welcome to the family. <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/">SVN sources are here</a>, <a href="https://jira.jboss.org/browse/JBIDE">JIRA's over here</a> and there's cold beer in the fridge*.
<p>But you say it's a pain in the tuchus to <a href="http://download.jboss.org/jbosstools/requirements/helios/">download over 25 zips</a> or add a whole bunch of <a href="https://www.jboss.org/tools/download/dev.html#noteBirt">update sites</a> and hope you get everything you need? Yeah, no argument there. If only there was an easier way to resolve all the dependencies you need to get building, much less to even RUN this stuff.
<p>To make this process simpler, I've created a <a href="http://download.jboss.org/jbosstools/updates/target-platform/latest/">p2 repo</a> (update site) from our <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/target-platform/e361-wtp322.target">target platform file</a>, which has been recently updated to include Helios SR1 dependencies. You can track subsequent work in progress here: <a href="https://jira.jboss.org/browse/JBIDE-6982">JIRA JBIDE-6982</a>. You can also report any issues there too.
<hr/>
<p>So, now, just add this <a href="http://download.jboss.org/jbosstools/updates/target-platform/latest/">single site</a>** into your vanilla Eclipse 3.6.1 Classic (or a Helios SR1 bundle), uncheck the box for 'Group Items by Category' and you can install everything listed. For great justice.
<p><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://1.bp.blogspot.com/_i21-98vOfTA/TKadr2A10mI/AAAAAAAAGGA/NqA_ZrH_wBw/s1600/Screenshot.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 214px; height: 320px;" src="http://1.bp.blogspot.com/_i21-98vOfTA/TKadr2A10mI/AAAAAAAAGGA/NqA_ZrH_wBw/s320/Screenshot.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5523275369575469666" /></a>
<hr/>
<p>Some handy links:
<ul><li><a href="http://download.jboss.org/jbosstools/updates/target-platform/latest/">Latest Target Platform Repo</a>
<li><a href="http://download.jboss.org/jbosstools/updates/target-platform/e361-wtp322.target_v3.zip">Archived Target Platform Repo</a>
<li><a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/target-platform/e361-wtp322.target">Target Platform Definition File</a>
<li><a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/">JBoss Tools SVN Sources</a>
<li><a href="https://jira.jboss.org/browse/JBIDE">JBoss Tools JIRA</a>
</ul>
<hr>
<p>Some handy HOWTOs:<ul>
<li><a href="http://community.jboss.org/wiki/HowtoBuildJBossToolswithMaven3">HOWTO build JBoss Tools (all or individual components) using Tycho</a>
<li><a href="https://www.jboss.org/tools/docs/testing">HOWTO run JBoss Tools' tests (all or individual components) using Tycho</a>
<li><a href="https://www.jboss.org/tools/docs/testing#remote">HOWTO attach Eclipse's debugger to JUnit or SWTBot tests running w/ Tycho</a>
<li><a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/target-platform/README.txt">HOWTO convert from a .target into a repo</a>
<li><a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/target-platform/README.txt">HOWTO update a .target to include newer versions based on contents of a local repo</a>
<li><a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/installation/README.txt">HOWTO install from p2 repo(s) into Eclipse using p2.director script</a>
</ul>
<hr/>
<p><i><small>* - Due to <a href="http://divby0.blogspot.com/2007/12/beer-to-peer-networking-part-1.html">beer2peer limitations</a>, YMMV.</small></i>
<p><i><small>** - I'm aware that the update site throws a 403 if you open it in a browser. I can't be arsed to generate an index.html just yet, nor are there categorized features. Because really, you don't need either - this site is only meant to be used by p2.</small></i>nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com0tag:blogger.com,1999:blog-17823979.post-12088334836611391522010-08-11T11:39:00.004-04:002010-08-11T11:51:58.908-04:00p2 Repository Association And The Fine Art Of Forceably Enabling Disabled Sites 2: Tycho Edition<a href="http://divby0.blogspot.com/2010/02/p2-repository-association-and-fine-art.html">Back in February</a>, I blogged about how to add associate sites to an update site generated w/ the Buckminster or B3 aggregator.
<p>
Since we've moved our build infrastructure to use Tycho + Maven, I had to port the script over to work there for our <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/aggregate/">update site aggregation</a>.
<p>Here are the moving pieces:<ul>
<li><a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/aggregate/site/pom.xml">pom.xml</a> that builds the site, listing input sites as <code><repository></code> entries, and a shout-out to <code>build.xml</code>, below, to do extra stuff during the <code>install</code> phase
<li><a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/aggregate/site/build.xml">build.xml</a> ant script called by <code>pom.xml</code>, above, to do additional work after the creation of the <code>site_assembly.zip</code> update site zip
<li><a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/aggregate/site/aggregateSite.jbosstools.properties">aggregateSite.jbosstools.properties</a> properties file listing the aggregate sites to add
</ul>nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com0tag:blogger.com,1999:blog-17823979.post-30672758294449641932010-07-20T14:33:00.005-04:002010-08-11T11:55:53.592-04:00Troubleshooting Eclipse.org Mirrors / Using Profiles & Target Platform Definition Files With Tycho<blockquote><i>If you look in the mirror and you say his name 5 times, he'll appear behind you breathing down your neck.</i> - <a href="http://www.imdb.com/title/tt0103919/quotes">Candyman</a></blockquote>
<p>If only troubleshooting mirrors was so simple. Have you ever been running a build or an install which stalls at the provisioning step, with a message like:
<blockquote><i> [INFO] Fetching org.eclipse.birt.integration.wtp.ui_2.6.0.v20100617-1315.jar.pack.gz (4.3MB of 46.13MB at 171.56kB/s) from http://mirrors.xmission.com/eclipse/birt/update-site/2.6/plugins/org.eclipse.birt.integration.wtp.ui_2.6.0.v20100617-1315.jar.pack.gz</i></blockquote>
<p>The solution here is often simply to force p2 to pick <b>a specific mirror</b> rather than letting it choose any mirror it wants.
<p>How, you ask?
<p>Well, assuming you were polling this site looking for artifacts to install or update...
<blockquote><i><a href="http://download.eclipse.org/birt/update-site/2.6/">http://download.eclipse.org/birt/update-site/2.6/</a></i></blockquote>
<p>... you would then change that URL to this, and look at the list of available mirrors:
<blockquote><i><a href="http://www.eclipse.org/downloads/download.php?file=/birt/update-site/2.6/">http://www.eclipse.org/downloads/download.php?file=/birt/update-site/2.6/</a></i></blockquote>
<p>Now it's a trivial matter to select a mirror that's close to you and try that instead of the download.eclipse.org mirror, such as:
<blockquote><i><a href="ftp://mirror.csclub.uwaterloo.ca/eclipse/birt/update-site/2.6/">ftp://mirror.csclub.uwaterloo.ca/eclipse/birt/update-site/2.6/</a></i></blockquote>
<hr/>
<p>If you're running a Tycho build, this URL should be changed in your <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/parent-pom.xml">parent-pom.xml</a> ...
<blockquote><i>
<url>http://download.eclipse.org/birt/update-site/2.6/</url>
</i></blockquote>
<p> ... or your <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/target-platform/e36-wtp32.target">.target platform file</a>, depending on which way you're building.
<blockquote><i>
<repository location="http://download.eclipse.org/birt/update-site/2.6/"/>
</i></blockquote>
<p>If you rely on a <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/parent-pom.xml">parent-pom.xml</a>, make sure you're activating the profile with the revised URL...
<blockquote><i>mvn3 clean install -U -B -fae -Phelios</i></blockquote>
<p>... or, if you're building against a .target platform file, make sure you update the URL in that file, and that your build points to the profile which will load the <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/target-platform/e36-wtp32.target">.target file</a>.
<blockquote><i>mvn3 clean install -U -B -fae -P!helios,helios-no-target</i></blockquote>
<p><b>UPDATE, 2010/08/11</b>: Forgot to mention that there are a number of p2 update site zips available here to help with your offsite mirroring: <a href="http://download.eclipse.org/athena/repos/">http://download.eclipse.org/athena/repos/</a>.nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com4tag:blogger.com,1999:blog-17823979.post-64432526612319640882010-06-30T11:10:00.004-04:002010-06-30T11:26:56.218-04:00Update Site Aggregation: B3 vs. Tycho<p>Last year, I used to aggregate update sites with the Buckminster Aggregator, but since that won't install into Eclipse 3.6 (Helios), I had to <a href="http://wiki.eclipse.org/Eclipse_b3/aggregator/Migration_From_buckminster">migrate to the B3 Aggregator</a>. This new version of the Aggregator is greatly expanded and worked fine until recently, when it has begun to suffer from a rather nasty p2 problem: instead of the Director just installing the aggregator into an Eclipse instance prior to then running the aggregation, I get "<a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=317656">The copies of profile SDKProfile are not in sync</a>," and the whole process dies.
http://www.blogger.com/img/blank.gif
<p>So, in order to find a workaround, I went back to <a href="http://divby0.blogspot.com/search/label/tycho">Tycho</a>, and discovered you can merge update sites with little more than two simple files:
<blockquote><ul><li>a <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/aggregate/site/site.xml">site.xml</a>, which lists the features to aggregate and how to categorize them, and <li>a <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/build/aggregate/site/pom.xml">pom.xml</a> to list the source sites and drive the aggregation.</ul></blockquote>
<p>Unfortunately the Tycho solution doesn't <a href="http://divby0.blogspot.com/2010/02/p2-repository-association-and-fine-art.html">include the ability to add associate sites to the metadata after generation</a>, but that can simply be done as a downstream step (<a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/drools/pom.xml">Tycho can call Ant</a> using the maven-antrun-plugin).nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com1tag:blogger.com,1999:blog-17823979.post-20987774879776722832010-05-25T16:19:00.004-04:002010-05-25T16:52:39.749-04:00My love-hate with SVN, Part 8: Installation Ease Of Use (UPDATED)<p>
Back in July 2009, I blogged about <a href="http://divby0.blogspot.com/2009/07/my-love-hate-with-svn-part-6.html">My love-hate with SVN, Part 6: Installation Ease Of Use</a>. With <a href="http://eclipse.org/helios/">Helios</a> just around the corner, I wanted to produce an updated repo for use with the latest & greatest Eclipse 3.6.
<p>Now the <a href="http://wiki.eclipse.org/Equinox/p2/Ant_Tasks#Partial_Mirroring"><code><p2.mirror/></code></a> <a href="http://wiki.eclipse.org/Equinox/p2/Ant_Tasks/Partial_Mirroring/Example">script</a> will fetch and use Ant-Contrib automatically.
<p>Here's the updated 15M update site zip, which includes the following:
<blockquote><a href="http://download.jboss.org/jbosstools/requirements/helios/Subvsve079.I201005121900_SVNconn222.I201005121900_SVNKit133.6648_JNA323_ECF310.v201005082345-Update.zip">Subversive 0.7.9<br/>SVN Connector 2.2.2<br/>SVNKit 1.3.3<br/>JNA 3.2.3<br/>ECF 3.1.0</a></blockquote>
Any problems, please report them in <A href="http://bugs.eclipse.org/284077">bug 284077</a>.nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com0tag:blogger.com,1999:blog-17823979.post-13894095786943527292010-04-16T03:11:00.008-04:002010-04-16T03:41:15.398-04:00HOWTO: Build a XulRunner 1.9.1.2 Update Site with Tycho 0.8 + Maven 3<p>0. Install Maven 3 by downloading then unpacking the tar.gz:
<pre class="brush:shell">cd /tmp; \
wget http://mirror.csclub.uwaterloo.ca/apache/maven/binaries/apache-maven-3.0-alpha-7-bin.tar.gz; \
tar xvzf apache-maven-3.0-alpha-7-bin.tar.gz; \
chmod 755 apache-maven-3.0-alpha-7/bin/mvn; \
alias mvn3='/tmp/apache-maven-3.0-alpha-7/bin/mvn'</pre>
<p>1. Check out sources:
<pre class="brush:shell">cd /tmp; \
svn co http://anonsvn.jboss.org/repos/jbosstools/branches/modular_build/xulrunner/; \
wget http://anonsvn.jboss.org/repos/jbosstools/branches/modular_build/parent-pom.xml</pre>
<p>2. Run build:
<pre class="brush:shell">cd xulrunner; mvn3 -fae clean install</pre>
<p>3. You will get a p2 repo / update site in the target folder of the site project, from which you can install XulRunner or XPCOM into your Eclipse.
<pre class="brush:shell">cd site/org.mozilla.xulrunner.site/target/site</pre>
<hr/>
Note that the parent-pom.xml used above can in fact be much simpler. You only need the following:
<pre class="brush:xml"><?xml version="1.0" encoding="UTF-8"?>
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>org.jboss.tools</groupId>
<artifactId>org.jboss.tools.parent.pom</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>JBoss Tools Parent</name>
<packaging>pom</packaging>
<properties>
<tychoVersion>0.8.0</tychoVersion>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.sonatype.tycho</groupId>
<artifactId>tycho-maven-plugin</artifactId>
<version>${tychoVersion}</version>
<extensions>true</extensions>
</plugin>
<plugin>
<groupId>org.sonatype.tycho</groupId>
<artifactId>target-platform-configuration</artifactId>
<version>${tychoVersion}</version>
<configuration>
<resolver>p2</resolver>
<environments>
<environment>
<os>macosx</os>
<ws>cocoa</ws>
<arch>x86</arch>
</environment>
<environment>
<os>macosx</os>
<ws>carbon</ws>
<arch>x86</arch>
</environment>
<environment>
<os>win32</os>
<ws>win32</ws>
<arch>x86</arch>
</environment>
<environment>
<os>linux</os>
<ws>gtk</ws>
<arch>x86</arch>
</environment>
<environment>
<os>linux</os>
<ws>gtk</ws>
<arch>x86_64</arch>
</environment>
</environments>
</configuration>
</plugin>
</plugins>
</build>
</project></pre>nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com2tag:blogger.com,1999:blog-17823979.post-2804398723188283002010-04-15T16:51:00.003-04:002010-04-15T16:57:47.923-04:00Better Bootstrapping For Athena Projects<p>Inspired by Tycho and Maven, I've simplified the self-bootstrapping process for <a href="http://wiki.eclipse.org/Category:Athena_Common_Build">Athena</a> builds.
<p>Previously, you needed a start.sh script to fetch org.eclipse.releng.basebuilder and org.eclipse.dash.common.releng before you could run a build; if on build.eclipse.org this is done for you automatically. Or, if building locally (or on Windows), you had to fetch these yourself.
<p>Now, you can simply tell your build.xml to bootstrap itself, and it will fetch those projects for you. You can also tell it which version of basebuilder or common.releng to use, rather than being tied to the defaults.
<p>For details, see <a href="http://wiki.eclipse.org/Common_Build_Infrastructure/Getting_Started/Bootstrapping">Getting Started - Bootstrapping</a>.nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com0tag:blogger.com,1999:blog-17823979.post-87229495294145234902010-03-31T16:55:00.006-04:002010-03-31T17:12:31.266-04:00HOWTO: Build Plugin & Feature Projects, Then Run Their Unit Tests w/ Tycho :: GEF Example<p><i>Note that the instructions below are for Linux (or MacOSX). On Windows, your YMMV, but the process is the same.</i>
<p>1. Check out the <a href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.gef/?root=Tools_Project">entire source tree of your project</a> from CVS, SVN, or Git into ~/build.
<p>2. If needed, move plugins, features & test plugins into:
<pre style="brush:shell">~/build/plugins/
~/build/features/
~/build/tests/</pre>
<p>(Test features should go into features/ folder too.)
<p>3. Install scala from <a href="http://www.scala-lang.org/downloads">http://www.scala-lang.org/downloads</a>
<p>4. Fetch <a href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.gef/genpom.scala?root=Tools_Project&view=co">genpom.scala</a> and <a href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.gef/parent-pom.xml?root=Tools_Project&content-type=text%2Fplain&view=co">parent-pom.xml</a>; save in <code>~/build</code> or equivalent.
<p>5. Tweak <code>parent-pom.xml</code> to suit your needs or use as-is.
<p>6. Run this to generate pom.xml files for plugins, features, and tests:
<pre style="brush:shell">cd ~/build/plugins/; scala ../genpom.scala
cd ~/build/features/; scala ../genpom.scala
cd ~/build/tests/; scala ../genpom.scala</pre>
<p>7. Download Maven 3 from <a href="http://www.apache.org/dyn/closer.cgi?path=/maven/binaries/apache-maven-3.0-alpha-7-bin.tar.gz">http://www.apache.org/dyn/closer.cgi?path=/maven/binaries/apache-maven-3.0-alpha-7-bin.tar.gz</a>
<p>8. Install Maven 3
<pre style="brush:shell">sudo su; cd /opt; tar xvzf apache-maven-3.0-alpha-7-bin.tar.gz
ln -s apache-maven-3.0-alpha-7 maven3</pre>
<p>9. For convenience, alias mvn3 to the new maven:
<pre style="brush:shell">alias mvn3='/opt/maven3/bin/mvn 2>&1 clean install | tee buildlog.latest.txt'</pre>
<p>10. Build the plugins, features, and finally tests:
<pre style="brush:shell">cd ~/build/plugins/; mvn3
cd ~/build/features/; mvn3
cd ~/build/tests/; mvn3</pre>
Look in <code>~/build/plugins/org.eclipse.*/target/</code> for generated jars.<br/>
Look in <code>~/.m2/repository/org/eclipse/*</code> for published jars.
<hr/>
To automate steps 6 and 10, you can run this script in the ~/build/ folder:
<pre style="brush:shell">#!/bin/bash
for f in plugins/ features/ tests/; do \
cd $f; scala ../genpom.scala; \
/opt/maven3/bin/mvn 2>&1 clean install | tee buildlog.latest.txt; \
cd ..; \
done</pre>
<p><small><i>(<a href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.gef/README.build.with.Tycho.txt?revision=1.1&root=Tools_Project">The above blog is also posted here</a>.)</i></small>nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com2tag:blogger.com,1999:blog-17823979.post-86245587137790679862010-03-28T13:07:00.008-04:002010-03-28T15:11:56.271-04:00Dash Athena: Post-EclipseCon Wrap-Up<p>Updates to Athena since Feb 2010</p>
<h2>Ease of use</h2>
<ul><li>
<a href="http://wiki.eclipse.org/Common_Build_Infrastructure/Migration_to_Ant">New documentation available for migrating from Bash to Ant</a>
</li></ul>
<ul><li> <a href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.myproject.releng/build.xml?root=Technology_Project&revision=1.2&content-type=text%2Fplain" class="external text" title="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.myproject.releng/build.xml?root=Technology_Project&revision=1.2&content-type=text%2Fplain" rel="nofollow">build.xml</a> is now simpler. <a href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.myproject.releng/?root=Technology_Project" class="external text" title="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.myproject.releng/?root=Technology_Project" rel="nofollow">Sample project template</a> updated. You can now use the same build.xml script to run a build in Eclipse or in Hudson from an Ant-based job. Note that the page <a href="/Common_Build_Infrastructure/Getting_Started/Build_In_Hudson/Ant_Script" title="Common Build Infrastructure/Getting Started/Build In Hudson/Ant Script">Build In Hudson/Ant Script</a> is no longer current. See also <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=304800" class="external text" title="https://bugs.eclipse.org/bugs/show_bug.cgi?id=304800" rel="nofollow">bug 304800</a>.
</li></ul>
<ul><li>You can now run <code>buildExtra.xml#extraPackaging</code> via <code>build.steps=buildExtra</code> rather than requiring <code>build.steps=buildZips</code>.
</li></ul>
<ul><li> Publishing support will improve thanks to <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=306854" class="external text" title="https://bugs.eclipse.org/bugs/show_bug.cgi?id=306854" rel="nofollow">bug 306854</a>... anyone want to contribute to writing a Hudson job, please feel free!
</li></ul>
<h2> Packaging Support</h2>
<ul><li> <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=306300" class="external text" title="https://bugs.eclipse.org/bugs/show_bug.cgi?id=306300" rel="nofollow">bug 306300</a> Athena removes .jar files and only contains pack 200 files in update site - new default setting (keep both artifact types) is <code>removeUnpackedJars=false</code>, but can revert to old behaviour (and smaller update site) with <code>removeUnpackedJars=true</code>
</li></ul>
<ul><li> <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=307016" class="external text" title="https://bugs.eclipse.org/bugs/show_bug.cgi?id=307016" rel="nofollow">bug 307016</a> SDK and Runtime zips produced with buildZips target are missing {notice,epl-v10}.html files; if root files exist they will now be copied into the SDK, Runtime, Examples zips; ${allZip} will also include this so it can be copied from there too (custom packaging in buildExtra.xml#extraPackaging task)
</li></ul>
<h2> Testing Support</h2>
<ul><li> <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=296352" class="external text" title="https://bugs.eclipse.org/bugs/show_bug.cgi?id=296352" rel="nofollow">bug 296352</a> Can't connect to vnc server - fixed using Xnvc option in Hudson job and improvements to testLocal task
</li></ul>
<h2> Publishing Support</h2>
<ul><li> <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=302170" class="external text" title="https://bugs.eclipse.org/bugs/show_bug.cgi?id=302170" rel="nofollow">bug 302170</a> Work around Hudson's missing lastS*Build folders - promote.xml will now recurse into Hudson job tree looking for correct build to publish
</li></ul>
<ul><li> Publishing support will improve thanks to <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=306854" class="external text" title="https://bugs.eclipse.org/bugs/show_bug.cgi?id=306854" rel="nofollow">bug 306854</a>... anyone want to contribute to writing a Hudson job, please feel free!
</li></ul>
<h2> Bug Fixes</h2>
<ul><li> <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=304800" class="external text" title="https://bugs.eclipse.org/bugs/show_bug.cgi?id=304800" rel="nofollow">bug 304800</a> Temporary regression caused by adopting new build.xml script with too-aggressive cleanup default
</li></ul><ul><li> <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=307016" class="external text" title="https://bugs.eclipse.org/bugs/show_bug.cgi?id=307016" rel="nofollow">bug 307016</a> SDK and Runtime zips produced with buildZips target are missing {notice,epl-v10}.html files; if root files exist they will now be copied into the SDK, Runtime, Examples zips; ${allZip} will also include this so it can be copied from there too (custom packaging in buildExtra.xml#extraPackaging task)
</li></ul>
<h2> Documentation & Branding</h2>
<ul><li>
<a href="http://wiki.eclipse.org/Common_Build_Infrastructure/Migration_to_Ant">New documentation available for migrating from Bash to Ant</a>
</li></ul>
<ul><li> <a href="http://eclipse.org/athena/doc/eclipsecon2010/" class="external text" title="http://eclipse.org/athena/doc/eclipsecon2010/" rel="nofollow">EclipseCon 2010 Presentation</a>, "Dash Athena Exposed" is available here.
</li></ul><ul><li> <a href="/Common_Build_Infrastructure/Getting_Started" title="Common Build Infrastructure/Getting Started">Common Build Infrastructure/Getting Started</a> and <a href="/Common_Build_Infrastructure/Getting_Started/Build_In_Eclipse" title="Common Build Infrastructure/Getting Started/Build In Eclipse">Common Build Infrastructure/Getting Started/Build In Eclipse</a> updated.
</li></ul><ul><li> <a href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.myproject.releng/?root=Technology_Project" class="external text" title="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.myproject.releng/?root=Technology_Project" rel="nofollow">Sample project template</a> & generic <a href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.myproject.releng/build.xml?root=Technology_Project&revision=1.2&content-type=text%2Fplain" class="external text" title="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.myproject.releng/build.xml?root=Technology_Project&revision=1.2&content-type=text%2Fplain" rel="nofollow">build.xml</a> updated.
</li></ul>
<ul><li> <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=272723" class="external text" title="https://bugs.eclipse.org/bugs/show_bug.cgi?id=272723" rel="nofollow">bug 272723</a> Logo design contest for Athena under way: vote early, vote often!
</li></ul>nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com2tag:blogger.com,1999:blog-17823979.post-62659171454861948492010-03-24T14:43:00.009-04:002010-03-24T16:55:51.296-04:00I'm in love with Tycho 0.8 and Maven 3<p>Monday:
<p>Signed up for GitHub, downloaded apache-maven-3.0-alpha-7-bin.tar.gz, and <a href="https://docs.sonatype.org/display/TYCHO/BuildingTycho">built Tycho</a> from source. Took a bit of path wrangling (namely figuring out that maven runs based on current directory), but it built and most of the tests passed. Not bad for my first day ever using Maven.
<pre style="brush:shell"># First download Maven3 and unpack into /opt/
export MAVEN_OPTS="-Xmx512m"
export TYCHO_TARGET_PLATFORM=/home/nboldt/eclipse/35clean/eclipse
mvn=/opt/maven/bin/mvn
$mvn clean install -e -V -Pbootstrap-1
$mvn clean install -e -V -Pbootstrap-2 -Dtycho.targetPlatform=$TYCHO_TARGET_PLATFORM
$mvn clean test -e -V -Pits -Dtycho.targetPlatform=$TYCHO_TARGET_PLATFORM</pre>
<p>The really confusing part was "what now? what did I just build? And how do I use it?" Answer: You can now run maven3 from <code>.../sonatype-tycho/tycho-its/target/apache-maven-3.0-alpha-7/bin/mvn</code>
<hr/>
<p>Tuesday:
<p>Attended the <a href="http://www.eclipsecon.org/2010/sessions/?page=sessions&id=1571">Sonatype-sponsored Tycho Build Workshop</a>, and amazed <a href="http://lenettoyeur-on-eclipse.blogspot.com/">Pascal</a> by saying <b>good things</b> about Tycho and Maven3. Apparently I'm known to complain about all things p2, despite my numerous blogs and wiki contributions touting its success and usefulness. Also installed <a href="http://m2eclipse.sonatype.org/installing-m2eclipse.html">m2eclipse</a> to see how effective the pom editor is.
<p>During the workshop, I got the following JBoss Tools projects to build: <b>jmx, archives, as, flow, jbpm3/4, drools, bpel, smooks, common</b>. That's nearly half our projects! Thanks to <a href="http://www.sonatype.com/people/author/jason/">Jason</a> and Igor for their invaluable assistance.
<p>The reason it was so easy was that <a href="http://relation.to/Bloggers/Max">Max</a> had already created a <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/genpom.scala">scala script</a> to generate all the pom.xml files we need to build JBoss Tools, and a <a href="http://anonsvn.jboss.org/repos/jbosstools/trunk/parent-pom.xml">parent-pom.xml</a> on which those tiny pom.xml files depend. (For me as a Maven noob, not having to think at all about creating poms made this transition a no-brainer. Speaking of <a href="http://delicious.com/nickboldt/zombies">no-brainers</a>, my slides from <a href="http://eclipse.org/athena/doc/eclipsecon2010/">Monday's talk about Dash Athena and Zombies are here</a> (PDF).
<a href="http://delicious.com/nickboldt/zombies"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 308px;" src="http://2.bp.blogspot.com/_i21-98vOfTA/S6p8KbwkHpI/AAAAAAAAGDw/cAQ45LBB0lQ/s320/miserymachine-cropped.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5452306817577393810" /></a>
<p>As an added bonus, Tycho, being more strict than PDE, exposes missing information in MANIFEST.MF files such as dependencies on plugins. PDE-based builds are apparently more forgiving because they generally include more crap than is actually needed in the target platform against which the build runs; Tycho builds against what you tell it you need. So, the mavenification of JBoss Tools will actually cause us to have better described plugins, and therefore make them easier to build anywhere. I'm very impressed so far.
<p>Unlike Dash Athena which needs to be told all the IUs (features/plugins) against which your project needs to build, Tycho determines this information from your manifests, so the information need not be duplicated.
<p>I still need to look at how to build update sites from the output of a Tycho build, and how to hook in SWTBot/JUnit4 tests. I have a suspicion this may be handled using Buckminster tools such as their <a href="http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.dash.common.releng/tools/scripts/aggregateRepos.xml?root=Technology_Project&view=markup">Aggregator</a>, or my own custom Ant hackery using the <a href="http://wiki.eclipse.org/SWTBot/Ant">SWTBot wiki docs</a>. Time will tell.
<hr/>
<p>Wed
<p>
This morning on a whim I decided to see if I could port Max's scala script (aside: I've never seen scala before yesterday) to generate pom.xml files for EMF or GEF. Success! With some minor tweaks, I've managed to build GEF 3.6 into my local maven repo (~/.m2). Obviously there's more to be done here... but damn, this is easy.nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com1tag:blogger.com,1999:blog-17823979.post-12747892714167978702010-03-05T17:07:00.004-05:002010-03-05T17:33:50.681-05:00Beyond CVS 0.8.9 Released!<p>It's true that the shoemaker's children often go shoeless; so to my little Sourceforge project, <a href="http://beyondcvs.sourceforge.net/">Beyond CVS/SVN</a> - maintained tirelessly over the years by Chris Callendar - has had to wait *years* to finally get its own update site.
<p>Sad, I know. But thanks to <a href="http://divby0.blogspot.com/2010/03/athena-p2-repos-w-associate-sites.html">a recent breakthough</a> in being able to insert associate sites into p2 repo metadata, you can now install Beyond CVS/SVN without having to hunt down Subclipse or Subversive's update sites.
<p align="center"><img src="http://beyondcvs.sourceforge.net/bc32t.png"></p>
<p>Here's the new site: <a href="http://beyondcvs.sourceforge.net/update/0.8.x">http://beyondcvs.sourceforge.net/update/0.8.x</a>. If you point Eclipse at it, you can install the features therein. If you point a browser at it, you can see instructions for how to do the install. And if you want an offline install, here's the zip: <a href="https://sourceforge.net/projects/beyondcvs/files/Beyond%20CVS/Version%200.8.9/org.eclipse.externaltools-Update-0.8.9.v201003051612.zip/download">org.eclipse.externaltools-Update-0.8.9.v201003051612.zip</a> (<a href="https://sourceforge.net/projects/beyondcvs/files/Beyond%20CVS/Version%200.8.9/README.BeyondCVS.0.8.9.txt/download">README</a>).nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com0tag:blogger.com,1999:blog-17823979.post-6024027229021831412010-03-04T20:23:00.005-05:002010-03-04T20:36:05.470-05:00Athena: p2 repos w/ associate sitesWant to insert associate update sites into your user's Eclipse when they go to install your features from an update site/zip? With an Athena build this is now a trivial exercise.
For example... say you're <a href="https://sourceforge.net/projects/beyondcvs/files/">building a plugin</a> which supports integration with Subclipse or Subversive.
In a traditional install from an SDK or runtime zip, you'd unpack into eclipse/, and wonder why the new plugins didn't work.
In the new p2 install world, you point Eclipse at the update site or zip, and are told that you cannot install the SVN integration because missing requirements cannot be found. So you hunt down the required sites, add them to Eclipse, and try again; this time you are allowed to install the new features.
To improve the ease of use for the end user, Athena now allows you to define associate.sites in your <a href="http://beyondcvs.cvs.sourceforge.net/viewvc/beyondcvs/org.eclipse.externaltools.releng/build.properties?view=markup">build.properties</a>. If this property is set, URLs will be added into your p2 repo's metadata and Eclipse will automatically add those sites when it scans your zip or site. <b>No more manual copy+paste from documentation needed!</b>
<pre class="brush:shell">associate.sites=\
http://subclipse.tigris.org/update_1.6.x,\
http://download.eclipse.org/technology/subversive/0.7/update-site/,\
http://community.polarion.com/projects/subversive/download/eclipse/2.0/update-site/</pre>
And, of course, you can reuse these sites when resolving dependencies against which to build or test:
<pre class="brush:shell">repositoryURLs=\
http://download.eclipse.org/releases/galileo/,\
http://download.eclipse.org/athena/repos/eclipse-Update-3.5.2-201002111343.zip,\
http://download.eclipse.org/eclipse/updates/3.5/,\
${associate.sites}
IUsToInstall=org.eclipse.sdk.feature.group+org.eclipse.sdk.ide+\
org.eclipse.team.svn+org.eclipse.team.svn.core+org.eclipse.team.svn.ui+\
org.tigris.subversion.subclipse.core+org.tigris.subversion.subclipse.ui</pre>nickbhttp://www.blogger.com/profile/09200865148587349560noreply@blogger.com4