MANIFEST.MF and feature.xml versioning rules

I'm forever forgetting what the rules are for dependency declarations in MANIFEST.MF and feature.xml for osgi plugins and features. And Googling often results in frustration rather than an answer. So, because today I actually found a concise list of the rules, I thought I'd repost them here, with some minor edits to help clarify.

OSGi Plugin Version Ranges

Dependencies on bundles and packages have an associated version range which is specified using an interval notation: a square bracket “[” or “]” denotes an inclusive end of the range and a round bracket “(” or “)” denotes an exclusive end of the range. Where one end of the range is to be included and the other excluded, it is permitted to pair a round bracket with a square bracket. The examples below make this clear.

If a single version number is used where a version range is required this does not indicate a single version, but the range starting from that version and including all higher versions.

There are four common cases:

• A “strict” version range, such as [1.2.3,1.2.3], which denotes that version and only that version.

• A “half-open” range, such as [1.2.3,2.0.0), which has an inclusive lower limit and an exclusive upper limit, denoting version 1.2.3 and any version after this, up to, but not including, version 2.0.0.

• An “unbounded” version range, such as 1.2.3, which denotes version 1.2.3 and all later versions.

• No version range, which denotes any version will be acceptable. NOT RECOMMENDED.

The complete text of the above snippet can be seen here (or here as PDF).

Example:

Require-Bundle: org.eclipse.core.runtime;bundle-version="[3.4.0,4.0.0)",
 org.eclipse.core.resources;bundle-version="[3.4.0,4.0.0)",
 org.eclipse.ui.ide;bundle-version="[3.4.0,4.0.0)",
 org.eclipse.ui.navigator;bundle-version="3.5.100",
 com.ibm.icu

See also:

• plugin manifest (plugin.xml)
• osgi bundle manifest (MANIFEST.MF)

---

In terms of feature manifest (feature.xml) rules, help.eclipse.org has pretty good documentation, but the most important thing to remember - and what I often have to look up - is how to state the matching rules for required upstream features & plugins. Experience says it's always better to state things explicitly so there's no downstream guesswork needed and anyone reading your manifest knows EXACTLY what version(s) are required for or compatible with your feature. Plus, while YOU might be using PDE UI to build, someone else might be using Tycho and Maven, and every tool can interpret missing metadata their own way.

When in doubt, spell it out.

Valid values and processing are as follows:

• if version attribute is not specified, the match attribute (if specified) is ignored.
• perfect - dependent plug-in version must match exactly the specified version. If "patch" is "true", "perfect" is assumed and other values cannot be set. [1.2.3,1.2.3]
• equivalent - dependent plug-in version must be at least at the version specified, or at a higher service level (major and minor version levels must equal the specified version). [1.2.3,1.3)
• compatible - dependent plug-in version must be at least at the version specified, or at a higher service level or minor level (major version level must equal the specified version). [1.2.3,2.0)
• greaterOrEqual - dependent plug-in version must be at least at the version specified, or at a higher service, minor or major level. 1.2.3

The complete text of the above snippet can be seen here.

Example:

<requires>
  <import feature="org.eclipse.m2e.feature" version="1.0.0" match="compatible"/>
  <import feature="org.maven.ide.eclipse.wtp.feature" version="0.13.0" match="greaterOrEqual"/>

  <plugin id="ch.qos.logback.classic" version="0.9.27.v20110224-1110" match="greaterOrEqual"/>
  <plugin id="ch.qos.logback.core" version="0.9.27.v20110224-1110" match="greaterOrEqual"/>
  <plugin id="ch.qos.logback.slf4j" version="0.9.27.v20110224-1110" match="greaterOrEqual"/>
  <plugin id="org.slf4j.api" version="1.6.1.v20100831-0715" match="compatible"/>
  <plugin id="com.ning.async-http-client" version="1.6.3.201106061504" match="equivalent"/>
  <plugin id="org.jboss.netty" version="3.2.4.Final-201106061504" match="perfect"/>
  <plugin id="org.hamcrest.core" version="1.1.0.v20090501071000" match="equivalent"/>
</requires>

HOWTO: Find osgi dependencies in features

Say you're trying to build something with Tycho & Maven 3 and while resolving dependencies before compilation, you're told:

[INFO] [Software being installed: 
  org.eclipse.tm.terminal.local.feature.group 0.1.0.v201006041240-10-7w312117152433, 
  Missing requirement: 
    org.eclipse.tm.terminal.local 0.1.0.v201006041322 
      requires 
        'bundle org.eclipse.cdt.core 5.2.0' 
          but it could not be found, 
  Cannot satisfy dependency: 
    org.eclipse.tm.terminal.local.feature.group 0.1.0.v201006041240-10-7w312117152433 
      depends on: 
        org.eclipse.tm.terminal.local [0.1.0.v201006041322]]

To quickly verify where this dependency is coming from, you can go look into the feature.xml for the org.eclipse.tm.terminal.local feature jar... but if you don't have it installed, this is somewhat more cumbersome; besides, you then have to unpack the jar before you can look inside it.

And maybe that feature contains a number of OTHER dependencies that you'll also need to resolve in your target platform when building. Sure, there are UI tools to do this within Eclipse, but when you're working on remote servers sometimes UI isn't available.

Workaround? Assuming you have a mirror of the update site(s) from which you're trying to resolve the dependency (eg., Helios) or can ssh to dev.eclipse.org, you can simply run a quick shell script to do the investigative work for you:

$ cd ~/downloads/releases/helios/201009240900/aggregate/; ~/bin/findDepInFeature "*tm*" cdt

./features/org.eclipse.tm.terminal.local_0.1.0.v201006041240-10-7w312117152433.jar
      <import feature="org.eclipse.cdt.platform" version="7.0.0" match="greaterOrEqual"/>
      <import plugin="org.eclipse.cdt.core" version="5.2.0" match="compatible"/>
      <import plugin="org.eclipse.core.runtime"/>

Where the script looks like this:

#!/bin/bash
# find plugins/feature deps by searching in some folder for feature jars, and searching through their feature.xml files for dependencies

# 1 - featurePattern - pattern of features to search (eg., "org.eclipse.tptp" or "\*" for all features)
# 2 - dependencyPattern  - pattern of plugins/feature deps for which to search (eg., "org.eclipse.tptp.platform.instrumentation.ui")
# 3 - location       - directory in which to search, if not "."

if [[ ! $1 ]]; then
        echo "Usage: $0 <featurePattern> <dependencyPattern> <location>"
        echo ""
        echo "Example: $0 tm.terminal cdt"
        exit 1
fi

# if no location, look in current dir (.)
if [[ $3 ]]; then location="$3"; else location="."; fi

# if no featurePattern, search all features for dependencyPattern
if [[ ! $2 ]]; then featurePattern="*"; dependencyPattern="$1"; else dependencyPattern="$2"; featurePattern="$1"; fi

rm -fr /tmp/findinfeature/; mkdir -p /tmp/findinfeature/features/
for f in $(find "$location" -type f -name "*${featurePattern}*" | egrep -v "pack.gz|source" | grep features | egrep "${featurePattern}"); do
        #echo "$f [$featurePattern, $dependencyPattern]"
        unzip -q $f -d /tmp/findinfeature/ feature.xml
        #       <import feature="org.eclipse.cdt.platform" version="7.0.0" match="greaterOrEqual"/>
        #       <import plugin="org.eclipse.cdt.core" version="5.2.0" match="compatible"/>
        if [[ ! $(cat /tmp/findinfeature/feature.xml | egrep "<import" -A3 | egrep "plugin=|feature=" -A1 -B1 | egrep "\".*${dependencyPattern}[^\"]*\"" -A1 -B1) ]]; then
                rm -fr /tmp/findinfeature/feature.xml
        else
                mv /tmp/findinfeature/feature.xml /tmp/findinfeature/${f}_feature.xml
                echo "${f}"
                cat /tmp/findinfeature/${f}_feature.xml | egrep "<import" -A3 | egrep "plugin=|feature=" -A1 -B1 | egrep "\".*${dependencyPattern}[^\"]*\"" -A1 -B1
                echo ""
        fi
        rm -fr /tmp/findinfeature/feature.xml
done

HOWTO: Find the feature that contains a plugin

Tycho is awesome.

However, like all build systems, it has its limitations.

One such limitation is that when you're building against a target platform, and something's missing, you get errors such as these:

[INFO] Cannot complete the request.  Generating details.
{org.osgi.framework.executionenvironment=OSGi/Minimum-1.0,OSGi/Minimum-1.1, osgi.ws=cocoa, osgi.arch=x86, osgi.os=macosx, org.eclipse.update.install.features=true, org.osgi.framework.system.packages=}
[Software being installed: org.jboss.tools.tptp.feature.feature.group 1.2.0.qualifier, Missing requirement: org.eclipse.tptp.platform.instrumentation.ui 4.4.1.v201009092123 requires 'bundle org.eclipse.hyades.probekit [4.2.0,5.0.0)' but it could not be found, Cannot satisfy dependency: org.eclipse.tptp.platform.instrumentation.ui.feature.group 4.3.1.v201009092123-797908s73533D4H6D56 depends on: org.eclipse.tptp.platform.instrumentation.ui [4.4.1.v201009092123], Cannot satisfy dependency: org.jboss.tools.tptp.feature.feature.group 1.2.0.qualifier depends on: org.eclipse.tptp.platform.instrumentation.ui.feature.group 4.3.0]
[ERROR] Internal error: java.lang.RuntimeException: org.eclipse.equinox.p2.core.ProvisionException: No solution found because the problem is unsatisfiable. -> [Help 1]
org.apache.maven.InternalErrorException: Internal error: java.lang.RuntimeException: org.eclipse.equinox.p2.core.ProvisionException: No solution found because the problem is unsatisfiable.

The important part of that error message is as follows:

org.jboss.tools.tptp.feature.feature.group 1.2.0.qualifier
   requirement: org.eclipse.tptp.platform.instrumentation.ui 4.4.1.v201009092123 
      requires 'bundle org.eclipse.hyades.probekit [4.2.0,5.0.0)' 
         but it could not be found

org.eclipse.tptp.platform.instrumentation.ui.feature.group 4.3.1.v201009092123-797908s73533D4H6D56 
   depends on: org.eclipse.tptp.platform.instrumentation.ui [4.4.1.v201009092123]
     dependency: org.jboss.tools.tptp.feature.feature.group 1.2.0.qualifier 
        depends on: org.eclipse.tptp.platform.instrumentation.ui.feature.group 4.3.0]

So, how do you find which feature contains that plugin, so that you can add it to your target platform?

First, you need access to the repository. If you have direct server access to the repository from which the plugin comes (eg., the TPTP Helios update site), you can run this script in the root of that repository.

If you don't have server access (eg., you can't ssh to dev.eclipse.org and look in ~/downloads/tptp/updates/helios), then you can pull down a zip of the site (or use a p2.mirror script to fetch a copy of the site to your local machine)... and then run this script in the root of that repository.

Essentially the script finds matching feature jar files, unpacks them to extract the feature.xml files therein, and then greps those files for lines which suggest an included plugin matching the pattern for which you're searching:

$ findInFeature platform.probekit
./features/org.eclipse.tptp.platform.probekit_4.5.1.v201009092123-7H7BF8PAkF7B77ZARCNEK.jar
   <plugin
         id="org.eclipse.tptp.platform.probekit"

./features/org.eclipse.tptp.platform.trace_4.5.1.v201009092123-7L7O8bBgJ9E99jAfGWEM.jar
   <plugin
         id="org.eclipse.tptp.platform.probekit.launch"

From there, it's a trivial exercise to add another line item into your target platform file. First, paste in the feature jar:

./features/org.eclipse.tptp.platform.probekit_4.5.1.v201009092123-7H7BF8PAkF7B77ZARCNEK.jar

Then use vim to pattern-replace that string:

:%s/.\+\/\(org.\+\)_\(\d\+.\+\)\.jar/\t\t\t/g

And you'll end up with a new .feature.group added to the target:

<unit version="4.5.1.v201009092123-7H7BF8PAkF7B77ZARCNEK" id="org.eclipse.tptp.platform.probekit.feature.group"/>

Pattern Match Searches Within Multiple Files And Archives

Have you ever wondered what plugin is locking a particular file extension, and forcing it to be opened with a certain editor? Or wanted to scan some plugins to check their copyright information? Whatever the filetype, this script will allow you to search inside zips and jars for text files matching a given regular expression. Sure, there's probably a graphical way to do this, but nothing beats the purity of the almighty commanline.

Here's some sample output, looking for the plugins that lock *.mod files:

$ find.in.zip.sh ~/eclipse/eclipse-plugins-other "*.jar" "plugin.xml" "extensions.*.mod" -verbose

** [1] /tmp/plugins/org.eclipse.wst.dtd.core_1.1.200.v200710021541.jar!plugin.xml **
41-     <extension point="org.eclipse.core.runtime.contentTypes">
42-             <content-type
43:                     file-extensions="dtd,mod,ent"
44-                     priority="normal"
45-                     name="%DTD_Content_Type_Extension_Element.name"

** [2] /tmp/plugins/org.eclipse.wst.xml.ui_1.0.400.v200711071909.jar!plugin.xml **
16-     <extension point="org.eclipse.wst.xml.ui.catalogFileType">
17-             <fileType
18:                     extensions="dtd, ent, mod"
19-                     description="%_UI_PREF_DTD_FILES"
20-                     id="org.eclipse.wst.xml.core.ui.catalogFileType.dtd">

Looking for files that aren't in zips? Use this simpler script:

$ find.sh ~/eclipse/eclipse-plugins-enable34/ "feature.xml" "2007 by" -verbose

** [1] /tmp/features/org.eclipse.eclipsemonkey_1.0.0.200706060947/feature.xml **
11-
12-   <copyright>
13:      Portions copyright 2007 by Aptana Inc.
14:Other portions copyright 2006-2007 by Eclipse Foundation Inc.
15-   </copyright>
16-

Oh, and if you don't see red when you run these, make sure you have this in your ~/.bashrc, /etc/bashrc, or /etc/bash.bashrc file:

export GREP_OPTIONS='--color=auto'

M & 3s

You and I should get away for a while
I just want to be alone with your smile
Download some features and plugins to update the IDE
We'll blast the stereo and we'll build with PDE

Because when I'm with you there's nothing I couldn't do
I just want to be your only fool
I'm grasping out at straws thinking back to what I saw
That night I tried to use that other tool

IRC was getting so bland
There are only so many ways to type '~faq' with one hand
Sometimes it makes me want to laugh
Sometimes I want to take the toaster in the bath

Because when we're with you there's nothing we couldn't do
And now it's time to share that love
We all oughta be testin' out Gany M3
Cuz there's no free help coming down from above

Who wants to live the bleeding edge?
I want to to live the bleeding edge.
Are you going to help testin'
Or are you assuming it'll work out in the end?

Blink 182 - M & Ms

EuroFM is on the air!

Europa Fall Maintenance is out today, so in tribute...

(Ctrl-) Space Down

Hey Eclipse you know you drive me crazy.
Ctrl-3 puts the rhythm in my hand.
Still I'll never understand how you can do so much
Plugins, features and such
Install Europa bundles from a mirror
Tell yourself it's so gonna rock your world again
You fire it up and then you swear a little

Do you feel like a geek
When you push its buttons?
Do you feel better now Ctrl-Space'ing a ton?
Well I'll tell you my friend no way this world's going to end
IDEs crumble down, they're eclipsed by this one

A pebble in the water makes a ripple effect
Every action in this world will bear a consequence
If you code by hand forever you will surely drown
I see what's goin' down.
Don't be a fool and use vi, emacs or notepad,
Startin' from scratch again,
Heed my lecture

Do you feel like a geek
When you push its buttons?
Do you feel better now Ctrl-Space'ing a ton?
Well I'll tell you my friend no way this world's going to end
IDEs crumble down, they're eclipsed by this one

In June '08 when we've all had enough,
Time for the Gany train!
[x2]

Do you feel like a geek
When you push its buttons?
Do you feel better now Ctrl-Space'ing a ton?
Well I'll tell you my friend no way this world's going to end
IDEs crumble down, they're eclipsed by this one

Downloadin' the code, she said, 'The mother lode!'
She said, I've finally found the best!
[x2]

Red Jumpsuit Aparatus - Face Down

10,000 Hits

Europa!
Europa!

One more download, yay! I know what I want
And my want will be an upgrade all right, an upgrade all right
Just another day when all that I want
Will mark me as an Eclipser tonight, an Eclipse geek tonight, yeah

[Chorus]
Jupiter's moons that can block out the Sun
If this disturbs you then walk away
You will remember: use -vmargs and -vm
10,000 Googlin' away!

Plug-ins un-restrained, dead on the mark
Is what we will deliver tonight, deliver tonight
"Features suck," they say, "but Update finally works!"
With even plugins tonight? Yes, even plugins tonight. Yeah!

[Chorus]

We are the ones that will open your mind
Leave the weak and the NetBeans behind
[x4]

[Chorus]

10,000 Googlin' away! 10,000 reqs per half-day!

Disturbed - Ten Thousand Fists

HOWTO: Install Eclipse for Multi-User Linux Systems

Here's my advice for installing Eclipse onto a Linux box shared among multiple users. I see three solutions with varying amounts of disk space use / user control.

1. Full User Control / Maximum Disk Usage

   Each user installs their own Eclipse in ~/eclipse. This occupies maximum disk usage (perhaps as much as 400M per user), but allows each and every user to configure what plugins/features they need in their Eclipse.

2. No User Control / Minimum Disk Usage

   Root performs a single central install of Eclipse in /opt/eclipse (or their distro's preference). Minimal disk space needed, but users have NO control over updates or of what plugins/features they have access to, just their private workspace(s) in their home dir. This means that if you have some CDT users and some PDT users, everyone will be able to do everything -- and all users will have the same pile of plugins loaded on startup, regardless if they ever use those plugins.

3. Hybrid Install: Partial Control / Medium Disk Usage

   Each user installs the Eclipse Platform Runtime Binary (~40M) in their ~/eclipse. All other plugins/features are installed into one or more "extension locations" in (for example) /opt/eclipse/modeling, /opt/eclipse/pde, /opt/eclipse/jdt, /opt/eclipse/pdt, etc.

   Users can then cherry-pick the feature groups (extension locations) they want depending on their needs via Help > Software Updates > Manage Configuration > Add an Extension Location, but can't run updates -- only root can. This means everyone's install *could* be the same, but you could also have Modeling users, PDT users, and PDE/JDT users who only have installed to their Eclipse the plugins they actually need to perform their job.

   

   If they need a feature that's not available in the common shared /opt space, they can install it themselves into a new extension location, such as ~/eclipse-plugins-phpeclipse. Then, if root notices that several users have downloaded and installed the same features, they could copy those features to /opt/ and symlink the users' extension location folder(s) to the common folders in /opt/ to save disk space.

In all three cases, I would also ensure that users have read the FAQ entries on starting Eclipse and tuning memory usage, so that as they add more features they don't crash their Eclipse or hog system resources. As root, it would be easy to create desktop shortcuts (and/or shell scripts) for a number of different canned Eclipse startup methods, using -vm /opt/jdk50/bin/java or -vm /opt/jdk60/bin/java, -data ~/workspace1 or -data ~/workspace2, and with -vmargs -Xmx256M -XX:PermSize=64M -XX:MaxPermSize=128M

This is how I would address this issue. How have other sysadmins solved this?

I did it all for the cookie...

No, seriously, I don't test for schwag, or for the cookie, though some muppets do. I test because I like to work on and with tools, sites, projects, frameworks... that just work, without jumping through hoops or needing to recite black magic incantations. (Which doesn't mean I won't accept schwag if people want to send it to me. So far the best schwag-like mail I've gotten has been a clown nose from Kim, but that's another story.)

Anyway, inspired by Ian's bad experiences today with Europa, I thought I'd try to reproduce them. I couldn't, but I got something IMHO worse. While there's ways to work around the problems I hit, the issue here is Newbie Usability. I've been hacking w/ Eclipse for 4 years. The point of Europa is to be easy to use for the first-timers, who don't know about things like the eclipse.ini file (I've never looked at that file until today) or any/all of the OSGi startup flags you can use.

So, to synthesize the Newbie experience further, I googled for "-Dosgi" and searched the Eclipse wiki, but the best I could find is passing mention of these flags in some 145 Eclipse bugs. Another usability issue, IMHO -- insufficiently quick-findable documentation.

Oh, wait, I think (while still wearing the Newbie hat), what about help.eclipse.org? Sure enough, it's documented as Eclipse runtime options. OK, that's cool, and that also means if I installed the SDK I have those docs in my Europa install, but if the problem is starting Europa in the first place, I still have to go hunting outside the Eclipse Help system. At least I've found what I wanted.

Back to the original problem... installing 121 out of 122 Europa features and watching as my Eclipse install crashes and burns on the first startup. Not cool. Second startup is better, but has problems on shutdown. Third time's the charm, apparently.

This whole experience brings to mind five, no, three questions, which I'll pose here and cross-post to the cross-project-issues-dev list too.

1. Should the Buckminster SVN support be contributed to Europa, or should it ONLY be on Buckmister's own Discovery or Update site, in order to avoid a config problem right out of the box? I'd suggest it should not be in Europa because of the negative perception it might cause both Europa overall and Buckminster in particular. It portrays two issues: (i) the full suite (122 features) doesn't work 100% out of the box, and (if you'll forgive the rather damning phrase), (ii) "it's all Buckminster's fault". I don't mean to offend the good folks on the Buckminster team, only to suggest what *some* might say, and how to avoid that perception from ever being voiced anywhere but this rant. ;-)
2. Since those who want SVN support need to add another Update site anyway (or install SVN features manually some other way), is it so onerous to ask those people to jump thru one extra hoop and remove the hoop from everyone else who doesn't need/want SVN?
3. Who in their right mind will need to install 121 features (451M) on day one of their Europa experience? I, for example, only really need about 75M of features on a regular basis (EMF, XSD, Mylyn, phpeclipse, PDE, JDT, CVS) on top of the base 40M platform (68M unpacked).

I agree with Michael Scharf (who, incidentally, was the only person who could follow instructions) that Eclipse can be compared to ketchup in that it's:

[U]sed everywhere. It often does not fit. ... It comes in massive amounts (like the 18 million lines of code of the Europa release). It's often hard not to use it.

But that said, I don't think installing Europa should be harder than banging on the bottle to shake the last dregs of ketchup down the neck and onto my fries. It's hard *not* to use Eclipse, true, but should it be this hard to *use* it too?

That Ol' PDE Black Magic

... or more reasons why features still suck.

In the past 24hrs, I've encountered two feature-related PDE issues, which I think warrant discussion as they are, to the uninitiated, seemingly black magic. The first of these was discovered and fixed last night while watching The Dresden Files in ten-minute chunks while waiting for builds to complete, but I'm sure that's just a coincidence.

Bug The Firste: feature.xml should not specify version numbers when including required plugins like org.eclipse.test [192231].

This issue manifests when your build stops working one day and complains, "eclipse/test.assembly/eclipse/plugins/${org.eclipse.test} not found." The fix is simple: make sure you set version="0.0.0" in your feature.xml when including org.eclipse.test or other plugins referenced in your map file and required by your build. See also Build Problems.

Bug The Seconde: PDE will build features in the order they're listed in feature.xml [192292].

This issue manifests when your source feature, for example, builds OK but contains no source because PDE is building it BEFORE your runtime feature(s). Once again, the fix is simple. You just have to reorder your feature.xml to put your source feature(s) last.

Bug The Thirde (Bonus Bug!): If your plugin or feature doesn't have the right Nature and Builder in its .project file, it won't behave the way you'd expect [192247] in Eclipse.

This one's a no-brainer, but also a handy tip when working with PDE. If your plugin project doesn't have the Plugin nature, you can't open a MANIFEST.MF file with the Manifest Editor; if your feature doesn't have the Feature nature, PDE won't give you warnings/errors if your feature is misconfigured (eg., if you have a version set to "o.o.o" instead of "0.0.0"). Just add something like this:

<buildSpec>
  <buildCommand>
    <name>org.eclipse.pde.FeatureBuilder</name>
    <arguments/>
  </buildCommand>
</buildSpec>
<natures>
  <nature>org.eclipse.pde.FeatureNature</nature>
</natures>

Request For Comment: Will anyone miss the EMF Standalone Zip?

With the advent of jarred plugins, improvements to Update Manager, and all the new smaller EMF 2.3 features (including 7 core "runtime" ones in Europa's Enabling Features category -- see bugs 106804 and 189295), the EMF team is evaluating if the Standalone Zip introduced in EMF 2.1 is still meaningful and useful to consuming teams, projects, and products.

For more on this topic and to learn what's actually in this zip, see:

http://wiki.eclipse.org/index.php/EMF_2.3_Standalone_Zip

To voice your opinion (be it 'I need that zip!' or 'not useful to me'), please post your feedback in bug 191837.

Thanks!

Managing Plugins and Features with Link Files and Extension Locations

After loitering and latering (but no lootering!) in IRC for a few hours today, I've come to the conclusion that not enough people know about .link files. I guess I could wiki this, but I thought I'd blog it instead.

Scenario 1: You're an Eclipse user. You use Eclipse a lot. In fact, you use multiple versions of Eclipse for testing, debugging, and even *gasp* developing. You waste a ton of time unpacking zips and running Update Manager.

Scenario 2: You use Eclipse and occasionally do something silly like deleting files outside of Eclipse or updating CVS files using commandline CVS instead of the click-click-wait method in Eclipse. (This is usually not a problem, but can sometimes lead to ...)

Scenario 3: Something Wicked This Way [Came] and your Eclipse has become corrupt. It's time to blow away your config files, .metadata files, or run with -clean. If that doesn't work you probably have to reinstall (oh crap) everything -- Eclipse + all your trusty plugins. You could also copy carefully from the busted Eclipse to the minty-fresh one, but that's both time consuming and possibly dangerous. Is there a better way?

Scenario 4: You produce a product based on Eclipse but want to keep your stuff and Eclipse's stuff separate when you bundle it all up for easier digestion by your users. Maybe for license reasons, maybe for legal reasons, maybe just for file system hygiene or to make support easier.

Scenario 5: You're a control freak and just like to keep stuff in different places according to your own particular... um... idiom.

In all of these scenarios, .link files can help. Here's how to set one up:

1. In your Eclipse install folder, create a links/ folder next to features/ and plugins/.
2. In your eclipse/links/ folder, create a textfile called whatever.link. This file need only be one line, with newline and no trailing slash(es), and can be called anything ending with .link.

   path=/home/nickb/eclipse/eclipse-plugins-phpeclipse
   -or-
   path=D:/eclipse/eclipse-plugins-phpeclipse
   -or-
   path=D:\\eclipse\\eclipse-plugins-phpeclipse

3. Unpack a zip full of plugins and features into the path specified in your .link file.

   /home/nickb/eclipse/eclipse-plugins-phpeclipse/
       eclipse/
           plugins/
           features/

4. Start up Eclipse. Check Help > About to see if all your plugins and features are listed.
5. Rinse, lather, repeat.

You can have lots of .link files and lots of install locations, which allows you to easily swap between different versions of plugins and features for testing purposes, or to share an install location between multiple Eclipse installs. And, when you upgrade Eclipse, you can just copy the old links/ folder into the new Eclipse and voom, all your old plugins and features are there. Please note that when changing between versions of Eclipse, sometimes things will end up disabled, if the new Eclipse won't work with the old plugins. If this happens, you can easily switch back to the old one, look for an update, or download a new zip.

You can also manage your plugins inside Eclipse. So, if you prefer clicking to typing, you can convert a .link file location into a Update Manager (UM) install location, so that you can use UM to install updates to plugins that you created by unpacking a zip, or just to reuse an existing folder for later UM installs. Here's how:

1. Complete steps above. Verify all your plugins & features are properly available in Eclipse.
2. To convert a linked folder to an "Extension Location" as used by Update Manager, you just have to put a single file in the eclipse/ folder inside the linked folder. Here's a sample:

   $ cat /home/nickb/eclipse/eclipse-plugins-europa/eclipse/.eclipseextension
   id=org.eclipse.platform
   name=Eclipse Platform
   version=3.3.0

3. Remove or rename the .link file that references the folder you just converted so that you're not trying to add the same plugins twice.
4. Start up Eclipse. Check Help > About to make sure you no longer have the plugins & features you just disabled. Then launch Help > Software Updates > Manage Configuration > Add an Extension Location to re-add those plugins & features.
5. Restart Eclipse.

Happiness in Codeslavery

Oh. My. God. It's over. Bug 106804, "rearrange the features to minimize external dependencies", is finally done. It's been quite the ride, as you can see in the bug. The last three days are a blur, but it's done. Marcelo and I have been fighting the good fight solidly from Tuesday until this morning to contain this change in time for the M7 + 1 deadline. Thanks to the PDE folks for providing invaluable assistance in deciding what paths to take. Pascal, Kim, Sonia, Andrew: you guys rock. Thanks also to everyone else I've bugged with 'have you ever done this with your features / does your project do this?' questions to determine if we were treading into impossible territory or following an existing path.

EMF now contains smaller features which can be installed via Update Manager to compose whatever legal combination you want. Thanks to Update Manager finally working with features' dependencies on plugins, you can use PDE to generate a feature's dependencies and have the Update Site actually work when you hit 'Select Required' to pick up all the associated required features and their contained plugins. (You have no idea how freakin' exciting this, and how long I've been waiting for this to work.) I still think features suck, but for now I'll allow that they suck a little less than last week.

So, here's the obligatory shout-out to the community to try out EMF 2.3.0M7, and let us know any issues you might have w/ migration & adoption. You can grab it from the usual places. Please don't hesitate to speak up in bug 106804: if there are any unforeseen migration pains, I'll use your comments to draft documentation explaining what's different, why, and how to roll with the changes next week after I've slept a little.

Thanks, everyone, and happy EMF'ing.

EMF: It's A Beautiful Thing

At the moment, there are at least 11 different ways to get EMF, topped just the way you like it:

• 9 zips: 3 SDKs (EMF-SDO-XSD, EMF-SDO, XSD SDK), 3 runtimes (EMF-SDO, XSD & Standalone), examples, tests, and models [1]
• 2 different Update Manager methods (install EMF-SDO-XSD SDK or install individual features) [2]

Is this sufficient? Some say it's already overkill, but others say no, and with the lofty goal of pleasing most of the people, most of the time, we're now exploring creating a whole new set of features in EMF [3].

This new set of features [4] is intended to add more options via Update Manager, as that's the most commonly used mechanism (80% of all requests tracked by eclipse.org last year) for downloading EMF. The old features remain -- they're just being chopped up more finely.

Those who want fries with that can order a combo meal and drive forward to the second window to pick up their zips; those who want EMF à la carte will now be able to order off the Update Manager menu, so they can pick and choose the toppings, er, features, they want for their special-needs diet.

Please comment or vote for bug 106804 if you have concerns or kudos.