Re-Return to Athena

Thanks to Miles for taking the time to slap Athena into submission on his local system - frequent and regular stress testing is what's going to make this facade over PDE bridge the gap between today and the far-flung future (namely B3).

Because he took the time to itemize his concerns / problems / gotchas, I thought I'd take the time to explain why these happen... and which are bugs vs. features. Speaking of bugs and features, Athena has a handy New & Noteworthy wiki page which I update about once a month. If you've never seen the N&N, it's here.

# skipPack=true is useful if you want to test locally. I found that the update manager does not work with pak'd jars when running locally. Perhaps P2 is relying on something on the web server side..?

If p2 isn't properly unpacking pack'd jars, the problem is either: you're using a JDK with pack200 bugs (like Sun 1.6), or you have jars which should never be packed. How can you exclude jars from packing? See JarProcessor Options.

# The build scripts appear to be simply searching for the occurrence of strings to determine wether a given map entry type is being used. They do not appear to respect the comment lines. I guess I should file a bug report on this one -- it cost a bit of time because I have a dodgy SVN setup on my home laptop. I'm actually still not sure what is going on with that (and does the mysterious Java 134 error msg have anything to do with it) but I'm trying to learn not to fix things for which I have a work-around, i.e. if ti works on a different machine, just use that instead!

The purpose of the collectMapTypes task is to simply know what types of maps are being used so that warnings can be echoed to the user if appropriate. If you use SVN on linux, there's a good chance you'll get this message:

[echo] If you get a SIGSEGV or Java Result: 134 when fetching from SVN, remove 
[echo] subversion-javahl and svnkit to force PDE to use commandline svn client.

Why? Because it's a pain in the butt to debug, and at least having a message in the build log gives you some tea leaves to read. I favour documentation over performance in this case.

# There is a somewhat lengthy description at http://wiki.eclipse.org/Common_Build_Infrastructure/Getting_Started of how to setup your local build to use a local cache CVS site but I'm not sure what scenario that would really be helpful in. I just use COPY and it works fine. I suppose that using the CVS approach might excercie some issues that you wouldn't run into with a plain COPY. And I'm not sure wether the local build copies over the binary files or not. If it does, you could get different build results if say your local environment happens to place some artifacts in the copied directories that aren't cleaned out by the PDE build. Not going to worry about it!

A more useful scenario is to take a dump of a CVS, SVN, or Git tree, and point your build at that folder using localSourceCheckoutDir=/path/to/full/source/tree/root in your build.properties file. Then it'll build against that local source cache instead of having to fetch from the repo. Once you have the dump of the repo, from Eclipse do File > Import > Existing projects, and you can have those same projects compilable/editable in your workspace. You can also use Eclipse (eg., with Subversive, Subclipse, eGit) to update the cache before doing a build.

# A useful hack when building locally is to pack up your difficult dependency(s) into a zip file and refer to them in your dependencyURL. It doesn't matter what you use for the URL, because it will use the cached file instead. For example, GEF3D doesn't yet have a build or update site. (We've been collaborating a bit on this so hopefully the build I've got working will help them get one up soon.) So instead of solving the the SVN stuff right away, I put that off until everything else worked. So I created a zip and added an entry for http://download.eclipse.org/tools/gef/fake/gef3d.zip. You have to be careful to have your zip file packaged up correctly or it won't work. That's easy to get wrong..your file structure needs to conform to the standard structure, i.e. eclipse/plugins/some.plugin. On the other hand, you have to be careful to remove these fake dependencies when you begin testing the real fetched versions! The fetch part then fails silently; your build succeeds but using the previously downloaded files.

I generally add comments (TODO, FIXME) in my build.properties to mark temporary hacks, or use a new file, like build.properties.local.hack so I'm not breaking the server-friendly version. (Just make sure you tell build.xml which build.properties file to use, if not the default one -- and DON'T CHECK IN the changed file.)

# But in any case getting the SVN map file right has been a major headache. Again, the only reason I want this is to grab the GEF3D sources, but I can't seem to let this one go as it seems that it should be so easy. The Athena build seems to mangle things in the fetch script; or perhaps this is a PDE Build issue. So I tried the sourceforge style entry and after a couple of hours of fiddling (see below) I found a magic incantation that seems to work -- plugin@org.eclipse.draw3d=SVN,trunk,http://dev.eclipse.org/svnroot/technology/org.eclipse.gef3d,,org.eclipse.draw3d@HEAD

@HEAD is optional; you only need to specify a tag if it's not the HEAD of the branch or trunk. Here's a sample SVN map.

# It would be super nice if there were a built-in dependency analyzer so that one could find the root plugin issues easily. As it is, if you have a bad dependency you get the whole tree and you have to walk the tree by hand to find out the root cause. Overall I think this is another demonstration of the ultimate weakness of the standard build systems out there. They are by nature linear (batch) approaches and so usually you can't do post hoc or inline analysis of issues. There is no semantic information in logs! You end up having to parse them back if you want to do any useful analysis. In a parallel lifetime, I would love to work on a simple semantic logging system that would allow structured analysis of this sort.

I believe the Buckminster guys are doing work in this area, so this may make its way into B3.

# Here is something else that I would love to do given time. The map files are pretty well structured - it would be nice to whip up a quick XText editor that would at least check syntax and more ambitiously provide a mechanism for hooking in semantic checks -- wouldn't it be nice if you could check that that SVN url is correct while editing the map file itself?!

Add that to the wishlist for B3!

# I'm still not sure how Athena determines what goes in the sdk vs. runtime vs. example zips so for now I'm just building the examples manually. Anyone come across documentation for this?

This arbitrary allocation of files is done in common.releng/tools/scripts/buildAllHelper.xml, lines 1063-1097, and is only done if the build.steps entry 'buildZips' is enabled. This behaviour is as far as I'm concerned deprecated, but in place simply to ease transition for existing builds using the old Common Modeling Build to the Athena Common Build. IMHO, no one needs these zips - you only need an update site zip. If you don't like what's in the zips, you can omit the 'buildZips' step and package your own zips by hand, eg:

 <zip destfile="${buildDirectory}/${buildLabel}/${SDKZip}" update="true">
   <zipfileset src="${buildDirectory}/${buildLabel}/${allZip}" dirmode="775" filemode="664" excludes="**/*.pack.gz, **/pack.properties, **/features/*.jar, **/${domainNamespace}.*.all*, **/${domainNamespace}.*.all*/**" />
 </zip>

(For those unfamiliar with the story of the Re-Return, you need to watch HIMYM, S1E15.)