Annoyed that your boss didn't approve that expense to fly over to Germany for Eclipse Summit Europe? Well, fret no more!
Have we got a deal for you!
For only $40 you can attend Ontario [GNU] Linux Fest 2009 - a whole day of open source geekery from the following speakers! And, while there, stop by the Eclipse and Fedora tables for your chance to win some schwag!
I'd like to say we'll be giving away Slap-Chops or Sham-Wows too, but Vince will be rappin' & reppin' elsewhere. :)
About a week ago, Wayne Beaton approached me with an unexpected question: would I like to be drafted into the rank and file of the Eclipse Architecture Council?
I was honoured. Flabbergasted. And a bit apprehensive: could I measure up?
Well, it seems my first act as an AC member was, Thursday night, to cripple our shared Hudson build server by dumb-assedly performing a seemingly innocuous update. Heartfelt thanks to the Webmasters for once again extracting my cheeks from the fire and getting everyone's builds back up and running.
My second act, Friday afternoon, was to commit some "fixes" to the Athena Common Build code which effectively prevented it from reading your project's build.properties. Thankfully, the ever-vigilant Dave Carver spotted the snafu, filed a bug, and I had it my error fixed shortly after my return from Zombieland last night (aside: best zom-rom-com since Shaun Of The Dead!)
Thanks to Matt, Dave C, David W, Denis, Gunnar, and everyone else for reporting &/or fixing these issues, and to the AC for welcoming me into their fold. It's going to be interesting!
Have you heard the latest? Here's what's been going on in the Dash Athena project lately.
product=Dash&component=Common+Builder to product=Dash+Athena. Here's what's changed this week.
See also Older New & Noteworthy.
Thanks to the folks at my IT helpdesk for this tip on how to enable your BB to get mail from within Zimbra folders:
Any NEW messages routed to those folders will now be located in the folders on the device as well. As new messages, they'll appear in your Messages folder along with SMSs, MMSs, BBIMs, and other email. Once read, they'll disappear into the appropriate folder(s).
Thanks to Miles for taking the time to slap Athena into submission on his local system - frequent and regular stress testing is what's going to make this facade over PDE bridge the gap between today and the far-flung future (namely B3).
Because he took the time to itemize his concerns / problems / gotchas, I thought I'd take the time to explain why these happen... and which are bugs vs. features. Speaking of bugs and features, Athena has a handy New & Noteworthy wiki page which I update about once a month. If you've never seen the N&N, it's here.
# skipPack=true is useful if you want to test locally. I found that the update manager does not work with pak'd jars when running locally. Perhaps P2 is relying on something on the web server side..?If p2 isn't properly unpacking pack'd jars, the problem is either: you're using a JDK with pack200 bugs (like Sun 1.6), or you have jars which should never be packed. How can you exclude jars from packing? See JarProcessor Options.
# The build scripts appear to be simply searching for the occurrence of strings to determine wether a given map entry type is being used. They do not appear to respect the comment lines. I guess I should file a bug report on this one -- it cost a bit of time because I have a dodgy SVN setup on my home laptop. I'm actually still not sure what is going on with that (and does the mysterious Java 134 error msg have anything to do with it) but I'm trying to learn not to fix things for which I have a work-around, i.e. if ti works on a different machine, just use that instead!The purpose of the
collectMapTypes task is to simply know what types of maps are being used so that warnings can be echoed to the user if appropriate. If you use SVN on linux, there's a good chance you'll get this message:
[echo] If you get a SIGSEGV or Java Result: 134 when fetching from SVN, remove [echo] subversion-javahl and svnkit to force PDE to use commandline svn client.Why? Because it's a pain in the butt to debug, and at least having a message in the build log gives you some tea leaves to read. I favour documentation over performance in this case.
# There is a somewhat lengthy description at http://wiki.eclipse.org/Common_Build_Infrastructure/Getting_Started of how to setup your local build to use a local cache CVS site but I'm not sure what scenario that would really be helpful in. I just use COPY and it works fine. I suppose that using the CVS approach might excercie some issues that you wouldn't run into with a plain COPY. And I'm not sure wether the local build copies over the binary files or not. If it does, you could get different build results if say your local environment happens to place some artifacts in the copied directories that aren't cleaned out by the PDE build. Not going to worry about it!A more useful scenario is to take a dump of a CVS, SVN, or Git tree, and point your build at that folder using
localSourceCheckoutDir=/path/to/full/source/tree/root in your build.properties file. Then it'll build against that local source cache instead of having to fetch from the repo. Once you have the dump of the repo, from Eclipse do File > Import > Existing projects, and you can have those same projects compilable/editable in your workspace. You can also use Eclipse (eg., with Subversive, Subclipse, eGit) to update the cache before doing a build.
# A useful hack when building locally is to pack up your difficult dependency(s) into a zip file and refer to them in your dependencyURL. It doesn't matter what you use for the URL, because it will use the cached file instead. For example, GEF3D doesn't yet have a build or update site. (We've been collaborating a bit on this so hopefully the build I've got working will help them get one up soon.) So instead of solving the the SVN stuff right away, I put that off until everything else worked. So I created a zip and added an entry for http://download.eclipse.org/tools/gef/fake/gef3d.zip. You have to be careful to have your zip file packaged up correctly or it won't work. That's easy to get wrong..your file structure needs to conform to the standard structure, i.e. eclipse/plugins/some.plugin. On the other hand, you have to be careful to remove these fake dependencies when you begin testing the real fetched versions! The fetch part then fails silently; your build succeeds but using the previously downloaded files.I generally add comments (TODO, FIXME) in my build.properties to mark temporary hacks, or use a new file, like build.properties.local.hack so I'm not breaking the server-friendly version. (Just make sure you tell build.xml which build.properties file to use, if not the default one -- and DON'T CHECK IN the changed file.)
# But in any case getting the SVN map file right has been a major headache. Again, the only reason I want this is to grab the GEF3D sources, but I can't seem to let this one go as it seems that it should be so easy. The Athena build seems to mangle things in the fetch script; or perhaps this is a PDE Build issue. So I tried the sourceforge style entry and after a couple of hours of fiddling (see below) I found a magic incantation that seems to work -- plugin@org.eclipse.draw3d=SVN,trunk,http://dev.eclipse.org/svnroot/technology/org.eclipse.gef3d,,org.eclipse.draw3d@HEAD@HEAD is optional; you only need to specify a tag if it's not the HEAD of the branch or trunk. Here's a sample SVN map.
# It would be super nice if there were a built-in dependency analyzer so that one could find the root plugin issues easily. As it is, if you have a bad dependency you get the whole tree and you have to walk the tree by hand to find out the root cause. Overall I think this is another demonstration of the ultimate weakness of the standard build systems out there. They are by nature linear (batch) approaches and so usually you can't do post hoc or inline analysis of issues. There is no semantic information in logs! You end up having to parse them back if you want to do any useful analysis. In a parallel lifetime, I would love to work on a simple semantic logging system that would allow structured analysis of this sort.I believe the Buckminster guys are doing work in this area, so this may make its way into B3.
# Here is something else that I would love to do given time. The map files are pretty well structured - it would be nice to whip up a quick XText editor that would at least check syntax and more ambitiously provide a mechanism for hooking in semantic checks -- wouldn't it be nice if you could check that that SVN url is correct while editing the map file itself?!Add that to the wishlist for B3!
# I'm still not sure how Athena determines what goes in the sdk vs. runtime vs. example zips so for now I'm just building the examples manually. Anyone come across documentation for this?This arbitrary allocation of files is done in common.releng/tools/scripts/buildAllHelper.xml, lines 1063-1097, and is only done if the build.steps entry 'buildZips' is enabled. This behaviour is as far as I'm concerned deprecated, but in place simply to ease transition for existing builds using the old Common Modeling Build to the Athena Common Build. IMHO, no one needs these zips - you only need an update site zip. If you don't like what's in the zips, you can omit the 'buildZips' step and package your own zips by hand, eg:
<zip destfile="${buildDirectory}/${buildLabel}/${SDKZip}" update="true">
<zipfileset src="${buildDirectory}/${buildLabel}/${allZip}" dirmode="775" filemode="664" excludes="**/*.pack.gz, **/pack.properties, **/features/*.jar, **/${domainNamespace}.*.all*, **/${domainNamespace}.*.all*/**" />
</zip>
(For those unfamiliar with the story of the Re-Return, you need to watch HIMYM, S1E15.)
As there's still no Hudson Helper for Blackberry (David: hint, hint!), I've been forced to do my own monitoring in a browser (when not monitoring by email, that is).
Now, with Opera 10 (and its new tab layout options), it's even easier to monitor multiple builds via a single view:
Converting a single AVI to DVD format is easy.
However, if you want to merge multiple files into a single DVD image, you must:
for f in $(ls /path/to/season1/{GI_Joe.S1E0*.avi,GI_Joe.S1E1*.avi,GI_Joe.S1E2{0,1,2,3}*.avi}); do \
g=${f/.avi/.mpg}; g=${g/season1/season1_mpg}; \
transcode -i $f -y ffmpeg --export_prof dvd-ntsc --export_asr 3 -o movie \
-D0 -s2 -m movie.ac3 -J modfps=clonetype=3 --export_fps 29.97; \
mplex -f 8 -o $g movie.m2v movie.ac3; \
rm -fr movie.m2v movie.ac3; \
done; \
cd /path/to/season1_mpg
<dvdauthor dest="DVD_Season1_Ep01-05">
<vmgm />
<titleset>
<titles>
<pgc>
<vob file="GI_Joe.S1E01.The_Further_Adventures_of_G.I.Joe.mpg" chapters="0,8:00,16:00,21:00"/>
<vob file="GI_Joe.S1E02.Rendezvous_in_the_City_of_the_Dead.mpg" chapters="0,8:00,16:00,21:00"/>
<vob file="GI_Joe.S1E03.Three_Cubes_to_Darkness.mpg" chapters="0,8:00,16:00,21:00"/>
<vob file="GI_Joe.S1E04.Chaos_in_the_Sea_of_Lost_Souls.mpg" chapters="0,8:00,16:00,21:00"/>
<vob file="GI_Joe.S1E05.Knotting_Cobras_Coils.mpg" chapters="0,8:00,16:00,21:00"/>
</pgc>
</titles>
</titleset>
</dvdauthor>
...merge the MPEG files into a single disc image:
dvdauthor -x dvdauthor_s1e01-05.xml
xine dvd:/full/path/to/DVD_Season1_Ep01-05
growisofs -Z /dev/dvd1 -dvd-video DVD_Season1_Ep01-05/
Yo, Joe!
Re Wayne's blog about an Eclipse.org Maven Repo:
EMF has had an undocumented/unmarketed Maven2 repo for about 4 years now.
All you need is to take an update site zip, unpack it, rearrange the folder structure, and rename the jars. Then you create little XML files called .poms to describe the jars in the tree, and Maven-aware tools can read the tree. It's fairly trivial. http://download.eclipse.org/modeling/emf/emf/maven2/ is the URL, IIRC. About once every 2 years someone asks about our providing such a repo, and I give out the URL. Clearly not a huge demand for it.
One might argue that creation of such a folder structure is in the purview of Athena's publishing scripts, which today eases the process of copying your bits to download.eclipse.org, then unpacks your Update site so it can be scanned by p2 rather than downloaded as a single archive. It too is fairly trivial. I would not be adverse to converting my existing shell script for the EMF repo creation into a generic Ant script for use by Athena users.
Frankly though, I think it would be more valuable if the m2eclipse folks added support for reading/converting p2 repos. Publishing yet another file format would require another release-train-like workflow (we already have two: EPP and buckybuilder for Galileo) and more people maintain it. Even if every project published their own maven repo, we'd want for the sake of ease of use to aggregate them into a central place for easier navigation and discovery by maven tools. So, like with Ganymede, we'd have each project's bits copied to two places on disk for each build. (Galileo used composite repos to POINT at project repos rather than copying them saving tons of disk space and CPU cycles. AFAIK, Maven does not support this concept, but I could be wrong.)
There's also another benefit to having tooling to support converting from p2 repo to maven2 repo: the aggregate repo could be housed at apache.org and suck THEIR bandwidth and support resources instead. Thus just as Eclipse.org is upstream from Fedora's Eclipse project .rpms (which are upstream from Debian/Ubuntu's .debs), Eclipse.org p2 repos could be upstream from Apache's Maven repo(s). After all, Apache already collects maven artifacts for non-apache.org projects to facilitate the use and adoption of maven, so this is entirely in line with their standard operating procedures.
Thoughts?
Posted from Blackberry using Opera Mini, by the side of Shawnigan Lake, Vancouver Island, BC
SVN metadata appears when searching
Why would anyone ever want to see .svn folders and their children in Eclipse? If they're hidden from the Package Explorer, why can't I hide them from a Search? And if I did a naive find and replace, wouldn't this corrupt the metadata and prevent me from committing the update to the repo?
My experiment last week with sshfs came to a grinding halt when I realized that while the share works, and can be automatically started, the permissions do not work and Hudson can't actually use the sshfs share; also, I couldn't get sshfs to compile for OpenSolaris, so it's really a non-starter.
Next up was mount_nfs, but after fighting with that, /etc/nfs.conf and /etc/exports for a couple days off and on, I realized there's a much simpler solution using Virtual Box itself: vboxsf. Note that it's sf (shared folder) not fs (file system).
Thanks to David Herron for the inspiration.
Here's how to set up a vboxfs share:
System Preferences > System > Accounts, and create a "hudsonbuild" user with User ID and Group ID set to 500.
Note that uid 500 and gid 500 are the defaults for the first user on a Fedora system; on Ubuntu it's 1000; on other systems YMMV.
/shared. Put some files there. Give it full write permissions, and set its ownership to the user who'll be sharing the files:
find /shared -type f -exec chmod 666 \; find /shared -type d -exec chmod 777 \; chown -R 500:500 /shared
Settings > Shared Folders > New (+). Add a shared folder called "shared" mapped to /shared (or call it something else and map it to a different path on your host).
Devices menu. Add a shared folder called "shared" mapped to /shared.
shared /shared vboxsf auto,exec,rw,uid=500,gid=500 0 0
# mount everything in /etc/fstab mount -a # mount using mount (and optional -o options) mount -t vboxfs -o auto,exec,rw,uid=500,gid=500 shared /shared # mount using mount.vboxsf (and optional -w and -o options) /sbin/mount.vboxsf -w -o rw,uid=500,gid=500 shared /shared
!/bin/bash
#
# Init file for Hudson server daemon
#
# chkconfig: 2345 65 15
# description: Hudson server
# HOWTO:
# 1. rename this file on target server (Hudson slave node) to /etc/init.d/hudson
# 2. to enable this script to run on startup, as root type the following:
# Fedora:
# chkconfig --add hudson; chkconfig --list hudson
# OpenSolaris:
# for n in 0 1 6; do cd /etc/rc${n}.d; ln -s /etc/init.d/hudson K15hudson; done
# for n in 2 3 4 5; do cd /etc/rc${n}.d; ln -s /etc/init.d/hudson S65hudson; done
# svccfg add hudson; svcs | grep hudson
RETVAL=0
#run Hudson as a slave node
java -jar /opt/hudson/slave.jar -jnlpUrl \
http://vboxsvr:8080/computer/${slaveNodeName}/slave-agent.jnlp &
/sbin/mount.vboxsf -w shared /shared &
exit $RETVAL
Contact:: Nick Boldt:: Scarborough, ON Canada Projects |
  |
JBoss Tools / Dev Studio Releng
Athena: Eclipse Common Builder
Modeling Project Releng
Basebuilder Tools
| Eclipse Community Award Winner Top Ambassador 2009 | |
 | |
|
|
