Automatic Eclipse mirror selection / Better download pages?

Have you ever wanted to fetch a whole stack of Eclipse project runtimes so you can build against them? For example, say you want all the Galileo M7 builds from Platform, TPTP, BIRT, DTP, WTP, and dependencies (EMF, GEF, XSD). You can find the URLs for each zip you want on the projects' pages, and download them one-by-one from the closest mirror, but that's time- and bandwidth-consuming, esp. if you want these on a remote server, not your local box.

Enter the "&r=1" option on http://www.eclipse.org/downloads/download.php, which will fetch from the closest mirror automatically.

So, now, you can script the M7 stack fetch like this:

for u in $*; do
  if [[ ! ${u##*file=*} ]]; then # add the r=1 suffix
    u=${u}"&r=1"
  fi
  echo "wget $u ..."
  wget --no-clobber "$u"
done

Then run it like this:

./fetch.sh \
"http://www.eclipse.org/downloads/download.php?file=/tools/gef/downloads/drops/3.5.0/S200905011522/GEF-runtime-3.5.0M7.zip" \
"http://www.eclipse.org/downloads/download.php?file=/birt/downloads/drops/M-R1-2.5M7-200905061338/birt-report-framework-2.5M7.zip" \
"http://www.eclipse.org/downloads/download.php?file=/birt/downloads/drops/M-R1-2.5M7-200905061338/birt-wtp-integration-sdk-2.5M7.zip" \
"http://www.eclipse.org/downloads/download.php?file=/datatools/downloads/drops/N_DTP_1.7/dtp-1.7.0M7-200905052200.zip" \
"http://www.eclipse.org/downloads/download.php?file=/eclipse/downloads/drops/S-3.5M7-200904302300/eclipse-SDK-3.5M7-win32.zip" \
"http://www.eclipse.org/downloads/download.php?file=/eclipse/downloads/drops/S-3.5M7-200904302300/eclipse-SDK-3.5M7-linux-gtk.tar.gz" \
"http://www.eclipse.org/downloads/download.php?file=/eclipse/downloads/drops/S-3.5M7-200904302300/eclipse-SDK-3.5M7-linux-gtk-x86_64.tar.gz" \
"http://www.eclipse.org/downloads/download.php?file=/eclipse/downloads/drops/S-3.5M7-200904302300/eclipse-SDK-3.5M7-macosx-carbon.tar.gz" \
"http://www.eclipse.org/downloads/download.php?file=/modeling/emf/emf/downloads/drops/2.5.0/S200905041408/emf-runtime-2.5.0M7.zip" \
"http://www.eclipse.org/downloads/download.php?file=/tptp/4.6.0/TPTP-4.6.0M7-200904260100/tptp.runtime-TPTP-4.6.0M7.zip" \
"http://www.eclipse.org/downloads/download.php?file=/webtools/downloads/drops/R3.1/S-3.1M7-20090505073946/wtp-S-3.1M7-20090505073946.zip" \
"http://www.eclipse.org/downloads/download.php?file=/webtools/downloads/drops/R3.1/S-3.1M7-20090505073946/wtp-jpt-S-3.1M7-20090505073946.zip" \
"http://www.eclipse.org/downloads/download.php?file=/modeling/emf/emf/downloads/drops/2.5.0/S200905041408/xsd-runtime-2.5.0M7.zip" 

So, now the only problem is that every project structures & styles their downloads pages differently...

1. DTP -> choose file(s)
2. GEF -> choose file(s)
3. EMF/XSD -> choose file(s)
4. WTP -> choose build -> choose file(s)
5. Platform -> choose build -> choose file(s) & platform(s) -> click through warnings
6. TPTP -> choose branch tab -> choose build -> choose file(s)
7. BIRT -> More Downloads -> Recent Builds -> choose build -> choose file(s)

Am I the only person that finds this inconsistency annoying? Is it time for a more consistent UI? I'm exploring what to do for Athena-based builds, and welcome suggestions in bug 275682. What pages do you like best? Worst? Which are easiest to use? Hardest? Do you prefer the old blue-and-white pages? The purple Phoenix pages? The grey Nova pages? Any UI designers want to contribute?

Or, really, are downloads obsolete? If we could collect statistics on p2 jar downloads, I'm sure we'd see that most people prefer that approach, and I for one would certainly prefer to just build against a p2 repo (or 7) than a pile of zips. I suppose the hybrid solution for now is to provide zipped p2 repos for download, many projects do today (Modeling, GEF, PDT, VE ...).

Clarifying Our All-In-Ones

In order to make it clearer what's in the various flavours of all-in-one zips provided in Modeling, GEF, and PDT, I've concocted some new icons to better differentiate them.

This is actually moving in the opposite direction from what I'd proposed in bug 240223 regarding Simplified EMF Downloads as we're now providing more, not less... but IMHO that's okay as long as the community is well served. If you have an opinion on update site vs. update zips vs. SDK zips vs. EPP bundles, please feel free to weigh in on bug 240223.

And if you hate the new icons, please let me know. Better still, design me a replacement. Thanks to iconeden.com for the source pixels.

50,000 Zips

Since turning on Google Analytics for tracking Modeling project zips on September 15, we've gotten over 50,000 pageviews (33,000 unique) of the download tracker. While this doesn't necessarily mean every one of those hits resulted in a download, it certainly shows a lot of interest in downloading EMF, UML2, UML2 Tools, GMF, Query, OCL, and XSD, to name but a few of the more than two dozen Modeling Project components.

Unfortunately, this only scratches the surface -- most of our downloads come from EPP bundles and update sites. But thanks to bug 239668, we will one day be able to track downloads done via p2. If download stats interest you, please vote for bug 239668.