Much ado about scripting, Linux & Eclipse: card subject to change

2009-08-27

HOWTO: AVI to DVD Conversion, Part 2: Merging Multiple AVIs

Converting a single AVI to DVD format is easy.

However, if you want to merge multiple files into a single DVD image, you must:

  • Convert the AVI files to MPEG:
    for f in $(ls /path/to/season1/{GI_Joe.S1E0*.avi,GI_Joe.S1E1*.avi,GI_Joe.S1E2{0,1,2,3}*.avi}); do \
      g=${f/.avi/.mpg}; g=${g/season1/season1_mpg}; \
      transcode -i $f -y ffmpeg --export_prof dvd-ntsc --export_asr 3 -o movie \
        -D0 -s2 -m movie.ac3 -J modfps=clonetype=3 --export_fps 29.97; \
      mplex -f 8 -o $g  movie.m2v movie.ac3; \
      rm -fr movie.m2v movie.ac3; \
    done; \
    cd /path/to/season1_mpg
  • Next, using a more complex dvdauthor.xml file...
    <dvdauthor dest="DVD_Season1_Ep01-05">
      <vmgm />
      <titleset>
         <titles>
           <pgc>
             <vob file="GI_Joe.S1E01.The_Further_Adventures_of_G.I.Joe.mpg" chapters="0,8:00,16:00,21:00"/>
             <vob file="GI_Joe.S1E02.Rendezvous_in_the_City_of_the_Dead.mpg" chapters="0,8:00,16:00,21:00"/>
             <vob file="GI_Joe.S1E03.Three_Cubes_to_Darkness.mpg" chapters="0,8:00,16:00,21:00"/>
             <vob file="GI_Joe.S1E04.Chaos_in_the_Sea_of_Lost_Souls.mpg" chapters="0,8:00,16:00,21:00"/>
             <vob file="GI_Joe.S1E05.Knotting_Cobras_Coils.mpg" chapters="0,8:00,16:00,21:00"/>
           </pgc>
         </titles>
      </titleset>
    </dvdauthor>
    ...merge the MPEG files into a single disc image:
    dvdauthor -x dvdauthor_s1e01-05.xml
  • Verify the video and audio will play.
    xine dvd:/full/path/to/DVD_Season1_Ep01-05
  • Burn the DVD.
    growisofs -Z /dev/dvd1 -dvd-video DVD_Season1_Ep01-05/

Yo, Joe!

2009-08-13

We Don't Need Another Repo

Re Wayne's blog about an Eclipse.org Maven Repo:

EMF has had an undocumented/unmarketed Maven2 repo for about 4 years now.

All you need is to take an update site zip, unpack it, rearrange the folder structure, and rename the jars. Then you create little XML files called .poms to describe the jars in the tree, and Maven-aware tools can read the tree. It's fairly trivial. http://download.eclipse.org/modeling/emf/emf/maven2/ is the URL, IIRC. About once every 2 years someone asks about our providing such a repo, and I give out the URL. Clearly not a huge demand for it.

One might argue that creation of such a folder structure is in the purview of Athena's publishing scripts, which today eases the process of copying your bits to download.eclipse.org, then unpacks your Update site so it can be scanned by p2 rather than downloaded as a single archive. It too is fairly trivial. I would not be adverse to converting my existing shell script for the EMF repo creation into a generic Ant script for use by Athena users.

Frankly though, I think it would be more valuable if the m2eclipse folks added support for reading/converting p2 repos. Publishing yet another file format would require another release-train-like workflow (we already have two: EPP and buckybuilder for Galileo) and more people maintain it. Even if every project published their own maven repo, we'd want for the sake of ease of use to aggregate them into a central place for easier navigation and discovery by maven tools. So, like with Ganymede, we'd have each project's bits copied to two places on disk for each build. (Galileo used composite repos to POINT at project repos rather than copying them saving tons of disk space and CPU cycles. AFAIK, Maven does not support this concept, but I could be wrong.)

There's also another benefit to having tooling to support converting from p2 repo to maven2 repo: the aggregate repo could be housed at apache.org and suck THEIR bandwidth and support resources instead. Thus just as Eclipse.org is upstream from Fedora's Eclipse project .rpms (which are upstream from Debian/Ubuntu's .debs), Eclipse.org p2 repos could be upstream from Apache's Maven repo(s). After all, Apache already collects maven artifacts for non-apache.org projects to facilitate the use and adoption of maven, so this is entirely in line with their standard operating procedures.

Thoughts?

Posted from Blackberry using Opera Mini, by the side of Shawnigan Lake, Vancouver Island, BC

2009-08-07

My love-hate with SVN, Part 8: Unprotected & Unhidden Metadata


SVN metadata appears when searching

Why would anyone ever want to see .svn folders and their children in Eclipse? If they're hidden from the Package Explorer, why can't I hide them from a Search? And if I did a naive find and replace, wouldn't this corrupt the metadata and prevent me from committing the update to the repo?

2009-08-06

Hudson & Virtual Box, Part 2: Creating a common share with vboxsf

My experiment last week with sshfs came to a grinding halt when I realized that while the share works, and can be automatically started, the permissions do not work and Hudson can't actually use the sshfs share; also, I couldn't get sshfs to compile for OpenSolaris, so it's really a non-starter.

Next up was mount_nfs, but after fighting with that, /etc/nfs.conf and /etc/exports for a couple days off and on, I realized there's a much simpler solution using Virtual Box itself: vboxsf. Note that it's sf (shared folder) not fs (file system).

Thanks to David Herron for the inspiration.

Here's how to set up a vboxfs share:

  1. Ignore the Virtual Box wiki which suggests that you can't just use a Shared Folder: you can.
  2. Guest: Launch your guest, then install the Virtual Box Guest Additions onto it.
  3. Host: Create a user and set its uid and gid to some value which can be created on the guest / slave. For example, on Mac OSX 10.5 Server, launch System Preferences > System > Accounts, and create a "hudsonbuild" user with User ID and Group ID set to 500.
    Note that uid 500 and gid 500 are the defaults for the first user on a Fedora system; on Ubuntu it's 1000; on other systems YMMV.
  4. Host: Create a folder in the root of your host called /shared. Put some files there. Give it full write permissions, and set its ownership to the user who'll be sharing the files:
    find /shared -type f -exec chmod 666 \;
    find /shared -type d -exec chmod 777 \;
    chown -R 500:500 /shared
  5. Host: Add a Shared Folder. NOTE: If your guest is running, shut it down or see the next item. From the Virtual Box GUI (on the host), select Settings > Shared Folders > New (+). Add a shared folder called "shared" mapped to /shared (or call it something else and map it to a different path on your host).
  6. Guest: Alternatively, you can add a Shared Folder to a running guest from its Devices menu. Add a shared folder called "shared" mapped to /shared.
  7. Guest: As root, create a folder on the guest called /shared. Leave it empty, but make sure it's owned by your user.
  8. Guest: As root, add this to the guest's /etc/fstab:
    shared  /shared vboxsf  auto,exec,rw,uid=500,gid=500    0       0
  9. Guest: As root, mount the share one of three ways:
    # mount everything in /etc/fstab
    mount -a
    
    # mount using mount (and optional -o options)
    mount -t vboxfs -o auto,exec,rw,uid=500,gid=500 shared /shared
    
    # mount using mount.vboxsf (and optional -w and -o options)
    /sbin/mount.vboxsf -w -o rw,uid=500,gid=500 shared /shared
  10. Guest: If this mount doesn't start automatically when the VirtualBox guest is started, you can add one of the above commands to the script you use to start the Hudson slave, eg:
    !/bin/bash
    #
    # Init file for Hudson server daemon
    #
    # chkconfig: 2345 65 15
    # description: Hudson server
    
    # HOWTO:
    # 1. rename this file on target server (Hudson slave node) to /etc/init.d/hudson
    # 2. to enable this script to run on startup, as root type the following:
    #    Fedora: 
    #        chkconfig --add hudson; chkconfig --list hudson
    #    OpenSolaris:
    #         for n in 0 1 6;   do cd /etc/rc${n}.d; ln -s /etc/init.d/hudson K15hudson; done
    #         for n in 2 3 4 5; do cd /etc/rc${n}.d; ln -s /etc/init.d/hudson S65hudson; done
    #         svccfg add hudson; svcs | grep hudson
    
    RETVAL=0
    
    #run Hudson as a slave node
    java -jar /opt/hudson/slave.jar -jnlpUrl \
      http://vboxsvr:8080/computer/${slaveNodeName}/slave-agent.jnlp &
    
    /sbin/mount.vboxsf -w shared /shared &
     
    exit $RETVAL