Much ado about scripting, Linux & Eclipse: card subject to change

Showing posts with label releng. Show all posts
Showing posts with label releng. Show all posts

2017-02-24

Streamline Nexus releases with nexus-staging-maven-plugin

For years, I've had a many-step process for releasing Maven artifacts to the JBoss Nexus, which has made me reticent to do frequent releases due to the heavily manual process.
  1. Commit a change to the artifact's pom, replacing version x.y.z-SNAPSHOT with x.y.z
  2. Build the artifact in Jenkins (often triggered automatically via Github hooks) and deploy it to staging Nexus instance using server credentials in Jenkins
  3. Log into Nexus
  4. Browse for the Staging repos
  5. Sort the list by most recently updated
  6. Look for the "Implicitly created (auto staging)." entries
  7. For each entry, select the Content tab and drill down to see if that entry is the correct one (could be another job or person's staging bits)
  8. If the entry is correct, click the Close button and enter a description; if not, check the next one in the list until you find the correct one
  9. Click Refresh button
  10. Click the Release button
  11. Browse Nexus to verify the artifact is deployed

But no more! 

With the nexus-staging-maven-plugin, I can simplify that process by adding more Maven steps the job.
  1. Commit a change to the artifact's pom, replacing version x.y.z-SNAPSHOT with x.y.z
  2. Build the artifact in Jenkins (often triggered automatically via Github hooks) and deploy it to production Nexus instance using server credentials in Jenkins
  3. Browse Nexus to verify the artifact is deployed

So, how do you enable this?

  • Ensure you have all the correct metadata configured in your pom: description, URL, license, developers, SCM, issueManagement, etc. 

  • Perform three steps in Jenkins

That's it!

Or, to enable this selectively for only releases (but not SNAPSHOTS):

pom=${WORKSPACE}/path/to/pom.xml
pomVersion=$(grep "" ${pom} | head -1 | sed -e "s#.*\(.\+\).*#\1#")
MVN="/path/to/maven3/bin/mvn -Dmaven.repo.local=${WORKSPACE}/.repository/"
if [[ ${pomVersion} == *"-SNAPSHOT" ]]; then 
  ${MVN} deploy
else
  ${MVN} clean deploy -DskipRemoteStaging=true -f ${pom} \
    -DstagingDescription="[${JOB_NAME} ${BUILD_TIMESTAMP} ${BUILD_NUMBER}] :: ${pomVersion} :: deploy to local" 
  ${MVN} nexus-staging:deploy-staged -f ${pom} \
    -DstagingDescription="[${JOB_NAME} ${BUILD_TIMESTAMP} ${BUILD_NUMBER}] :: ${pomVersion} :: deploy to stage + close"
  ${MVN} nexus-staging:release -f ${pom} \
    -DstagingDescription="[${JOB_NAME} ${BUILD_TIMESTAMP} ${BUILD_NUMBER}] :: ${pomVersion} :: release"
fi

2014-12-11

A Day In The Life

Can't believe it's been a full two years since I last posted to this blog. Time flies when you're making change.


I read the blogs today oh boy
About more JS frameworks on their way
And though the news was rather sad
Well I just had to laugh
I saw the infograph

It was yet another approach
To solving the same things of yesterday
A crowd of people stood and trolled
They'd seen this stuff before
Nobody was really sure
If it was like IO or Node

I saw a tweet today oh boy
Toronto street car service is still slow
They've crowds of people every day
But I still have to drive
Cuz of where I live
I'd love more subways here

Woke up, fell outta bed
A Beatles earworm in my head
Found my way down th'hall and dialed my call
And sitting there I noticed sound was dead

Found my phone in seconds flat
Bluejeans worked, so that was that
Found my way online (had to reboot)
And somebody spoke and I went into a dream

I read the logs today oh boy
Thousands of commits by hundreds of folks
Though some issues were rather small
We had to count them all
Now we know how many tweaks it takes to make a new release
And how to turn it on

2012-12-07

The JBoss Developers' Song

Hey! Guess what?

JBoss Tools 4.0 and JBoss Developer Studio 6.0 are available today. So... a quick tune in tribute to all the hard-working people who made it happen.

Who squashes bugs users have found?
Who keeps the unresolved count down?
We do! We do!

Who answers to the Will of Max?

Who deals with all of the PEBCAKs?

We do! We do!

Who closed/resolved a thousand bugs?

Who attends all of those JBUGs?

We do! We do!

Who fills more roles than Tom Hanks?

Who gets the top Marketplace ranks? 1

We do! We do!


1 - As of 2012/12/07, JBoss Tools + JBoss Developer Studio successful Eclipse Marketplace installs for Helios, Indigo, and Juno total over 141,000. SpringIDE and Spring Tool Suite installs total over 163,000. This puts us #5 behind only Maven, Subclipse, Subversive, and Spring.

2012-06-19

Open Source is Painless (Theme from JBDS)

With JBDS 5.0.0.CR1 and JBoss Tools 3.3.0.CR1 out, and GA/Final releases just around the corner, it's high time for a new song.

Hell's bells, it's been over 3 years since my last music-related post.

The idea for doing this song came after spending a day creating a diagram showing how JBoss Tools and Developer Studio are built, followed by myarboro's reaction to the complexity.

Diagram by Dia

Through early morning mail I see
Visions of JIRAs for me
Build jobs red unexpectedly
I realize and I can see...

That Open Source is painless
Even with non-stop changes
And yet we give all this away for free

The game of rel-eng's hard to play
We struggle through it anyway
A business card I'll never lay
Deprecated: it's LinkedIn's day

Jenkins builds are painless
They track so many changes
And ever onward moves complexity

The use of Git trumps SVN
But migrating's hard to begin
When your code base is not so slim
The pain grows stronger... bear & grin

Using github's painless
Pull/push requests for changes
And I can take or leave it if I please

A Turing test once asked of me
To answer questions that are key
'To use Tycho or PDE?'
And I replied 'Not PDE!'

Releng'ing is painless
It is a game of changes
And still we give all this away for free
And you can do the same thing if you please...

Johnny Mandel & Mike Altman - Suicide is Painless (Theme from M.A.S.H.)

2012-06-06

Managing Jenkins job configurations

In JBoss Tools and Developer Studio, we manage a lot of build jobs in Jenkins. In fact, for the 3.2.x/4.x and 3.3.x/5.x streams, there are over 195 jobs. When we start building our next year's first milestone, we'll spawn another 40+ jobs.

Here are some of them:

To assist in performance, we use maven profiles in our parent pom to allow data to be shared outside the slaves' workspaces, without using shared workspaces (as that can lead to conflicts when multiple maven processes try to write to the same .m2 repo). Here's an example:

  <!-- same contents as jbosstools-nightly-staging-composite, but locally 
   available (to improve network lag) -->
  <profile>
   <id>local.composite</id>
   <activation>
    <activeByDefault>false</activeByDefault>
   </activation>
   <repositories>
    <repository>
     <id>local.composite</id>
     <url>${local.composite}</url>
     <layout>p2</layout>
     <snapshots>
      <enabled>true</enabled>
     </snapshots>
     <releases>
      <enabled>true</enabled>
     </releases>
    </repository>
   </repositories>
  </profile>

We also use profiles to turn on code coverage analysis or to take advantage of Jenkins variables like BUILD_NUMBER when setting a timestamped qualifier for features & plugins:

  <profile>
   <id>hudson</id>
   <activation>
    <property>
     <name>BUILD_NUMBER</name>
    </property>
   </activation>
   <properties>
    <local.site>file:///home/hudson/static_build_env/jbds/target-platform_3.3.indigo.SR2/e372M-wtp332M.target/</local.site>
   </properties>
   <build>
    <plugins>
     <plugin>
      <groupId>org.eclipse.tycho</groupId>
      <artifactId>tycho-packaging-plugin</artifactId>
      <version>${tychoVersion}</version>
      <configuration>
       <format>'v'yyyyMMdd-HHmm'-H${BUILD_NUMBER}-${BUILD_ALIAS}'</format>
       <archiveSite>true</archiveSite>
      </configuration>
     </plugin>
    </plugins>
   </build>
  </profile>

But how do you deal with hundreds of jobs' config files, and how do you update them all easily w/o hours of in-browser clicking?

We maintain the job config files (config.xml) offline.

To do this, we use a maven plugin I wrote to fetch jobs matching a given view & regular expression and store them locally on disk using the same structure as on the server.

Then that same plugin can be used to push the config.xml files BACK to the server after making changes to one (or all) the files.

For an extra level of auditing, we also commit these locally cached config.xml files to SVN so we can track their history. Admittedly Jenkins provides this functionality natively, but when you're POSTing changes to config.xml files the server doesn't always notice a change and record the delta, so having a backup (particularly one you can diff offline) is never a bad idea.

2011-12-11

Build Nomenclature Conventions: What's in a name?

The following post is inspired by Mickael Istria's recent blog, Call a spade a spade, and a Nightly a Snapshot.

When I was doing builds for the Eclipse Modeling Project, I-builds were weekly published nightlies -- same level of stability as a SNAPSHOT (to use Maven parlance) or nightly, but published on a weekly schedule to bridge the gap between nightly/daily/SNAPSHOT/CI builds and the every-6-weeks milestone releases. The goal was to provide something stable enough for early adopters to grab once a week, but without the non-stop flux of nightlies. Regardless of the label on the build, the process was the same: tag CVS, then build using that tag.

The Final/GA/Release ("R") builds were done as simple renames of the last good milestone or release candidate build, so as to ensure binary-compatibility w/ the last-tested milestone/RC. The same was true for "M" and "S" builds -- they were just renamed "I" builds, and the letter was there simply to differentiate between a maintenance build (M), a stable milestone (S), or release (R).

Branching only happened when a release was done and it was time to produce the maintenance stream vs. the ongoing next-year-release. Sometimes branching would happen AFTER the x.y.1 maintenance because it saved duplication of commits in the x.y+1.0 and x.y.1 streams.

--

Now at JBoss, we publish "nightly" builds, which are keyed to SVN changes and therefore could be as often as hourly or as infrequent as weekly, depending on what's happening in the repo.

We also do milestone builds about once ever 6-8 weeks (similar to the Eclipse.org release train schedules), which is more carefully vetted, tested, and QE'd. It is produced using the same *process* as the nightlies, but are named differently and pulled from a freshly-created stable branch in the repo (so its degree of change/churn is less). (Branching happens right before every milestone or release candidate so that hardening/stabilization/documentation can happen in the branch while trunk stays open for new development.)

--

Bottom line -- I've only ever needed three types of builds, regardless of nomenclature or labelling differences. And of these 3, the last 2 are the same thing but renamed to underline the build quality/stability:

* nightly/CI/integration/weekly/SNAPSHOT build (unstable, for bleeding edge adopters)

* development milestone (probably a re-christened nightly; stable, early adopters)

* stable release / Final / GA (probably a re-christened milestone; release quality)

--

So... does it matter if it's called nightly, integration or SNAPSHOT? or Stable, Milestone, Maintenance, Final, GA or Release? As long as it's easily reproducible (yeah, Tycho!), what's in a name?

2009-05-20

Use Your Metadata, Vol. 1

It's been a bad week for update sites and Galileo contribution from Modeling... and I confess I'm partly to blame. That and the fact that despite documenting processes, workarounds, tips, tricks, and advice... no one Reads The Fine Mediawiki (Category:Releng or Modeling Project Releng).

Highlights:

  1. The mysterious appearance of a new version of org.eclipse.osgi_*.jar in releng.basebuilder's R35_M5 tag, which caused an ant <copy/> used to rename a file to fail because copy can't merge two jars into one file. Still no idea why an old basebuilder tag would magically grow a new jars, but I've worked around the now-faulty assumption w/ smarter Ant code.
  2. A change to the way our sites are created, in an attempt to workaround what I believe (but can't yet prove) is a flaw in the way content.xml is produced - namely, if the xml file is > 21M, it gets truncated or corrupted. We used to cache 2 or 3 releases of a given project (eg., M5 and M6) on the same site, in order to give people a way to "back up" to the previous release; now, you only get the latest (bug 271486). I confess I screwed up here and instead of replacing a folder with new contents, I was copying INTO that folder - `cp -r one two` resulted in folder one/two/ instead of two/. I fixed that by changing to a move instead of a copy, but a downstream process failed because of the assumption that both one/ and two/ would exist. The lessons here are: a) shotgun debugging sucks, and b) don't change the way stuff is created after M7.
  3. People publishing two updates to a site at the same time, resulting in the appearance of two </site> tags in a site.xml file, causing p2 metadata generation to be incomplete or fail entirely; unfortunately, no error is logged when this happens so it's rather difficult to decipher the tea leaves. This may be the real source of the metadata corruption, if not the "file is too big" issue above.
  4. Observations about obsolete jars corrupting metadata, but no one taking it upon themselves to clean up the site or do some troubleshooting
  5. People inconsistently naming their milestones (it's 2.0.0M7, not just M7!) and corrupting our Release Notes database. This one amazes me the most since it takes seconds to see what was done last time (check any of the following: RSS feeds, release notes, downloads pages, update sites) and follow suit. And, of course, the conventions are documented, along with the rationale (consistent patterns == simpler code).

Or, to put it another way...

Sick of this life
Not that you'd care
I'm not the only one with
whom these feelings I share

Nobody understands
Quite why we're here
We're searchin' for answers
That never appear

But maybe if I looked real hard I'd
I'd see your tryin' too
To understand this life,
That we're all goin' through

Sometimes I feel like I'm beatin' a dead horse
And I don't know why you'd be bringin' me down
I'd like to think that your love's
Worth a tad more
It may sound funny but you'd think by now
I'd be smilin'
I guess some things never change
Never change

So, please, can we stop opening bugs (277172, 277105, 277034, 276928, 276641) and just use the tools and docs already available?

2009-04-18

All I really want is builds

With April 20 just around the corner, it's time to get ill... but first, something completely different.

Builds, all I really want is builds
And in the morning it's builds
'Cause in the evening it's builds

I like the way that they run
And when they're green they're so much fun
And I can always make 'em smile
Fetch all their sources and compile

Back in the day
There was this project 'round the way
It had no build system per se
But they kept codin', come what may

I asked them, "Please?" they said, "You may."
Their code was tight and that's ok
If they would code I'd be R.E.
And now it runs on build-dot-E

I hope you'll say, "Hey, me and you should try Athe-
na, more to come, but under way!"
We really need a website, eh?
For now slides, code, and the wi-kay

We'll move to Git when it's read-ay
But that won't be before this May
So get yourself a build today
Without the ol' P-D-E dismay

Builds - to do the fetchin'
Builds - do the compilin'
Builds - to do the testin'
Builds - and zipped-up filin'
Builds, that's all I really want is builds
Early adopters they want builds
In next-gen Hudson they want builds
Signed, packed, p2 repos...
Builds, builds, builds, builds, builds!

NOTE: Normally the artist name below links to the video on YouTube, but there's no official one, and everything else I found (school projects, Lego animation, stick figures, slideshows) were either incomplete, incorrect, or potentially NSFW. YMMV.

Beastie Boys - Girls

2008-11-02

Request For Comment: The End Of SDO?

Four months ago, I started a discussion about simplifying the EMF download experience. Consensus appears to be that we should keep the EMF-SDO-XSD "All-in-one" SDK zip, perhaps more.

Now, a new proposal is in the works: removing SDO from the EMF build, and from this year's coordinated release, Galileo. The source would still be in CVS, but we'd stop providing SDO binaries. This would also mean the much-loved* EMF-SDO-XSD "All-in-one" SDK zip would be renamed to EMF-XSD "All-in-one" SDK, but would otherwise continue as is.

For more on this topic see:

EMF 2.5 - Simplified Downloads (bug 240223)
Remove SDO from Galileo (bug 251402)
Consider improvements to the termination process (bug 249408)

To voice your opinion, please post your feedback in any or all of the three bugs linked above.

Thanks!

2008-07-09

Request For Comment: Will Anyone Miss All Those EMF Zips?

With the advent of p2 and the smaller EMF features introduced in EMF 2.3, the EMF team is evaluating if the All-In-One, Runtime (3), Sourcedoc (3) & Examples Zips are still meaningful and useful to consuming teams, projects, and products. We'd like to reduce those 8 zips down to a single archived update site zip.

For more on this topic see:

EMF 2.5 - Simplified Downloads

To voice your opinion (be it 'I need that/those zip(s)!' or 'I'm good with using Update'), please post your feedback in bug 240223.

Thanks!

2008-01-25

Building Change

Seems everyone's talking about change these days. Hell, people are even ranting about the way media is changing.

But real issues aside, in part two of this series, I'd like to briefly mention two changes happening closer to home, here in releng-land:

  • EMF, SDO, and XSD are now being packaged a little differently. Consider this a little social experiment to see who, if anyone, still uses zips to get their downloads, rather than using Update Manager. Do you like the new "sourcedoc" zips? Feel free to let us know what you think (here, on IRC in #eclipse-modeling, or in the newsgroups).
  • GEF is now being built by the Common Modeling Build, the first (of many, I'm sure) projects to come over to the dark side. Will DSDP or VE be next? Time will tell.

2007-11-15

Happy Usability

I'm sure you've all heard is a million times. Boy meets tech. Boy falls in love with tech. Boy documents tech. Boy shares tech with others. Others read documents, scratch heads. Doc gets rewritten. ;-) Yes, this comes from recent personal experience. Just a day in the life.

dev screams he thinks he knows what you want
dev screams thinks he has something well said
dev screams hears feedback but doesn't wanna listen
dev screams he's being beat into submission

don't open your eyes you won't like what you see
designing for truth or usability
don't open your eyes take it from me
i have found
you can find
happy usability [x2]

dev screams he spends his life rewriting wikis
dev screams he claims it's fun - yeah right - no really
dev screams he's got to start all over again
dev screams at least sometimes others share my pain

don't RTFM you won't like what you see
there's too much detail or a scarcity
don't open your eyes take it from me
i have found
you can find
happiness in ID [x3]
happiness

i don't know what i am but i know where i've been
releng work just words so much buildin'
stick my hands in the code of this endless routine
help the folks caught in this big modelin' machine

happy in codeslavery

nin - happiness in slavery

2007-10-21

Rules

NOTE: For the record, there should be no inferred or implied interpretation of the selection of the source tune used for this parody. Simply, I was out walking my dogs this afternoon through Curran Hall Ravine, listening to that album, and it struck me as a perfect rhythm to talk about the benefits of structure and rules in terms of clarity, consistency, and ease of use. Ed uses photos & analogies, I use punk songs & parodies.

I don't want anyone to think that I'm trying to imply that the EMO's rules are somehow bad, malicious, or simply just lies. ;-) Au contraire, Bjorn's readiness to improve processes and simplify overhead proves the opposite. I'm well aware that we're all in this together and the point of setting rules and standards is to ensure things are done legally, consistently, and without perceived bias toward any given member company or other.

Anyway, enjoy...

Rules

Stored conversations, open processes
Some think we're stupid, that's their thing
But we build useful tools, coordinate with rules
So we can overcome

Lambasting critics, with naught but rhet'ric
Your accusations have worn thin
But we've got work to do, and out in public, too
We've got one on you!

Rules... make it better
Rules... aren't forever
Rules... to be consistent
Rules... for the betterment
Rules... from the EMO
Rules... from the Councils
Rules... keep IP clean
Rules... 'till review's done... rules

Management office, process designers,
With best intentions at heart, yeah
Try to improve workflows
Despite critical shmoes
Truly this job blows

But when there's...

Rules... (to) make it better
Rules... (that) aren't forever
Rules... simplification
Rules... for automation
Rules... don't like 'em? change 'em
Rules... we'll rearrange 'em
Rules... keep things structured
Rules... 'nuff damn lectures...
[x2]

RULES!

Rules are designed to serve you
Committer reps: your way for change
Rules are designed to serve you
Contact their home on the range!

Rules... make it better
Rules... aren't forever
Rules... to be consistent
Rules... for the betterment
Rules... from the EMO
Rules... from the Councils
Rules... keep IP clean
Rules... 'till review's done...

Rules... (to) make it better
Rules... (that) aren't forever
Rules... simplification
Rules... for automation
Rules... don't like 'em? change 'em
Rules... we'll rearrange 'em
Rules... keep things structured
Rules... 'nuff damn lectures... rules

Billy Talent - Lies

2007-10-19

The Three Laws Of Eclipse

Bjorn just posted his first draft of The Three Laws Of Eclipse, ostensibly in response (in part anyway) to my perceived piercing from yesterday. Clearly not everyone was amused by my attempt at levity, and for that I'm sorry. That was not my intent.

In my humble opinion, The Three Laws is *EXACTLY* what we need. Something simple, concise, and easily referenced. (Unlike the rambling blog entry that will now follow.) ;-)

A couple of years ago I took a course about presentation skills called Think on your Feet (developed by McLuhan & Davies -- yes, that McLuhan), and one of the big things it stressed was what I'll call 'the rule of three':

  1. Three bullets to a section, three items on a slide.
  2. Three aspects of a topic (points in time, degrees of complexity, etc.).
  3. Three parts of a talk (intro, content, summary).

The human brain works well with the number three, for many reasons I won't bore you with. If you have a chance to take the course, I highly recommend it.

The reason I mention it is because the 'Three Laws of Eclipse' works perfectly in tune with the guidelines from that course -- you have three major rules, and the rules break into no more than 3 subpoints. This would work great as a powerpoint (for execs, PMCs, ID people, lawyers, policy people) and also as a quick wiki/webpage, as currently formed (for the geeks). Different audiences, different format; same content, same themes.

Because it's simple, it's memorable. Because it's memorable, it will stick with people.

And because it will stick, we'll stop feeling that the rules change too often. Or that there's too many.

Just for the sake of contrast, have a look at this 29-section legal-ish-style document.

Now, look at this page, where we find an explanation and history (in the linked bug) about why the document was changed, and everything relevant is conveyed in 3 (well, 4) easily-digested sections.

As a side note, perhaps annual changes aren't too often. As David says, the need for change is normal. Perhaps, as Mik points out, the problem is communication. This is IMHO twofold:

a) announcements of changes are not communicated to Joe Average Committer -- this could be my fault for not being subscribed to the right mailing list. Should these be done on cross-projects-issues-dev@ and/or the eclipse.org-planning-council@ lists? Or as a committer-wide announcement like the notes Denis sends once in a while about outages and new infrastructure? I'd say all of the above.

b) changes (due to (a)) seem to happen without a stated reason for the change. Again, this could be just the way my brain works: I need to know WHY a change is happening, even after I've accepted it's a good thing. And I need to be able to easily look up when and why so that when I've forgotten something (as I am wont) I can be shown the error of my ways by my own search or that of a friendly reminder from Bjorn. (This works both ways -- I nag Bjorn, he nags me, and over time we both fix things and improve. Win-win.)

So while I could probably pull the CVS history of documents and see when/why, it's not nearly as friendly as having a quick note sent to the committership at large saying "this rule has been changed from ___ to ___ because of problems with ____ so we're trying a new approach this year." Or better, having a bug to point at, like this page with this bug.

Oh, and just in case anyone out there thinks that from up on this pedestal of mine I don't strive to practice what I preach, contrast before with after, when I realized that the Modeling Releng doc had gotten too big for wikipedia to handle it, much less human users. (It's still too damn big, but hopefully more manageable when chunked up into smaller docs.) I'm open to suggestions if you have a better way.

Seems that 43 years later, it's still true. The way you convey information is often more important than the information itself.

2007-10-17

Release Review Today!

UPDATE: Clearly not everyone was amused by my attempt at levity here. My apologies.

Wake up, the EMO's on fire
And they're sick of policing th'entire
World of Eclipse releases
Cuz the PMCs should be

"Rename" is the last chance for hope
When there's no such thing as unrelease
Your GA build's in the ditch that
You dug yourself in

[Pre]
Last chance to get your slides in place
Nothing but hell to pay
If you forget your review

[Chorus]
Release Review Today - celebrate your SDK!
Post your N&N and hang 'em high
On the wiki and your home
Release Review Today - get on your knees and pray
Forget your review and you'll be screwed
But at least you're not alone

Marquee lights in the middle of June, now
Old bugs come back, no boon now
Your oldest allies are your long lost PMCs
Sit though the review and you'll find
That it's a +1'ing good time
Unless you're a hotbed of non-diversity

[Pre]

[Chorus]

Review Day! Vote +1!

[Pre2]
Last chance to get your slides in place
But the policies seem to change each day
When all you wanna do is... try to... comply

[Chorus] x 2

Review Day!

Green Day - Deadbeat Holiday

2007-06-29

So long, and whoops for all the spam

After zx challenged me to start talking more about what I do, I created a second blog -- this time at eclipse.org. I did this for three reasons:

  1. To provide a place to talk about release engineering away from the other clutter on this blog;
  2. To encourage other release engineering folks to provide posts here as well; and
  3. Because I'm a little sick of Blogger (preview sucks, no html shortcuts) and am ready to try WordPress for something new

Unfortunately, while importing old releng-related entries to the new site, I seem to have accidentally caused some old posts to be syndicated on Planet Eclipse, and so for everyone who's just been inundated with my old pre-syndication posts, my apologies.

Anyway, please watch the new releng blog for stuff about my work. If you want to be an author/contributor on the new blog, sign up, then drop me a line and I'll add you.

2007-06-23

And Now For Something Completely Different...

Much like Ian's post the other day, I've always wanted to blog that line. Anyway, it's been brought to my attention that of late I've been apparently posting less-than-positive things about Eclipse, so to balance that out, here's my top three cool UI features in Eclipse 3.3, in order from oldest to newest:

#3: Use PDE UI to generate an Ant build.xml script for a plugin project

This has been around for years, true, but it's certainly top on my list of time-savers. Here's a quick HOWTO for taking a plugin and ending up with a versioned, Update Manager-compatible OSGi bundle.

  1. Create, import, or extract a plugin project from CVS.
  2. Right-click the plugin's plugin.xml and select PDE Tools > Create Ant Build File.
  3. Open the Ant view (CTRL-3, Ant or SHIFT-ALT-Q,Q > Ant) then drag the newly-generated build.xml into that view to see all the available targets. It's like the Outline view, but you can launch targets by double-clicking them.
  4. Double-click the build.update.jar target to run that build. You should see a new jar created in the root of your plugin project. (If you don't, select the plugin project's folder in the Package Explorer, Project Explorer, or Navigator view, and hit F5 to refresh.)
  5. You can run other targets, like src.zip to create a source zip, or edit them to suit your needs. If you delete the build.xml you can always regenerate it from the latest template.

#2: Use Package Explorer + Working Sets + Filters to clean up your view

I've been using these for at least a year by now, but only recently discovered that they behave differently in the Package Explorer than in the other filesystem views, like Navigator and Project Explorer. The best by far, for me, is the way the Package Explorer does things. Here's a shot of how one of my rather project-heavy workspaces looks in the Navigator view, compared to the Package Explorer with Working Sets set as my Top Level Elements:

To make this work, open the Package Explorer view, then click the menu icon (an arrow pointing down in the top right corner of the view -- I'll just call it Menu for simplicity). Here's a few things you can do to alter your view:
  • Toggle Working Sets / Projects:
    Menu > Top Level Elements > {Working Sets | Projects}
  • Select what Working Set(s) to show (when TLE = Projects):
    Menu > Top Level Elements > Projects, then
    Menu > Select Working Sets...
  • Select, create, edit, or delete Working Sets (when TLE = Working Sets):
    Menu > Top Level Elements > Working Sets, then
    Menu > Configure Working Sets...
  • Hide closed projects, empty packages, libraries, non-Java files, etc.:
    Menu > Filters

#1: Use Autopin UI Tweaklet to manage open editor tabs

This is my most recent discovery, and once you get used to it, a great new way to manage editor tabs.

First, download & install the Tweaklet plugin(s) from either the Platform UI Incubator downloads or updates site.

Then, marvel as a whole new way of working with tabs is revealed: you only ever have one tab in use until you start editing that file. When you do, the Autopin tweaklet automatically pins the tab, and subsequent files are opened in a second one, then a third, and so on.

Like Working Sets, this is also fantastic when dealing with massive workspaces or when you have a lot of files that reference other files. Eg., when drilling down from one Java class into another and into another to find the method you need, or for PHP scripts that include() or require() others, again, looking for methods or variable declarations, or even in concert with the full-text search (CTRL-H, first tab) to find that elusive build-related metafile which needs to be fixed, without ending up with 20 open files. Kick ass.

The bottom line here is that there's lots of hidden features in Eclipse to make your life easier and your work more efficient. And because we all work a little differently, there's lots of ways to tweak the way the UI behaves to accomodate all our differences. Eclipse is you? Aye!

2007-06-19

Thirteenth Step

Ye gods. As of last week, my little release engineering world is responsible for a total of 13 builds, with two more on the way. Time to crank up a little APC and have a quick look back...

  1. 1Q2004: pre-PDE build system migrated to PDE thanks to Marcelo Paternostro.
  2. 2Q2004: EMF 2.0 released, using Eclipse 3.0 releng.basebuilder. [Builds/branches/codebases: 1/1/1]
  3. 2Q2004: UML2 1.0 released, using cloned/simplified version of EMF builder. [2/2/2]
  4. 2Q2005: EMF 2.1 released, updated to Eclipse 3.1 builder. [2/3/2]
  5. 2Q2005: UML2 1.1 released. [2/4/2]
  6. 3Q2005-4Q2005: EMF 2.2 and UML2 2.0 start development, adding two new branches.[2/6/2]
  7. 4Q2005: first EMF Technologies project starts releasing builds, just before Xmas. Before long, the infant project has crossed the Rubicon, with 4 components: OCL, Query, Validation, and Transaction. [Builds/branches/codebases: 6/10/3]
  8. 1Q2006: EODM and JET join the EMFT party, solidifying the 'cross-project' nature of the EMFT build. 8 different builds supported by 3 code bases, across 1 to 3 active CVS branches. [8/12/3]
  9. 2Q2006: JET Editor, Net4j, and CDO are added to EMFT, and begin releasing builds. Callisto released, including EMF 2.2. Also along for the ride as part of GMF are OCL 1.0 and QTV 1.0 (Query, Transaction, Validation). System updated to use 3.2 basebuilder. [11/15/3]
  10. 3Q2006: Teneo joins EMFT. [12/16/3]
  11. 4Q2006: EMF, UML2, OCL, QTV (3) and JET/JET Editor (2) start work on new branches, in time for M3. Common Modeling Build started, adding new MDT component UML2 Tools. System updated to use 3.3 basebuilder. [13/25/4]
  12. 1Q2007: EODM branches for M4. [13/26/4]
  13. 2Q2007: EODM, OCL, QTV, UML2, JET & JET Editor migrated to Modeling Build and moved into EMF, MDT and M2T websites, making the old UML2 build obsolete and merging the old JET and JET Editor builds into a single website/build. EMF Compare build added to new Modeling.EMFT build. Common Modeling Build now supports multiple branches (Eclipse 3.2/3.3) and nine different component builds (across 4 different projects: EMF, EMFT, MDT, M2T), with varying numbers of upstream dependencies (from 1 to 9) & support for bundling or build-time use of 3rd party code (Orbit and non-Orbit). Original EMF build that started it all supports 4 branches and several non-JUnit test types. Three semi-dormant Technology.EMFT projects left to migrate (Teneo, CDO, Net4j). [Builds/branches/codebases: 13/27/3]

And there's so much still to do! No wonder I'm so tired these days...

2007-06-12

That Ol' PDE Black Magic

... or more reasons why features still suck.

In the past 24hrs, I've encountered two feature-related PDE issues, which I think warrant discussion as they are, to the uninitiated, seemingly black magic. The first of these was discovered and fixed last night while watching The Dresden Files in ten-minute chunks while waiting for builds to complete, but I'm sure that's just a coincidence.

Bug The Firste: feature.xml should not specify version numbers when including required plugins like org.eclipse.test [192231].

This issue manifests when your build stops working one day and complains, "eclipse/test.assembly/eclipse/plugins/${org.eclipse.test} not found." The fix is simple: make sure you set version="0.0.0" in your feature.xml when including org.eclipse.test or other plugins referenced in your map file and required by your build. See also Build Problems.

Bug The Seconde: PDE will build features in the order they're listed in feature.xml [192292].

This issue manifests when your source feature, for example, builds OK but contains no source because PDE is building it BEFORE your runtime feature(s). Once again, the fix is simple. You just have to reorder your feature.xml to put your source feature(s) last.

Bug The Thirde (Bonus Bug!): If your plugin or feature doesn't have the right Nature and Builder in its .project file, it won't behave the way you'd expect [192247] in Eclipse.

This one's a no-brainer, but also a handy tip when working with PDE. If your plugin project doesn't have the Plugin nature, you can't open a MANIFEST.MF file with the Manifest Editor; if your feature doesn't have the Feature nature, PDE won't give you warnings/errors if your feature is misconfigured (eg., if you have a version set to "o.o.o" instead of "0.0.0"). Just add something like this:

<buildSpec>
  <buildCommand>
    <name>org.eclipse.pde.FeatureBuilder</name>
    <arguments/>
  </buildCommand>
</buildSpec>
<natures>
  <nature>org.eclipse.pde.FeatureNature</nature>
</natures>

2007-06-09

Request For Comment: Will anyone miss the EMF Standalone Zip?

With the advent of jarred plugins, improvements to Update Manager, and all the new smaller EMF 2.3 features (including 7 core "runtime" ones in Europa's Enabling Features category -- see bugs 106804 and 189295), the EMF team is evaluating if the Standalone Zip introduced in EMF 2.1 is still meaningful and useful to consuming teams, projects, and products.

For more on this topic and to learn what's actually in this zip, see:

http://wiki.eclipse.org/index.php/EMF_2.3_Standalone_Zip

To voice your opinion (be it 'I need that zip!' or 'not useful to me'), please post your feedback in bug 191837.

Thanks!