Friday, June 30, 2006

Callisto: the day that just flew by

About 20 (if not more?) committers showed up for the Callisto celebration at the foundation offices Friday afternoon. It was nice to see everyone, and I met a few committers I hadn't met before. Everyone anxiously waited for us to press the Big Button - the one that makes the release files available to the download servers -- but we played it cautiously and waited for a few mirrors to sync up.

Despite the release delays, a last minute break happened earlier in the morning when I got an e-mail from Neil at Georgia Tech saying "we're now an Eclipse mirror and we're up to date". Way cool -- these guys have loads of bandwidth, and I had asked them to become a mirror months ago. The same university gave us a second break in the afternoon, when they said "Callisto, huh? Need some help?". They were the first USA mirror with all the files.

So this year, despite a few hours of delays, it all went off without a hitch. Lots of planning and testing went into making sure the bits were easy to get to once released. peaked at about 215 Mbps late in the afternoon.

The following people are way cool: David Williams for being tireless; Platform Releng for beer; Neil at for being a saviour; John at iBiblio (I owe this guy a beer - thanks again for the torrents); Pascal for the humour; Mike, Wayne and Nate for guidance, testing and moral support, and last but not least, Matt, for coming up with the best ideas when I needed them most.

Update: link to iBiblio was wrong

Callisto is here! Really!

So it's a bit late... but Callisto is here! Head off to the downloads page now!

Packing the bits ... angry committers coming to Foundation office

David is graciously packing the final bits for Callisto. After that, we wait for a few mirrors...

Can't talk, must watch processes...

Callisto is delayed - celebrations are not

Callisto will be delayed for a bit. Unforeseen circumstances. Uncontrollable events. We don't have an ETA yet, but stay tuned for information. We do expect it to be released today.

Committers can still come celebrate with us at the Foundation offices this afternoon while we wait for mirrors to catch up. We'll then be able to "flick the switch" collectively. How cool is that?

Thursday, June 29, 2006

More Callisto beer donations!

After blogging about the generous $50 beer donation from Platform Releng, Mike offered to match the donation. So I now have $100 worth of beer, and I need help drinking it.

So I'm confirming my invitation to Eclipse Committers in Ottawa to kindly join us for a beverage this friday afternoon at the Foundation's offices, to celebrate the simultaneous release of 10 Eclipse projects (and to watch our servers explode). Doug expressed interest (bring your team!), as did some of the Ottawa IBM committers. Kim will be bringing snacks.

If you're coming, drop me a line indicating your favorite beer. You know how to reach me :)

Wednesday, June 28, 2006

Callisto beer donations going around!

Tuesday morning Matt and I received a card in the mail. The return address is platform-releng-dev. Hmm, what's this all about?

Inside, words of gratitude, "pop across the street for some Callisto juice" and a gift certificate for $50 from the BEER STORE, signed by Kim and Sonia.

FIFTY BUCKS! Wooohooo! How so very nice!

But we didn't do anything special. Callisto is a group effort, and we all deserve this treat. That's why we're inviting committers in Ottawa to stop by and have a beer with us Friday afternoon! The Foundation office is just a hop away from the IBM Ottawa lab.

Thanks, Kim and Sonia!

Wednesday, June 21, 2006

How much are mirror sites worth?

250,000 $CAD per year.

That's a million dollars every four years. Heck, with these savings, the Foundation could buy me a new Porsche every quarter. Wouldn't that be awesome.

Yesterday was Mirror Appreciation Day -- we disabled sending traffic to our mirror sites to test our setup for the upcoming Callisto event. Also to give our mirrors a break. But mostly to test.

I ended up concluding that, without mirror sites...

- we'd be spending an extra $250,000/year in bandwidth alone
- we'd sustain about 1200 Apache sessions on any given day, instead of 400
- we'd need to purchase additional highend firewalls and switches

Until each popular browser includes a built-in bittorrent client, mirror sites will continue to be budget savers for communities like ours.

Monday, June 19, 2006

Mirror site appreciation day - June 20, 2006 !

I'm declaring tomorrow, June 20 as "Mirror Site Appreciation" day. Why?

- Mirror sites provide valuable resources (bandwidth, servers, SysAdmins) for the entire community. Without them, getting Eclipse would be infinitely harder, so I thought that their precious contributions should be acknowledged in a special way.

- I need to test our servers and infrastructure to make sure they are ready for the Callisto onslaught next week.

To accomplish both goals, we will (attempt) to handle *all* of our download traffic for the entire day (why do I do this to myself???). As I understand it, this hasn't been attempted since the late Eclipse 2.x releases, back in the days when Eclipse didn't have any mirrors at all.

Oh, and I reserve the right to chicken-out of this insanity at any time during the day if I feel we can't handle it. Happy downloading!

Monday, June 12, 2006

Callisto bandwidth - how much will it cost?

Last year, when Eclipse 3.1 was released, we added our own mirror server with a dedicated 100 Mbps Internet connection to ease the bandwidth problems we were having. At the time, Ed Burnette and many others could only guess as to how much it costs to enable this, and this year we're adding 240 Mbps of bandwidth to our regular connection. So how much does it cost?

Two words: a lot. Each 100 Mbps connection requires one firewall and one switch port, and each connection is doubled up through two routers for failover (don't underestimate the monthly costs of using expensive Cisco router ports). Then there's bandwidth. Although we get volume discounts on bandwidth usage (the more we use, the cheaper each Mbps is) it adds up really fast when you're sustaining 150+ Mbps.

In other words, Callisto won't be free. But it will be free for you and I, and thanks goes to the Eclipse Foundation members. This is a good example of how their membership dues are used to help support the projects.

More bandwidth for Callisto

Callisto is approaching fast, and we received our new 100 Mbps network connections last week. To survive the Big Event, we'll be removing the 60 Mbps cap on our existing 100 Mbps and use these two 100 Mbps connections dedicated to downloads. The net sum of usable bandwidth should be about 280 Mbps, routed through three different Cisco 7200 routers for maximum performance/failover.

On the server side, the cluster will handle the usual web pages, CVS, e-mail and RSYNC using its beefed up connection, while downloads will be redirected to the two dedicated 100 Mbps connections, served by an IBM eSeries P5 550 server with 16 GB of RAM. We managed to get 3,500 simultaneous web connections to this IBM server alone, with plenty of room to spare. All tolled, we successfully tested our infrastructure to support 11,000 concurrent web connections, and if need be, we can balloon to 15,000 (our daily average usually floats between 400 and 800.)

All this new bandwidth isn't available to us -- yet. We still need to do some testing first. Sometime next week we'll host a "Mirror Site Appreciation Day" where, for the duration of a day, will attempt to host *all* its download traffic. The idea here is to load our systems with massive amounts of traffic to test our server performance and routing setup. Stay tuned...

Thursday, June 08, 2006

Back to Exquisitus

Back in November, I blogged about how we removed the password protection from the web-based News Portal at I wanted to make it easier for everyone to read and participate on the newsgroups. Well, the password is back. Actually, I'm surprised we managed to allow wide-open, passwordless, authenticationless and image-verificationless write access to our newsgroups for so long without getting bombarded with SPAM.

The eclipse.dsdp.mtj and eclipse.newcomers groups got heavily spammed this week, so we had no real choice but to re-implement it. For those of you using the Portal because your firewall doesn't allow you to use the NNTP server directly, have you tried connecting to directly on port 80?

Tuesday, June 06, 2006

Eclipse CVS snapshots now available

We've had a couple of requests from folks who want to download entire CVS snapshots for analysis. We've made them available here:

These snapshots are generated weekly, and the Eclipse project alone will yield you a 3 GB+ download, and require about 6 GB of disk space once uncompressed. This archive contains every version, revision, tag, branch and comment from every commit since the beginning of Eclipse time.

Of course, the uncompressed result is a CVS structure of RCS-encoded files (file.ext,v) that is usable by a cvs server, or the CVS command-line utility. If you need to analyze the Eclipse CVS repositories, this is the best way to go about it (as opposed to banging on our ViewCVS interface! Don't do it! Just don't!)

We'll also make the raw Bugzilla databases available at some point, as folks frequently request them (and also hammer on Bugzilla one bug at a time! Don't do it! Just don't). Our concern here is to remove e-mail addresses and user passwords from the tables before zipping them up.