Tuesday, October 30, 2012

EclipseCon Europe was amazing

Last week I was at EclipseCon Europe, and now that the dust has finally settled, I thought I'd share some highlights of the event.

Technical Track
I'm a server guy, not a developer, so I can't comment much on the track.  However, as I tend to talk to just about anyone while I'm there, the feedback I got was very positive.  I heard the talks were great, and most people seem to appreciate being introduced to different technologies that they can investigate for usage on their Eclipse products or work environment.

For a server guy like myself, there wasn't as much compelling content as there was in North America in March with the ALM track.  But isn't that why there are two conferences to attend?

Most of the attendees I have spoken to got real value in connecting with the actual developers and integrators of Eclipse-based technologies.  Something about "getting knowledge from the source", if that was never a pun  :)

As webmaster@eclipse.org
It's always great to sit down with committers and community members to discuss issues around the Eclipse.org infra, be it Sonar with Mickael Istria, Gyrex with Gunnar Wagankn Wagenkenc -- Gunnar, Community with Lars Vogel (I'm not old!), Hudson with Ed Willink, or just lending a helping hand to those people who don't normally get webmaster support in their timezone  :)

Wifi seems to be a recurring issue.  As someone pointed out, the Forum's wifi is likely more than adequate for 362 days out of the year.  For the remaining three, our atypical population seems to destroy even the best of intentions.  I did work closely with the on-site staff, and I'll be recommending some improvements for next year.  We'll see how it pans out.

One thing is for sure -- 1992 called and wants those 10Mbps hubs from the Power-Up Lounge back  :-)

The amount of first-time EclipseCon attendees is always staggering.  However, I was even more impressed to meet many EclipseCon newcomers at the Nestor bar after the day's schedule, despite their being immersed in a sea of unknown faces.  It was great to meet everyone.

Okay, let's face it: EclipseCon is always a fun time.  This year, we had the Nestor bar with its usual cast of characters; the band, who rocked us until they were out of songs to play; the Plug-Ins trio ... you simply had to be there; the circus, who dazzled us with their flame- and object-throwing acts; the contests, none of which a civilian like myself could understand...
.. and then there was the Scout crew with their cool Legos!  LEGOS!

Yet another amazing conference in the history books.  Props to Ralph, Anne and their respective crews for putting on such a great event!

Tuesday, October 02, 2012

Optimizing www.eclipse.org over the years

Since I took over the controls of the eclipse.org servers in October 2004 (hey, that was eight years ago yesterday!) our main website, www.eclipse.org, was optimized along the way to support the tremendous and steady growth in traffic.  I thought I'd share some of those optimizations:

1. Before the Eclipse Foundation, www.eclipse.org was one single server.  There was no scaling it, and there was no fault tolerance.

2. With new hardware, my first iteration of the "improved" www.eclipse.org consisted of four servers: one NFS/MySQL backend and three front-end nodes running Apache.  These front-end nodes also served everything else at eclipse.org, including CVS, Bugzilla, email and all our other websites.  A Cisco load balancer was used to direct traffic, and site files were served directly from NFS.

3. Years later, with some new hardware came a second iteration: segregation of the services.  Having Bugzilla, dev, wiki, cvs and www on the same set of servers was not efficient, and thanks to virtualization, www.eclipse.org was  hosted on three virtual servers.  Data was still served from NFS

4. Since NFS was introducing some I/O latency, we began publishing the website files to the local disks of the servers.

5. We enabled mod_gzip and mod_deflate, and added aggressive cache headers to reduce payload and reduce round trips.

6. With new hardware again, NFS and MySQL were separated on the backend-side.  This gave MySQL much needed breathing room.

7. We shortened the local directory path from a long /path/to/the/www.eclipse.org/website/html to a much shorter /site/.  This reduces stat() calls on the filesystem and reduces I/O overhead.

8. At the same time as 7., we also set Apache's AllowOverride to None so that a .htaccess file is not examined in each and every directory leading up to the desired file, cutting I/O calls drastically.

What's next?  SPDY?  Varnish? More hardware?  Magic?   With any luck, yes to all!