Wednesday, April 30, 2008

Bugzilla: mod_perl + mod_deflate = giggles

Bugzilla 3.0 supports huge performance gains via mod_perl, but I haven't enabled it on bugs.eclipse.org because the added RAM requirements would kill our PHP-tuned cluster. mod_deflate, which compresses HTML output on-the-fly, is not enabled either, as it adds too much CPU load to our already busy cluster. However, as we gear up to deploy newly donated AMD hardware, we can afford to use both mod_perl and mod_deflate for some serious performance gains.

Of course, before enabling such features on our live site, some benchmarks are in order.

A simple perl script is used to fetch random bugs. This is not typical Bugzilla usage, as one would read and write (and perform searches), but Bugzilla page fetches far outnumber writes, and it does allow a uniform way of measuring raw Bugzilla throughput.

#!/usr/bin/perl

while (1==1) {
$bug_id = int(rand() * 150000);

@lines = `wget --no-check-certificate --header='Accept-Encoding: gzip,deflate' \
https://bugs.eclipse.org/bugs/show_bug.cgi?id=$bug_id 2>&1`;

}

For each test I launch 20 of these scripts on 2 different computers. 'bugs.eclipse.org' refers to the new AMD-based Bugzilla server running in my office. SSL is enabled.

TEST 1: typical CGI, no mod_perl, no mod_deflate.
Pages/minute: 375
RAM used: 1.5G
Load average: 21.00
Typical latency for 1 page: 1 to 2 seconds
Network transit: 8.4 MB/minute (typical page size 23 KB)

This test reflects our current setup. Simply fetching a bug page yields a noticeable latency before any network transit occurs. Forking a Perl process and compiling show_bug.cgi are expensive.

TEST 2: mod_perl, no mod_deflate.
Pages/minute: 916
RAM used: 1.4G
Load average: 15.33
Typical latency for 1 page: 0 to 1 second
Network transit: 20.5 MB/minute (typical page size 23 KB)

Wow. A net gain of 541 pages/minute, with page latency near zero. With no perl to fork and page to compile, the Apache gets bits on the wire almost instantaneously.

TEST 3: mod_perl and mod_deflate with compression level '1' (least).
Pages/minute: 930
RAM used: 1.7G
Load average: 17.00
Typical latency for 1 page: 0 to 1 second
Network transit: 6.1 MB/minute (typical page size 6.7 KB)

Another Wow. Page throughput is up only slightly, but Network transit is cut by 70%. mod_deflate will be most helpful on pages with many comments, and search result pages.

TEST 4: mod_perl and mod_deflate with compression level '9' (most).
Pages/minute: 910
RAM used: 1.7G
Load average: 17.00
Typical latency for 1 page: 0 to 1 second
Network transit: 5.24 MB/minute (typical page size 5.9 KB)

Page throughput is down compared to minimal compression, and page size is decreased only marginally. I suspect I'm getting all I can out of this box (and/or, my two 'client' computers are hosed), but the results are conclusive: mod_perl makes Bugzilla giddy-up, and mod_deflate with compression set to 1 gives the most bang-for-your-buck on the network.

Expect to see noticeable performance gains for Bugzilla sometime in May.

0 Comments:

Post a Comment

<< Home