Wednesday, August 26, 2009

Bugzilla 3.4 coming soon

This Saturday I'll be upgrading our "ancient" Bugzilla 3.0 to the latest Bugzilla 3.4.  Lots of new features will be available to everyone, and many of those are features our committers have requested.  Just check out this dependency tree!

Here's a highlight of a few new features:

A new Guided Bug Entry wizard to help new users file bugs in the correct locations and reduce triage/duplication.





A much improved UI:



Uncluttered attachments by default:



.. and many more:

  • Better database storage for improved performance

  • Better email handling for improved performance

  • UTF-8 for better internationalization


I've set up a Bugzilla 3.4 sandbox for your testing enjoyment, and you can read about the new features in Bugzilla 3.2 and 3.4.

Friday, August 07, 2009

Squeezing more performance out of your Apache web server

You can learn a lot about your web server by strace'ing it.  Look at this, with my commentary in red:
www-vm1:~ # strace -p 5827
Process 5827 attached - interrupt to quit
poll(
^^Your shell will sit here until this process
receives a request

[{fd=14, events=POLLIN, revents=POLLIN}], 1, 15000) = 1
read(14, "GET /modeling/images/dl-more.gif"..., 8000) = 677
^^ A request for a plain old GIF file

gettimeofday({1249652331, 82192}, NULL) = 0
stat("/home/local/data/httpd/www.eclipse.org/html/modeling/images/dl-more.gif", {st_mode=S_IFREG|0654, st_size=111, ...}) = 0
^^ This is the absolute path to the file on disk.

lstat("/home", {st_mode=S_IFDIR|0755, st_size=192, ...}) = 0
lstat("/home/local", {st_mode=S_IFDIR|0755, st_size=120, ...}) = 0
lstat("/home/local/data", {st_mode=S_IFDIR|0755, st_size=72, ...}) = 0
lstat("/home/local/data/httpd", {st_mode=S_IFDIR|0755, st_size=112, ...}) = 0
lstat("/home/local/data/httpd/www.eclipse.org", {st_mode=S_IFDIR|0755, st_size=72, ...}) = 0
lstat("/home/local/data/httpd/www.eclipse.org/html", {st_mode=S_IFDIR|0750, st_size=4808, ...}) = 0
^^ Apache crawls the entire directory structure leading
up to the file

open("/home/local/data/httpd/www.eclipse.org/html/.htaccess", O_RDONLY) = -1 ENOENT (No such file or directory)
lstat("/home/local/data/httpd/www.eclipse.org/html/modeling", {st_mode=S_IFDIR|0750, st_size=768, ...}) = 0
open("/home/local/data/httpd/www.eclipse.org/html/modeling/.htaccess", O_RDONLY) = -1 ENOENT (No such file or directory)
lstat("/home/local/data/httpd/www.eclipse.org/html/modeling/images", {st_mode=S_IFDIR|0755, st_size=2056, ...}) = 0
open("/home/local/data/httpd/www.eclipse.org/html/modeling/images/.htaccess", O_RDONLY) = -1 ENOENT (No such file or directory)
^^ Unless you have AllowOverride None, Apache will
look for .htaccess files in each subdirectory of
the DocumentRoot


lstat("/home/local/data/httpd/www.eclipse.org/html/modeling/images/dl-more.gif", {st_mode=S_IFREG|0654, st_size=111, ...}) = 0
open("/home/local/data/httpd/www.eclipse.org/html/modeling/images/dl-more.gif", O_RDONLY) = 15
close(15) = 0
read(14, 0x555555a356b8, 8000) = -1 EAGAIN (Resource temporarily unavailable)
writev(14, [{"HTTP/1.1 304 Not Modified\r\nDate:"..., 170}], 1) = 170
^^ All of that hard work to simply respond "Use your cached copy"
to the client!

write(10, "192.168.0.1 - - [07/Aug/2009:"..., 256) = 256
^^ IP address changed to protect the innocent.

poll(
^^ That request is complete, wait for another

All of this happens lightning fast. But do consider: Had the request been for a PHP file, all the above would have been repeated for each nested require(), require_once() and include() file.  So the moral of this story is:

1. Don't nest your DocumentRoot too deeply.  We could trim /home/local/data/httpd/www.eclipse.org/html to save lots of CPU and disk cycles

2. Keep AllowOverride None to avoid accessing .htaccess files everywhere, unless you really need them.  We're investigating this seriously for www.eclipse.org.

3. Keep your web-visible directory structure short, too.  http://www.eclipse.org/some/directory/structure/that/is/really/deep/and/nested/page.php will obviously generate lots of file stats (especially with AllowOverride)

4. If you use PHP files, only include what you need, otherwise you're stat'ing (and reading, and possibly compiling) PHP files for nothing.

The more you reduce disk and CPU cycles for one request, the more your web server will scale.

Thursday, August 06, 2009

The endless list of Forums

First off: I'd like to thank Eric Rizzo for helping out with the new Forums.  As if he doesn't help out enough by answering questions and providing insight, now he's organizing the Forums, giving them descriptive names and pinging projects to get updated descriptions.

Now, on to the problem.  There are about 115 forums for you to browse through, and as more and more projects get created, the page will only gather more clutter. Do you have any thoughts as to how all these forums should be organized?  If so, we could really use your help in bug 284281.

Tuesday, August 04, 2009

99.999% uptime?

Even with gazillions of dollars in the bank, it seems like the Five Nines eludes even Google.