Cullen's Musings

Full disclosure: I am involved with the phpBB Bulletin Board Project as a moderator and junior developer.  However, I believe this test to be perfectly fair and accurate to all parties involved, and am more than willing to respond to any questions regarding its validity.

LETS GET READY TO RUMBLE!!!

~Michael Buffer

Why?

Regarding the subject of web based bulletin board systems, you will find plenty of fan boys from all sides.  One thing I’ve noticed however is in the constant arguments, one of the most important aspects of running a forum is consistently overlooked: performance.

When it is mentioned, it is also incredibly anecdotal: “I was using software X and when I switched to software Y it got a lot faster”.

The lack of true numbers behind these claims is astonishing to me: on the web, performance is king; if visitors believe your pages are taking a long time to load (or they actually are taking a long time to load), you could lose out on a significant chunk of your visitors.  This lack of numbers drove me to do benchmarking of my own to actually determine which software package performs best.

The Machine

This is my desktop machine running Ubuntu Server 9.10.  It’s not exactly web server hardware, but it gets the job done.

Ubuntu 9.10
2.6GHz Quad Core Athlon CPU
4GB RAM
10k disk drive
Lighttpd with php 5.2 fastcgi backend
Basic Linksys router

Lighttpd was set up incredibly simply: No virtual hosts, just different folders for each package.

The Boards

Early on in my testing, it was pointed out that a new board is one of the worst setups to test.  The database is empty and places very little stress on the server and software.  I decided to test the equivalent of a medium sized board, with 750 users, 10 forums, 1000 topics, and 10000 posts.  Knowing my way around the phpBB code base, I scripted up a forum-filler to do this for me (rather than asking 750 Facebook friends).

With phpBB being one of the most popular software packages available, luckily all of the packages I tested had converters from phpBB, so each could be compared exactly to the load of the others.

The Tests

Tests were done against the three most accessed pages on a bulletin board: the board index, a forum view, and viewing a topic.  I randomly selected a forum and topic from phpBB, then tested the corresponding forum/topic on all packages using the ab benchmarking tool, making 20 requests per second in batches of 2000.

After each test, the session tables of each package were cleared to reduce interference and to make everything fair.

The Results

Graph of benchmarking results.  Higher numbers are better.

It appears phpBB performed the best overall, but I’ll try and stay modest and not gloat.  One of the key focuses of phpBB is to keep the core package lightweight and free from extraneous features, and it appears to be successful in doing so.

Next?

This test obviously is incredibly limited, as it is only testing 5 software packages.  I did not benchmark more is I was unable to find clear conversion instructions for many of the packages I investigated.  As much fun as it would be to dive into the codebase of several new applications, I have way too much on my plate to do so at the moment.

If any other software packages would like me to benchmark them, I would be more than willing provided they provide clear conversion instructions from phpBB3.

Additionally, I only tested a few specific pages.  While these pages are the ones most likely to be bombarded by the slashdot effect (or digg, or reddit, or <insert social site here>), there are several other pages that are computationally expensive, such as search.  The problem is these pages are much more difficult to test, as they sometimes require authentication, which ab cannot simulate.  However, if people would like me to try to test such pages, I would be more than willing to do so.

What are your thoughts?

Share this post via: