Thoughts on G-WAN
While looking for a quick webserver to serve static content I came across a relatively unknown solution: G-WAN (a.k.a. GWAN, a.k.a. Global-WAN).
The extensive statistics and graphs on this blog show significant improvements over more well-known solutions such as Nginx and Varnish. In hindsight, I should have trusted my gut instinct on these benchmarks, but my trusting nature took me for a short ride…
Without hestitation, I launched my “sandbox” Amazon EC2 instance with Ubuntu 12.04 x64 and jumped straight into SSH to install G-WAN. Installation was a breeze. Configuring G-WAN is a bit different than usual though, but 15 minutes of reading documentation later, I had it up and running on non-standard port 8888 (G-WAN uses port 80 by default). Getting it running as a daemon took a bit longer, but in the end it was all running smoothly.
Then came the benchmarking. My initial tests involved simply reloading a small (~57KiB) image a bunch of time while watching `top` running in the shell. It didn’t take long for the problems to show up. Initially G-WAN didn’t give a peep. Using a single process with less than 1% of memory (Apache2 typically uses 1.5%-2% for each process and Nginx used ~0.5% per process); much better. CPU% remained at 0%. For a simple image request, Apache tends to show a short 1% peak whereas Nginx keeps a cool 0% as well.
Standardized testing methodologies
Trouble starts however, when I pressed the Shift+F5 (no client-side cache, please) a few more times. Memory remained low, but CPU%… wow. “Wow” meaning “jumping to 100% for a few seconds”. Quite impressive, though not of the right kind. For comparison, Apache2 just kept giving those small 1% peaks and Nginx, trusty old Nginx, still didn’t register any CPU%.
It wasn’t just CPU% though, the latency as measures by the Chrome browser’s excellent developer tools showed wildly varying numbers for G-WAN responses. Again, initially G-WAN performed an admirable ~45ms for that picture, but when hitting the F5 key some 50 times, I occasionally got responses of over 2 seconds. That’s two whole frickin’ seconds for a 57KiB image file! Shame! Apache2 handles the same image at right about 160ms each and every time. Nginx, as a reversal of G-WAN’s behaviour, starts out at ~150ms but quickly drops to ~90ms. Again, neither of these show any weird behaviour as far as CPU% is concerned.
As I said before the fold, in hindsight I should have known. The benchmarks on that blog (which are the same shown in the documentation and frequently mentioned on their forum) are somewhat… “unworldly”. By now you’re probably thinking that a smallish image isn’t a very reliable real-world benchmark. Compared to the 100-byte text file with only ‘x’ characters that is the “official” benchmark, it is practically science! Shame! I noticed this before I started testing myself, so I guess that shame is on me.
For what it’s worth, I’ve also tried the “100 ‘x'” file, using the same hightly acclaimed “Hit Shift+F5 like a spastic” method. CPU% for G-WAN remained at a comfortable zero this time, but again, latency slowly increased. In this test, at it’s best G-WAN performed equal to Nginx at ~45ms and at it’s worst equal to Apache2 at ~75ms. Apache2 peaked at 1% CPU, Nginx again remained at zero. Also notable is that both Apache2 and Nginx performed consistantly and reliably.
It smells funny
I’m keeping an open mind. The only thing I changed in G-WAN’s configuration was the port number, and this may be responsible for it’s lacklustre performance. Also, the G-WAN site clearly states it was tested on Debian and CentOS. Ubuntu derives from Debian, but they are not identical. Another thing to note is that G-WAN is also capable of dynamic websites. Then again, so do Nginx and Apache2. All this does not bode well for G-WAN. Perhaps somebody should do some quality benchmarking. As for me, the G-WAN benchmarks take effort to point out how bad Varnish performs, so that’s what I’ll be testing next.
Update; please read the comments and addendums for more.
Okay, installing Varnish was pretty easy. Although they claim to support LTS versions of Ubuntu, their site lists only the Lucid (Ubuntu 10.04) repository. Thankfully it was just a matter of search and replace. Simply add “deb http://repo.varnish-cache.org/ubuntu/ precise varnish-3.0” to /etc/apt/sources.list and just sudo apt-get update, sudo apt-get install varnish. Some minor configuration later and it’s running on port 888 with Apache2 as a backend (Varnish is different in that it can’t serve files on it’s own; it needs a backend). The 57KiB files has consistent latency in Chrome of about ~60ms (faster than any) with small 0.3% CPU peaks and 0.5% of memory combined over two processes (oddly enough, as I expected it to take the full 256MiB I configured). The small ‘x’-file runs past at ~35ms with same CPU%/memory%.
Running with Nginx as backend (itself with Apache2 as backend) makes no difference whatsoever. That means there is no point in having Nginx sitting between Varnish and Apache2. Alternatively, drop Apache2 and use only Nginx, ideal for pure static content. It should also be noted that Varnish does not handle PHP files by default, some minor configuration lets it cache only specific file extensions (or vice versa). This lets you use Varnish for a dynamic site as well. With Varnish configured like this, it’d make a great load-balancer/reverse-proxy when used on a separate server (due to memory requirements).
Addendum; Partial G-wan redemption.
As commenter Bert so aggressivly pointed out, I made a mistake regarding the “100-x” file. I again compare G-wan and V+N (ruining a perfectly good evening) with both the “official” ab.c test file and a number of other files. You can read my full findings in the comments below.
The summary conclussion is that, for some files, G-wan does indeed perform quite well. Not quite as well as Varnish+Nginx (although it does have the benefit of a lower memory profile than Varnish), but a lot closer than in previous tests. For some other files, G-wan still uses an extraordinarily large amount of CPU to the point where it dramatically impacts it’s own performance as well as the entire server. The bad news is that I can see no pattern in which files are delivered fast and which aren’t.
Sadly this still makes G-wan a bad choice for a production server, though it does show promise for the future.
If it can perform (nearly) on par with Varnish+Nginx, it may have a bright future ahead of it.