When it comes to page caching and WordPress, there are a number of options out there. Nginx caching, Varnish, Batcache, W3TC Page Caching, WP Super cache to name some. Without going into too much depth the two options I am testing are Batcache and Varnish.
Batcache
Batcache does static page caching via the WordPress Object Cache – this means you want to use a Persistant Object Caching plugin, ideally using Memcached so you can spread page caching across multiple servers. It’s of course possible to use the slightly faster object caching via APC, but doing that loses you multiple servers and the overhead of halving your cache hitrate would outweigh the performance increase from using APC.
Batcache is used on the WordPress.com infrastructure and is written in PHP so there is virtually no learning curve for a PHP developer to building complicated exclude rules etc.
Varnish
Varnish is an incredibly popular reverse proxy cache and is arguable the fastest of it’s kind. If you are running multiple web servers, generally speaking you will have Varnish sitting on a single server infront of the web servers. Having Varnish on each webserver will mean your overall hitrate reducing by your server count.
Varnish and Batcache are not exactly solving the same problem, but they should not be used together to cache page requests – one must make a choice. Here is an overview of the advantages / disadvantages when used in a WordPress context.
Varnish
- Fast! Like, really fast.
- Varnish will cache assets.
- Another tool to learn – VCL is another thing to learn on the stack, excluding cache for logged in users etc.
- As Varnish is on 1 load balancing server, you have all your caching is “in one basket”, restarting this server can overload your web heads.
Batcache
- Easy to learn as is just a PHP file.
- Easier to set up complex exclusion rules etc.
- Multiserver via Memcached
- Uses PHP, PHP processes are not cheap, so performance is not ideal.
- You will need to use something else like nginx caching for assets.
If you have read this far, the above points probably aren’t new to you — we know it’s a performance vs ease of use trade off. But saying “well, it’s easier to use, so let’s pick that one” is not an informed decision if you don’t know the magnitude of difference in performance. So, I thought I would do some basic benchmarking to see how different they do perform.
Setup Environment
I had 2 Amazon Ec2 Medium instances, both running Nginx, PHP-FPM and APC. They both used the same database server. A clean install of WordPress with no plugins. However, whatever is on the WordPress install doesn’t matter – we are testing cache hits. the Varnish server just had Varnish set up. The Batcache server have Memcached installed and WordPress is running the Memcache Object Cache plugin and Batcache.
To test sending requests I used Siege, a light weight HTTP benchmarking utility. I could have used Apache Bench, however this tool is quite old and not too well suited to concurrency benchmarking. I tested by concurrent request count only, running each test for 60 seconds. Primarily I was measuring the load of the 2 servers. Siege was running an a third EC2 instance.
Results
Lower is better. Both Varnish and Batcache were pretty comparable at 100 concurrent, load will probably keep increasing as the server struggles to keep on top of the requests coming in.
As expected, response time is going inline with server load — not much more to say there.
It’s worth noting I didn’t tweak any settings for Varnish or PHP-FPM/Nginx, arguably there would be more things one could do to squeeze more out of Batcache compared to Varnish.
Conclusion
In terms of low concurrency, Varnish and Batcache had pretty much the same response time which did surprise me somewhat, I had expected the overhead of PHP-FPM to be much more than it was, which is encouraging. It’s also worth mentioning that 250 concurrent connections is quite a lot when you have a response time of < 100ms, so the right hand side of the above charts is not going to happen for everyone.
For now, I think we will still be using Varnish — I have a pretty good VCL set up for our WordPress projects so the learning curve is not an issue. Also, as we are running lots of sites on EC2, CPU efficiency is pretty important for us.