12 Feb 2012

Benchmark of some popular Web frameworks

Category: Asymptote


I order to choose the web framework that meets best my need for a further big project (web application for a real estate company), I have decided to do some benchmarks because good performance is one of the constraints with agility development.

The benchmarks are made with Apache HTTP server benchmarking tool launching a minimalist html w3C compliant web page which presents the list (in tags ul, li) of ten user names stocked in a Postgresql data base.
I arbitrarily chose the web framework Zend Framework (ZF 1.11.11), Zend Framework 2 (which is currently in beta stage), Symfony 2, PHP No-Framework (pure php), Pyramid and Express. I would like to test PlayFramework but I do not have time :(

For the PHP web frameworks, all the benchmarks are made with APC enabled the option apc.stat=0 (no disk access)

CPUXeon 5500/Core i7
DiskSSD, buffered disk reads : 288.51 MB/sec
OSGnu/Linux (Debian wheezy/sid)

Basic computer specifications for the benchmarks

Here the table of ORM and template system used with the different frameworks :

Framework ORM Template system Serveur
PHP No-FrameworkNone (PDO)PHPApache (libapache2-mod-php5)
ZF1 & ZF2Zend_DB_TablePHPApache (libapache2-mod-php5)
SymfonyDoctrineTwigApache (libapache2-mod-php5)
PyramidSQLAlchemyChameleonApache mod_wsgi

First results

Framework Time taken for tests Requests per second (mean) Time per request (mean, across all concurrent requests)
Express 0.066 s 2342.20 0.427 ms
Pyramid 0.775 s 199.95 5.001 ms
PHP No-Framework 0.882 s 175.76 5.690 ms
ZF1 1.197 s 129.53 7.720 ms
Symfony 1.218 s 127.28 7.857 ms
ZF2 (*beta stage*) 11.091 s 13.98 11091.008 ms

155 requests completed with concurrency Level of 155


Here are some charts that shows the evolution of response time based on the number of simultaneous requests and the level of competition.

Express framework benchmark php benchmark
Pyramid python benchmark Symfony 2 benchmark
ZF1 framework ZF2 benchmark

Concurrency Level of 10

Express framework benchmark php benchmark
Pyramid web benchmark Symfony benchmark 155
ZF1 benchmark 155 ZF2 benchmark

Concurrency Level of 155

I leave the reader in interpreting the results. Personally, my preference will go to Pyramid, but I'm not completely decided yet since I am more at ease in php.

All the results and the charts can be downloaded HERE.