The code says php-raw refers to using PDO (which all project should be using) vs using an ORM or ActiveRecord.

I am completely stunned by the performance cost of using ORM/AR, and will be using this to shame our team lead into giving it up and going for raw queries.

Don't be so hasty. As others pointed out, caching often plays a factor on larger projects. A degree of portability provided by ORMs is nice (test mode hits SQLite, prod is pg or mysql, etc). Also, the readability of ORM-oriented code shouldn't be overlooked, especially as new people come in to the team. Two or three expressive lines in ORM vs 15-20 of nested SQL hitting weird tables names and aliased column names and such isn't easy to decipher the intent of (especially when there's a problem).

I typically use ORM for about 95% of a project, falling back to a few explicitly native SQL calls when performance can be shown to be a bottleneck in those locations.

Some of the ORM cost can be mitigated by using caching. In most cases this is essential in a production deployment.

Agreed. However, for our database tests we expressly wanted to stress-test the ORM and, where configurable, disabled caching.

We plan a subsequent test, time permitting, that enables caching.

Important to note that OP doesn't mention using APC with this, something that any production code would be using in a real world case. This would have an impact on the numbers he uses and on the diff we are seeing.

Gotcha. Thanks!

The best way to understand it is to look at the source:

php - https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

php-raw - https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

For "php", they used an ORM. For php-raw they used the stdlib (pdo).

Hi Chasing. We put a note about that suffix in the "Environment Details" section. The "raw" suffix means there is no ORM used. If there is no "raw" suffix, you can assume some type of ORM is used.

FYI on the PHP-DB-Raw approach: https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

By default, PDO does not do true prepare()s, it just does string interpolation. You need to pass this parameter with the options:

    PDO::ATTR_EMULATE_PREPARES => false

And then you'll actually be using MySQL prepared statements. You'll see a noticeable performance improvement for large amounts of queries.

Thank you! We'll get this modified and re-tested.