Oct, 01, 2011
With my upcoming presentation on SubSonic, I wanted to run benchmarks on some popular ORMs. I found ORM Battle which did exactly that and I dove in.
ORM Battle required a little setup. After downloading the project the first thing I had to do was run the sql scripts to generate the databases that the tests are run on. I'm using Sql Server 2008 Express to conduct the tests and they ran problem free and installed 3 databases including Northwind, DO40-Tests, and PerformanceTest. Next I had to configure the connection strings, which took a little more trial and error. Due to the nature of the different ORMs the connection strings are in a few different forms.
- The most obvious was the .config file ConnectionString settings. A find and replace on "data source=.;" to "data source=.\SQLEXPRESS;" handled that.
- Telerik's OpenAccess has an xml configuration file. Find and replace "<servername>(LOCAL)</servername>" to "<servername>(LOCAL)\SQLEXPRESS</servername>".
- SqlClient tests System.Data.SqlClient, and uses a hard coded connection string for the initial SqlConnection. Find and replace "Data Source=localhost" to "Data Source=.\SQLEXPRESS". There's only 1 instance of this.
The project still wouldn't build because I didn't have a license for DataObjects.NET. I'm not sure if I used a trial before or what, but I ended up just excluding the project and removing all references from it to get the project to build and not run the tests on DataObjects.NET. The ORM Battle tool was written by the developers of DataObjects.NET (http://www.x-tensive.com/), so that's kind of a bummer I couldn't include theirs.
Running ORM Battle Tests
A console app runs several tests on the ORMS. The performance tests run about 25 tests on each ORM several times with different amounts of items being queried. These quantites include 50, 100, 500, 1000, 5000, 10000, and 30000 by default. It will output the results with scores for each ORM after each performance test. The below video is about an hour long to run all the tests.