Some suggestions on spec performance

spec_performance.txt

Referencing http://twitter.com/#!/refinerycms/status/73536619038261248

Depends, do you want to speed up single test runs/running one spec file or speeding up the whole suite?


Speeding up a single test == shorten init time
-------------------------

Reduce the number of requires: 
When running on 1.9.2, require is an extremely costly operation. @xshay is currently working on this, should be better in 1.9.3. 
Usually a lot of gems in Gemfile do not need to be required. I for example have things like passenger or yard in my Gemfile 
to ensure they are there. Requiring them on each run wastes a lot of resources, so : 

gem 'passenger', :require => false


Don't use "rake spec", but directly run "rspec spec another_dir": 
The first think rake does, is initializing the whole environment => a lot of requires. The task "rake spec" just calls the command line 
with rspec and some params. So you first init the whole project with rake, which just calls a console command, which again requires 
the whole damn thing. In a large project of mine on my old laptop the init time is 20 seconds. So rake spec takes 40! seconds to start tests. 

Maybe use spork:
Spork preinitializes a rails env so the tests don't have to. My personal experience is, I never got it fully working - 
Problems with factory_girl I solved after a lot of work, but now request specs cause trouble - which you don't use in refinery so ... 
As usual your mileage may vary. 


Speeding up the whole suite
---------------------------

Buy a faster machine: Sounds trivial, but to show you some numbers, on my old laptop (core 2) a medium testsuite ran 520 secs, 
on my shiny new PC (yes, not mac ;-) ) with an I7-2600 including all the optimizations I'll explain afterwards, the suite 
is down to 80 secs. The time you are waiting usually costs a lot more than a faster machine. Throw in an SSD, if you can afford. 

After buying the new machine with many cores, make sure all are used while testing: The Gem https://github.com/grosser/parallel_tests 
does wonders here. It's not exactly double the cores => halve the time, but you will see a very big difference. 

If on Linux: http://blog.smartlogicsolutions.com/2009/06/04/mount-options-to-improve-ext4-file-system-performance/
The blog post explains it quite well and especially on a traditional HDD the difference is extreme. 

Tune your local database for tests: Databases go a long way to ensure data safety and integrity and a lot of these settings cost 
performance. Things like writing a committed transaction to disk at once or even storing running transactions on disk. 
On production, this is very important, but on testing no one cares for lost fixture data. So if we deactivate that 
(I repeat: ONLY ON DEV, so don't flame me), the usual rails test cycle "load fixtures -> test -> rollback" gets faster. 
On cucumber or request specs, where transactions won't work, you even have "load fixtures -> commit -> test -> truncate", 
where you will see an even greater difference. Increasing the memory and cache settings of your database helps too. 
My personal settings for mysql (for 8GB ram) are: 

[mysqld]
query_cache_limit = 20M
query_cache_size = 512M

innodb_buffer_pool_size = 4G
innodb_log_buffer_size = 4M
innodb_thread_concurrency = 16
innodb_flush_log_at_trx_commit = 2
innodb_log_file_size = 256M

The important one regarding transactions is innodb_flush_log_at_trx_commit. I haven't done a lot of research on the DB area, 
so there might be better settings. But they are definitely a lot faster than the default. 

If you use seeds in your tests, enclose them in a transaction: Sometimes it is useful to use seed data, especially while doing integration 
tests (cucumber). Example: for a refinery project, I make sure via seed, that there always is an admin user, as refinery redirects all 
routes to create_user otherwise. And I add the standard cms pages too, to ensure a working page and navigation. On integration tests, 
you always have to truncate the DB and reload the seeds, which is very expensive. So at least enclose them in a transaction. 

So anyone, other ideas?

Cheers
Marc