Test your JavaScript!

Tddium now supports tests for JavaScript-rich web applications with a number of tools:

Cucumber or RSpec tests running in our environment automatically have access to a private Selenium Webdriver and/or Webkit servers with Firefox and Chrome browsers.  Tddium will accelerate your test suite even if your tests normally use Webkit, but they depend on Selenium for a few special cases.

Broken Migrations and Missing schema.rb

ActiveRecord Migrations are slick.  They allow easy access to a powerful database with minimal knowledge of SQL, and they promise an easy way to reconstruct a DB schema.  Unfortunately, they’re fragile in the face of large teams and code that has changed significantly.

Migrations and schema.rb

There are two ways migrations can reconstruct a schema into a fresh DB:

  1. Restoring from db/schema.rb (or db/development_structure.sql)
  2. Replaying the full migration chain against a properly prepared empty template database

In small projects, or projects without many concurrent branches of development, schema.rb is authoritative. Branching, however, leads to guaranteed conflicts in schema.rb (on the schema_version). Therefore, schema.rb is often omitted from source-control.

In the absence of schema.rb, the complete migration chain is the only authoritative representation of the schema.

Breaking the Chain

However, the complete chain can be broken under a number of circumstances:

  • Concurrent development: Developers in different branches can check in migrations that semantically conflict, even though they don’t textually conflict (e.g., both rename column A, one to B and the other to C).
  • Stale Migrations: Migrations from the past often reference old models or other code that’s been deleted from the application, especially if they migrate data.  The “right way” is to make migrations self-contained, but it’s not obvious that you need to do this until a non-self-contained migration has already been applied.  Regardless, it’s a little tricky to test data migrations, so failures go unnoticed. See my earlier post about data migrations for an example.   The work involved in fixing old migrations, while noble, is hard to justify in the face of crunched schedules — it is by definition hard to relate to business value.

Large, long-lived projects inevitably encounter one or both of these situations. So, they fall back to using a live database instance as an authoritative schema — hopefully a protected DB whose only purpose is to remember the schema, but often this ends up being a dev, production or staging server.
Read the rest of this entry »

Testing Rails data migrations using RSpec

I’d like to share an example (Gist) of testing a staged ActiveRecord data migration using RSpec and Factory Girl.

The example here splits Author information out of an existing Post model.

The trouble with a data migration that  moves information is that its models are defined for the “new” schema, but has to operate on data stored in the “old” schema.  I see two alternative approaches:

  1. Use inline models to represent your old schema
  2. Run the migration in stages (1: add new model, copy data, 2: remove old fields)
I tried the second approach first, and that’s what’s implemented in this gist.

Points of interest and gotchas:

  1. Migrations are just more Ruby code.  Require the migration file and test it just like any other code you write.
  2. Factory girl makes it just as easy to seed “old” data as “new”.   Just use Factory.build, and in Rails3, use Model.save(:validations=>false) to skip validation checks.
  3. Use ActiveRecord::Migration.up_without_benchmarks to suppress output from migrations.  I didn’t find any documentation for this method, though up_with_benchmarks shows up in APIDock.

We’ve been thinking about the robustness of ActiveRecord migrations for a few weeks now.  Look for more detailed discussion of migrations in general in a follow on post.

Fast Browser-testing with WebKit and Selenium

We’re happy to announce that we’ve rolled out support for testing Rails and Rack applications using capybara-webkit!

We’ve been using this fine tool from Thoughtbot to test some of our own Rails applications, and we highly recommend it.

The traditional problem with browser-based testing frameworks (like Selenium, watir, webkit, envjs, etc.) is that they’re slow.  Tests often take 30-60 seconds to run, each!  That’s fine when you’re just starting a project, but after a few weeks of development,  you could be waiting an hour or more for your tests to finish.  With Tddium, those tests run in parallel in a few minutes, and they don’t hog the CPU on your laptop or workstation, freeing you to get back to work (or Facebook, or turntable.fm, or …).

We’ll round out our capybara testing capabilities in our next rollout, when we’ll offer Selenium 2 (in addition to the Selenium RC support we’ve had for several weeks).  Stay tuned!


Get every new post delivered to your Inbox.

Join 369 other followers