active_sanity: Spot invalid records in your database

ActiveRecord is great way to ensure your models are valid before saving them to the database. But as you add new validations or update existing ones, some of the existing records could become invalid. Do you have any invalid records in your production database?

It’s too late when you discover there are invalid records in your production environment.

  • Maybe a migration failed half way through when you deploy to production “It was working just fine on staging?!”
  • You get notified about a weird bug (and you can’t reproduce it on your dev machine)
  • Someone sends you an email saying that he can’t buy the dancing banana t-shirt on your webstore (and you’re lucky someone contacted you. How many people went to a concurrent webstore?)

Setup active_sanity, run it and fix your records.

  1. Add the following line to your Gemfile:

      gem "active_sanity"
    
  2. Run rake db:check_sanity on your production database. You should see something like:

    model       | id  | errors
    User        |   1 | { "email" => ["is invalid"] }
    Flight      | 123 | { "arrival_time" => ["can't be nil"], "departure_time" => ["is invalid"] }
    Flight      | 323 | { "arrival_time" => ["can't be nil"]
    
  3. Want to store those “invalid records” in the database to check them in your admin interface? Just run rails generate active_sanity to generate a migration and access the data through a model called InvalidRecord.

Run active_sanity against your production environment, say “OMG!” and go fix your records. Deployments will be less stressful and you’re gonna sleep better at night.

    • April 1, 2011
    • Vijay Aravamudhan

    Awesome – we actually implemented something very similar (called active_record_validator) for a client, but due to IP issues, I could not release the code. I havent looked at the code yet (definitely will), but
    1) are you allowing loading of classes from Rails.root/lib – and other [user] configurable directories? We had multiple projects which had custom directories. If the whole rails env is loaded, then this is not an issue, btw
    2) are you allowing configuration of some classes (like audit records) which can be excluded? We used to have an exclusion list read in from a yml in the config directory to achieve this. Also, some defaults, like abstract classes, BaseWithoutTable (from active_record_base_without_table plugin), etc could be automatically excluded by the gem itself.
    3) Finally, its a good idea to hook this up, not only for production, but also in the test environment, where the reference data is usually sitting on top of the bare/base schema. This will in turn become a no-op for OLTP data tables, but can validate your reference data during build time itself. You can probably put that into the description of the usage.

    Great Job btw,
    Vijay

    • April 1, 2011
    • Philippe Creux

    Thanks for your comment Vijay!

    1) We load the rails environment and then load all ActiveRecord::Base.subclasses. So you should get all the existing models in production environment.

    2) There is no way to exclude classes. Pull request are welcomed!

    3) We use it on staging, demo and production db actually. :)

    Cheers!

    • April 1, 2011
    • Vijay Aravamudhan

    Thanks – I have already forked on github
    Will send a pull request later this weekend. Btw: Dec instructions are incomplete – I had to create a new rails project to run the cucumber tests. My commit should hopefully fix that..

    • April 1, 2011
    • Vijay Aravamudhan

    Forget that last comment. There was a different bug which has been fixed in my commits. Have already sent a pull request.

    • April 4, 2011
    • Andrea Schiavini

    Nice one… but I suggest you to add that it depends on rails 3 :)