Debugging test cases

21 August, 2007

Hurray it's the middle of summer and obviously like most people I am quite focused on testing code. What? You mean that isn't a typical summer activity :D?

I've written a few tests with the CakePHP test suite now and can safely say that after you have written your first one, it's very easy to write another. One thing that is a great help but not so clear how to do is how to debug a test while you are writing it. So here are a couple of tips on the topic..

Follow the leader

If you are struggling to get to grips with using the test suite in general make sure it's installed correctly (for the test suite code itself you don't need to install anything but you do need to have simpletest installed as a vendor) and then find something similar to what you want to do and use it as a template. There are many, many test cases already written for the core, and more than likely there is something you can use as a guide for your own test cases. I will shamelessly point out the Tree behavior test case as I think it is pretty good example of how to test behaviors including setting up fixtures etc.

The test suite is robust

The test suite works very well. So, to explain by example, if you see an error message saying something like "missing table test_suite_your_models doesn't exist" there will be a misnamed file, class or variable somewhere. Debugging the testing core/lib classes may lead to the answer but almost certainly the answer lies in the (app!) test files you are editing.

Debug and Die

In my own code, I probably write most often die(pr($aVar));. Of course this never remains but is even more useful when writing test cases. Why? The test suite takes care of creating tables for tests which use models, managing the execution of each test (including clearing out the data and adding it again if defined by the fixture definition) and then dropping the test table. This last step can be a pain - if the test isn't doing what you are expecting and you can't see the table data that the test was querying, debugging becomes a bit difficult. However: debug, die and a quick trip to the db to see what's inside that test database table and the answer to the elusive "why does this test not do what I expect" may become apparent.

See the SQL

Maybe seeing a snapshot of the table data isn't enough, a great aide to figuring out why data/model test cases aren't doing what you are expecting is to see what sql is being executed, but by default this is hidden from you. However by overriding the start method for a test case as follows:

you will then see the familiar debug table at the bottom of your screen when running tests. But there is still a problem in doing this - a test case may generate a lot of sql, the sql log could fill up and stop logging before it gets to your problematic final test. A way to get around that is to only enable full debug when you actually want to see what's happening like so:

An alternative approach, which is probably very obvious, is to comment out all of the preceding tests such that the test you want to investigate is executed first. There could be other advantages in doing that also - if the test takes some time to run, time between runs, edits and reruns is reduced.

Wrapping Up

I've written here a few things I've learned whist getting to grips with writing tests with the CakePHP test suite. I hope they are useful, and if you have any additional pearls of wisdom please add a comment.

Bake on!