Since posting a simple script to install Yawns, the code behind this site, upon a clean Debian Sarge installation I've been having a few problems.

I have a reasonably thorough test suite containing more than 1500+ individual tests! Together these test the code and makes sure all the modules work correctly however despite working correctly on the live site, and on my development versions of the code at home (These typically run with a dump of the live site database) they were failing on pristine installations.

(Of course I still have problems which aren't caught by the test suite just yet - I'm in need of some cases to validate RSS feeds, etc. I will add those once I work out how. I think there should be a feed-validating command line tool available which I can invoke?!)

A couple of days ago I realised where the problems on new installations were coming from: poor assumptions upon my part.

For example I have a test case which attempts to post a comment on an article then verify that this new comment appears in the RSS feed for the relevant article. The code goes something like this:

1. Ensure article 0 doesn't exist
   • Create it.
2. Get the number of comments on this new article
   • Test there are none.
   • Test that the RSS comment feed for the article is also empty.
3. Post a new comment:
   • Author "Steve"
   • Title "Test title"
   • Comment body "Test body"
4. Now fetch the comments on the article
   • Ensure the new comment is present.
   • Ensure the new comment is in the RSS comment feed for the article.
5. Delete the article.
   • Ensure the comment is removed.
   • Ensure the comment is removed from the RSS comment feed.

Can you spot the mistake? Took me a very long time to see it, but in short on a new installation of the code there is no user "Steve". Since comments can't be posted by non-members of the site the "get comments" step was failing even though the comment was actually in the database.

To make the test case work as expected I simply create a new user at the start with a random name, use it for posting the comment, then delete it when the test finishes.

There were a few similar problems with other test cases, but now they are resolved.

I like having test cases .. even if I have to spend hours working out why they fail on new installations!

Now that all this is fixed, and my security fixes are live, I can go back to working on the notification system improvements.