This is the evolving post to show the full test suite fixing progression.

The beginning of this graph does not really show the reality of the passing/failing tests for the full test suite at the given dates: I just added the integration tests values on the 1st of May to previous values concerning unit tests only. Although this is not perfect, it gives a good sense of the work done and what is still on the table.

These tests don’t show the state of the libraries that have been extracted from Zena into gems (Property, Versions, QueryBuilder, Zafu and RubyLess).

red = errors, orange = failures, green = pass

1. [465] We are back at 76% tests passing (1/5/2010).
2. [475] Did not improve passing tests but test.host works and implemented part of the /rescue idea.
3. [403] Fixed error messages, set var_ and recursion
4. [366] Fixed custom queries and some other bad bugs in query builder C extension.
5. [313] More random corrections (urls, cross class query building, anchors, etc)
6. [298] Fixing ugly bugs related to OrderedHash.dup (took some time to find). I thought I was swimming in Quantum Entanglement...
7. [265] Fixed some bugs related to input fields rendering and role loading to properly set NodeContext.
8. [228] Fixed checkbox and many small bugs related to QueryNode .
9. [213] Nothing to say
10. [200] Implemented search and columns (green bump). Fixed form building, if, ..._if and is_ancestor?
11. [182] Implemented property indexing (June 4th) and relations using class as role name.
    It’s the first time we have more then 2000 tests passing!
12. [171] Fixed Controller tests, using idx_text_high for ‘title’ multilingual index (that’s funky!).
13. [138] (6/6) Fixed drag & drop and other ajax edit.
14. [111] (15/6) Fixed conditional, captcha, recursion, tag wrapping (implementation of toggle and query).

bug fix speed

Every recording (on average) represents a work day spent fixing bugs (~7 hours). This means that the average bug fixing “speed” is less then 1 bug per hour… This is because every bug fix in Zafu requires the rewrite of all the features related to the bug (since RubyLess changed everything).

Doing such massive rewrites without proper tests pass/fail validation (since many don’t pass because of other features missing) tends to hide bugs and make them harder to nail down.

A possible solution would be to disable failing tests and reenable them one by one, making sure all “enabled” tests pass at all times. Divide and conquer…