Sometimes, it’s nice to be able to open a console or mockup a little script to do some work on the data that lies on the server.

Some real life examples include:

1. moving a bunch of objects around
2. renaming due to new naming conventions
3. delete many elements
4. import content from another source (CSV, database, etc)

To make such operations fun, we have created the Zena::Remote library.

Zena::Remote

You can get all the details here, but a small IRB session should give you the gist of it:

$ irb
>> require 'zena/remote'
=> true
>> remote = Zena::Remote.connect("example.com", "super secret token")
=> ...
>> c = remote['Contact'].new(:title => 'Zumbi', :parent_id=>1)
=> ...
>> c.save
=> {"fullpath"=>"Zumbi", "position"=>0.0, "klass"=>"Contact", "name"=>"Zumbi", "created_at"=>Mon Aug 30 19:37:10 UTC 2010, "title"=>"Zumbi", "updated_at"=>Mon Aug 30 19:37:10 UTC 2010, "event_at"=>nil, "log_at"=>nil, "v_status"=>50, "id"=>393, "parent_id"=>"1", "node_name"=>"Zumbi", "ref_lang"=>"fr", "kpath"=>"NRCC"}
>> remote.all("contacts")
=> [...]
>> remote.delete("contacts where title = 'Zumbi'")
=> [...]

This XML based API is very powerful (you can mass-delete, mass-update, use relations as methods, change links, etc).

Have fun !

After a long a difficult path with lots of rewrite, the 1.0 branch is finally ready for experimenters between the beach and the disco (or between two hiking trails).

Main changes

We are very excited about this 1.0 release because it will solve many issues we had on the 0.x branch. Issues mainly related to scalability (increasing complexity without decreasing performance). The biggest problem was “how do we store multi-lingual, versioned custom attributes ?”. We also wanted to enable proper multi-lingual sorting in the database. We solved this issue with the property gem.

The second issue was extensibility: it should be easy to evolve the templating language or adapt it for a custom application. Method declaration should be trivial and safe. See rubyless for details.

Extract from the changelog:

• complete rewrite of the storage engine (property gem)
• rewrite of the template engine zafu to use rubyless
• rewrite of the query compilation engine to use RubyLess and offer better introspection
• XML API with support for pseudo sql queries
• enhanced attributes support (indexing, inheritance, type)
• changed bricks layout to be like gems
• pdf generation support
• started PostgreSQL support
• inline JS windows instead of popup with attach/dettach support

Why it’s not a release candidate

Here is a short list of the things this release is missing in order to replace the current 0.x branch or to have our confidence as a production version:

• missing features in Zafu: not all features from the 0.x branch have been rewritten for this branch. This means that until we have ported all existing websites that we manage on the 1.0 branch, we have no guarantee that everything that was done on the 0.x branch can be done in the 1.0.
• field bugs: this version, although it is extensively tested way beyond what was done before has not been tested in the field. This means that there might be some annoying bugs that our test suite does not catch.
• possible API changes: the templating language has been rewritten and we hope the beta versions will help us correct things that are counter-intuitive.
• missing documentation: this is not really related to our code quality expectations as above, but it is part of the ecosystem. We need more tutorials, reworked website navigation, etc.
• ticket tracking cleanup: during our big rewrite we have pushed the bug tracker on the side. We need to update the content there to reflect the current version.

Experiment and give us some feedback !

Install the gem with:

# sudo gem install zena --pre
# zena myapp
# cd myapp
( eventually change config/database.yml for MySQL )
# rake zena:init RAILS_ENV=production

Please report any bug on the bug tracker with the tag beta. You can also discuss issues on the mailing list, but beware that it’s a “summer” release with slow response times so don’t panic if replies take some time.

Thanks a lot for your patience (this one took a while to come out).

This is the evolving post to show the full test suite fixing progression.

The beginning of this graph does not really show the reality of the passing/failing tests for the full test suite at the given dates: I just added the integration tests values on the 1st of May to previous values concerning unit tests only. Although this is not perfect, it gives a good sense of the work done and what is still on the table.

These tests don’t show the state of the libraries that have been extracted from Zena into gems (Property, Versions, QueryBuilder, Zafu and RubyLess).

1. [465] We are back at 76% tests passing (1/5/2010).
2. [475] Did not improve passing tests but test.host works and implemented part of the /rescue idea.
3. [403] Fixed error messages, set var_ and recursion
4. [366] Fixed custom queries and some other bad bugs in query builder C extension.
5. [313] More random corrections (urls, cross class query building, anchors, etc)
6. [298] Fixing ugly bugs related to OrderedHash.dup (took some time to find). I thought I was swimming in Quantum Entanglement...
7. [265] Fixed some bugs related to input fields rendering and role loading to properly set NodeContext.
8. [228] Fixed checkbox and many small bugs related to QueryNode.
9. [213] Nothing to say
10. [200] Implemented search and columns (green bump). Fixed form building, if, ..._if and ancestor?
11. [182] Implemented property indexing (June 4th) and relations using class as role name.
    It’s the first time we have more then 2000 tests passing!
12. [171] Fixed Controller tests, using idx_text_high for ‘title’ multilingual index (that’s funky!).
13. [138] (6/6) Fixed drag & drop and other ajax edit.
14. [111] (15/6) Fixed conditional, captcha, recursion, tag wrapping (implementation of toggle and query).

bug fix speed

Every recording (on average) represents a work day spent fixing bugs (~7 hours). This means that the average bug fixing “speed” is less then 1 bug per hour… This is because every bug fix in Zafu requires the rewrite of all the features related to the bug (since RubyLess changed everything).

Doing such massive rewrites without proper tests pass/fail validation (since many don’t pass because of other features missing) tends to hide bugs and make them harder to nail down.

A possible solution would be to disable failing tests and reenable them one by one, making sure all “enabled” tests pass at all times. Divide and conquer…

I did some testing to compare MySQL with SQLite3 on simple operations:

• display index
• create an entry
• edit entry
• publish
• delete entry

Here is the result:

As you can see, most operations are slower on SQLite3 and “write” operations (create/update/delete) are really bad.

All this to say: do not use SQLite3 in production !

tutorial

Manu has made an nice tutorial on how to deploy Zena within an fcgi based hosting (processes started on the fly by apache).

Follow the link: zena@alwaysdata

We are talking with the nice guys at alwaysdata.com to see if we can preinstall some gems to make this even easier… Stay tuned…