All your folders are belong to us

My hate for a folder tree in MiniG never was a secret :

• It requires lot of screen real estate
• very hard to manage in a web context
• a tree helps for mail triage. When I began using internet, altavista was becoming a leader and yahoo was loosing marketshare. Google didn't exist. At this time yahoo was trying to "categorize" the web while altavista understood that trying to create a tree on top of internet-scale data was a lost fight.
• Displaying a tree is the easy solution : first thing I learned while attending Algorithm101 was that in computer science, everything can be represented with a tree.

A tree was clearly not the way I wanted to go. It's fine for a small bunch of data. It's also fine when you don't have any other way to do the triage of your data.

To make things short, folder tree is out of minig, for good. As I have to live with IMAP servers, I still had to find an efficient replacement. The replacement has the following features :

• dynamic displays that only shows folders with unread emails by default
• incremental search : a text field on top pushes the search paradigm of minig to your folders. Just type "m" in the field and you'll get your "MiniG" "Mini Me" "foo/mmm/bar" folders
• dynamic color assignment : each folders gets a color. This color is used when showing search results to easily identify where they are stored (in your inbox, in a shared folder, in your sent messages)

Even some of my users that felt in the "A folder tree on the left of a mail client is mandatory" category are saying some nice things about the new display.

BaraFolder (internal name of the new folder widget) (lot's of private jokes around that name) is available in the latest stable MiniG release we did today (2.2.15.1166). This release will hit about 1500 customer's mailboxes next week.

A release candidate of this minig version was shown to one of our customers last week and he said "no, I don't need the calendar demo, I want this" (ok, he was an easy target as we deployed horde/imp for him years ago). Given the time me, david and others dedicated into this release, I hope users will enjoy it.

Next steps are not set in stone, but big minig deployments (as in 8000 mailboxes) are planned. My work items should be :

• clustered deployement : N-frontend, M-backend
• bugfixes
• showing the filter feature in the UI. MiniG can write sieve filters, but it's disabled in released build as it conflicts with some obm features (vacation, forwarding, etc)
• bugfixes
• composer. this part of minig really needs some love. Even more when you see what google wave can do (no, the plan is not to be able to insert a "poll widget" in your signature)

PS : MiniG is free software, released as GPLv3. No paying-customers-only-crap-for-the-others version.

Random links of the weekend

Still exploring what Cassandra could provide as a clustered mail storage.

Easy read on cassandra data model : http://arin.me/code/wtf-is-a-supercolumn-cassandra-data-model

Using this article, I drafted an early data model :

Conversations : { // CF
    INBOX/123-1 : { // Row key is the conv id
 timeuuid_1 : messageId1
 timeuuid_2 : messageId2
    }      
    INBOX/123-2 : { // Row key is the conv id
         timeuuid_3 : messageId3
         timeuuid_4 : messageId4
    }       
}

Folders : { // CF
     INBOX : {
           timeuuid_1 : INBOX/123-1
           timeuuid_3 : INBOX/123-2
     }
}

Messages : { // CF
   messageId1 : { // row key
       subject: bla bla
     to: xx
     cc: yy
     bcc: zz
     preview : hello world
                   body: hello world ! qdqsd
                   hasAttach: true
                   hasInvitation: false
                   attachments: attach_123, attach_456
   }
}

Attachments : {
    attch_123 : {
        filename: fileName1
 size: 123456
        content: xazeqsdqdqsd_base_64
    }
}

Article on lucene/solr + cassandra : http://nicklothian.com/blog/2009/10/27/solr-cassandra-solandra/

As I wanted to store my tests on those technologies somewhere, I created the BigG project on googlecode.