MiniG slow down; OBM 2.2 deadline coming soon

As you may have noticed, MiniG development is stalling. Last commits were a major feature : reading emails forwarded as attachments.

OBM 2.2 database model will be frozen on monday, we are working pretty hard on fixing most schema glitches.

Lots of things are coming in OBM in 2.2 :

- MySQL won't be the default database, PostgreSQL 8.3+ will be (if you wonder why, give us 64bit timestamps, jdbc compliant driver _by default_, predictable optimizer choices and we will reconsider. A fix for yesterday's mysql busy lockup on a simple query at one of our biggest production site would be a plus).

- Unicode support (ok, as I'm working on the java parts of the solution, this one makes me laugh)

- Improved contact model & revamped sync servers to provide a "smartphones compliant" model

- Usable mozilla lightning support, with asynchronous synchronisation and greatly improved response times !

In my next post, I'll probably give information on how our sync servers went from 5/sync per sec (single threaded benchmark, mysql) on empty database to something like 30sync/sec (postgresql, single thread, no optimisation (full logs & timing on pg)).

Chat in MiniG, Jabber in OBM

As jabber integration is important (think lotus sametime, online meetings, etc), I did some prototyping on jabber support in MiniG ala GTalk.

I did a little screencast of what OBM+MiniG can do with an Openfire jabber server tonight.

Of course jabber integration is OBM 2.3 stuff, I'm just investigating what would be possible.

The MiniG / Jabber video is also available in poor quality on youtube:

Drag'n'drop support in MiniG

Well title says it all : you can move you conversations by drag'n'dropping them from the list to the folder tree.

It works for one or multiple conversations. My preferred way to move conversations is still with the actions combo-box, as I find it less error-prone.

As a screenshot does not really work for this kind of features, you can download an xvid screencast or watch the youtube version :

Turned 11

Now linagora GSO, previously Aliasource and Aliacom, the company I worked with for 7 years turned 11, cheers !

planet.obm.org

Thanks to Sylvain, my blog is now federated on planet.obm.org. It's quite cool to have a central place to report OBM and MiniG progress.

As I am working on MiniG (and bdf12) most of my time, let's report what happened in the last days :
- we switched to gmail sorting of conversations
- David fixed lots of ui bugs
- search results are now ordered by date desc (David too)
- email (un)starring is coming soon (the \Flagged imap tag)

Another good news : debian packages of obm & minig will be available soon. Installation nightmare is over. By the way, Obama got elected.

OBM technical integration

While in Agde, at OBM developpers summit, I did the first steps of technical integration of MiniG with OBM :

* Private & public contacts from OBM are visible as contact groups, used for email address autocompletion
* Email address gathering : recipients of emails sent through MiniG are automatically added as private contacts in OBM. They will be available for autocompletion next time the MiniG composer is used.
* OBM user settings are pushed to the MiniG ui, to avoid redundent preferences (language, timezone, etc)

The game is over

OBM developper summit is now over. Week summary :
- lots of commits
- a lot of fun
- some wine, some beers

Late night coding

6 laptops, some wine, 1 o'clock... and el paquito !

The double while pattern (episode two)

Well, I could write about the second loop around the first while pattern, used to compute statistics about how buggy "buggyProgram()" is, but... pictures are worth thousand words !

The double-while pattern

Ok, this started with a mysql bug with foreign keys : the schema just crashes randomly with syntax errors, but "sometime" succeeds.

No problem, here comes the "While Pattern".

Problem to solve :
A program crashes randomly but sometimes work

Solution :

boolean success;
do {
  try {
    buggyProgram();
    success = true;
  } catch (... well anything) {
  }
} while (!success);

Bonus point : the low voltage version with a 1sec sleep after the buggy program call.

Stay tuned for the double while pattern ;-)

OBM developper summit in Agde

The whole OBM team is spending one week in Agde to work on the major features of our 2.2 version, coming at the end of the year (hope so).

The first two days were pretty productive, with time zone support nearly done, conversation previews in MiniG. Debian packaging of alpha version is in good shape now.

Good coding week for MiniG. "from:tom filename:pdf" just works (all your mails from tom with a pdf attachment). All Gmail advanced search features are implemented and working :D

Next week I'll be giving a Tomcat training, but the minig team will be focused on email forwarding, templates implementation and UI polishing.

New minig release

Just released new MiniG backend & frontend binaries on minig.org. Also made an announcement on freshmeat to grab new users/testers.

This new release features folder tree, instant unread count updates, html email cleanups to strip images & javascript and my new imap4r1 library.

Just commited my new Java IMAP client library to MiniG svn. My lib is based on apache mina and looks like ristrerro imap api, with speed added. Pretty happy with the results.

Ristretto was fine, but switching to a mina based implementation cut the time for fetching an email from 170ms to 8ms :-) My new API still use a lot of Ristretto for header decoding & mime parsing, they did a great job.

MiniG can now send emails with attachments. Next step is attachments downloading.

One day, one history, today it is :

117 cd
107 ls
51 svn
27 emacs
18 cat
12 rm
11 scp
10 ll
9 zless
8 start_vm.sh

MiniG is getting attachments support. Pretty hard task, but somewhat fun.

Olivier made my day

Just got new minig logo. From MiniGmail to MiniGoat :p

History meme

history | awk '{a[$2]++}END{for(i in a){print a[i] " " i}}' | sort -rn | head

124 cd
94 ls
59 svn
15 ll
15 emacs
13 rm
10 dpkg
9 start_vm.sh
9 cat
8 sudo

Just came back from a training about enterprisedb, a flavour of PostgreSQL with complete Oracle compatibility. Never heard of them before, and had a good surprise : 3 of the 7 pgsql core team members are working for them. They are the guys that designed the WAL system which finally made PostgreSQL an enterprise class database (at least for me). Their implementation uses a pretty cool dynamic tuning system to get a correct PostgreSQL setup for your server with only one setting.

Side note : London is a very expensive city, 63 pounds to go from heathrow to town center by cab, 4 pounds for an underground ticket :/

Moved my webmail code from our internal svn to http://code.google.com/p/minig and registered http://minig.org/ as the domain was available. Registered MiniG on ohloh.net and freshmeat.

Kubuntu saved my life. I was playing with the partitions on my laptop hd. I first used gparted, but as it didn't want to move to start of an extended partition to the right, I rebooted on a partition magic 8 cd. It asked me to convert the partition to extendedX, and I said no, as I didn't know what an extendedX partition was.

It proposed me to convert my /dev/sda5 to a primary partition, so I did it... reboot... and BANG, grub error 17. Reminded me of lilo old times when you forgot to run lilo after compiling your fresh new kernel. My "/boot/grub" was on /dev/sda5 at this time, and I forgot about it. Starting to panic, I don't have any rescue cd around. Grabbed my keys, 10min car drive & I had a kubuntu 7.10 cd rom, as one was available with "planete linux".

Boot kubuntu, sudo -s, mkdir sda3, mount /dev/sda3 sda3, chroot sda3, cd /boot/grub, grub --device-map=device.map, root (hd0,2), setup (hd0). My system is now up & running again.

GWT meeting took place in atlanta last week. I'm eagerly waiting GWT 1.5 release with generics support.

Tonight I spent 2 hours reading & commenting stevey's blog about code size. Here is my comment to his article :

I Think you should try the "play" programming language. Its semantic is quite rich : the program "Play;" will run a game with all the features of your 500k game.

Ok, I'm just kidding but you have some very valid points : trying to factor (I'm not saying refactor) your code, and reusing instead of duplicating is a mandatory thing.

What I'm not buying is your focus on the "LoC" metric. Your 500k lines of code break eclipse, and this is good thing. For me it means that the way the code is architected, eclipse must load all the code to figure it out (I mean compile / index it) : exactly like if the code was a single 500k line script, where line 500.000 depends on line 1.

Divide to win

Let's think about it for a few seconds, what do you have in a game : the AI, the graphics engine, the physics engine, the audio engine, lots of data file and a "scripting engine" to glue everything together. It is pretty clear that your AI does not care about the colors of the textures on your polygons.

Now, would you consider a technology like OSGi / Eclipse plugins ? You focus on writing the smallest possible piece of code. Everytime you're willing to add a new feature, you ask yourself : "Can I add an extension point here and program my new feature as an external plugin ?".

Now my feature is a separate project Y in my IDE. I create a new project Ytests, that depends on Y, and unit test the code in Y. When Y performs correctly, I package it (right click / export / export as deployable plugin & fragment), push the code to git/svn/you name it, and close the project Y in eclipse.

Another point is, when I start writing a program, do I use plugin / modular technologies from the start ? Can I know in advance that my little project will be successful, and will grow to X lines of codes ?

Finally reattached my blog to my google account. Will stop wondering what login / password I might have used while filling the registration form.

New year promise : I will never ever do a birthday party because my last blog entry turned 2.