Active Directory SSO

ActiveDirectory authentication

Blue Mind already had sign-on ActiveDirectory capability. This one is about re-using the windows session and kerberos login.

The video shows what Blue Mind 3 can do:

Yes, simple re-use of your window session to authenticate in Blue Mind !

Behind the curtain

b1...b7, rc1....rcN, X.0

Depuis quelques temps maintenant nous sortons des versions de Blue Mind. On leur donne des noms "marketing" genre beta1, beta7, rc1... mais qu'est ce qui ce cache derrière tout ça ?

Scrum & versions

L'équipe utilise une méthode scrum avec des sprints. Ces sprints peuvent se retrouver dans nos numéros de version :

• 0.22.1234 correspond à l'itération 0, branche de développement de la version 1. Au 22ième sprint, et au build 1234 de notre intégration continue.
• 1.0.1566 est une version stable. Notre première normalement. Le numéro de sprint restera toujours à 0, les bugfix entrant dans cette branche étant indépendant de planning agile. 1566 est toujours notre numéro de build jenkins.
• 1.1.1755, une version intermédiare, on the road to "2.0.x".

Numéros de build

Les numéros de builds sont importants pour Blue Mind. Il permettent d'identifier de manière unique une version. Ils sont utilisés par le Setup Wizard dans la version pro pour exécuter les mises à jours : je vais de 0.22.1234 à 1.0.1666. Je dois exécuter tous les programmes de mise à jour entre ces versions.

Comme pour beaucoup de projets, ces numéros sont généralement soit :

• un numéro de révision svn
• un numéro de build jenkins

Le notre est "presque" un numéro jenkins. Mais voilà, Blue Mind a de nombreux composants et son intégration continue est compliquée....

Un build complexe

D'après ohloh, Blue Mind c'est 64% de java et 18% de javascript (le reste étant de la plomberie XML/sh autour de ça). 

Ceci n'est pas complètement vrai. Ohloh n'indexe que la version community. La version pro ajoute son lot de fonctionnalités, et de languages :

Totals grouped by language (dominant language first):

java:        206174 (93.63%)

cs:           11981 (5.44%)

sh:            1789 (0.81%)

jsp:            149 (0.07%)

cpp:            106 (0.05%)

Ci dessus le resultat de sloccount sur le repository "closed" de Blue Mind. Les 11000 lignes de c-sharp du connecteur outlook sont une partie de problème.

Pour compiler ce genre de choses il faut :

• des slaves jenkins sous windows
• visual studio, VSTO, etc

Et même la version community n'est pas si simple :

• compilation de cyrus sur :
• lucid
• precise
• rhel6
• compilation du connecteur thunderbird sur une lucid 32bits
• déploiement des artifacts de compilation (deb, rpm, xpi, etc) et execution des tests unitaires sur ces plateformes

Et pour publier tout ça :

• des dépots public pour distribuer les beta/rc/versions
• des dépots pour nos clients avec les briques propriétaire

Ce qui represente pas mal de machines virtuelles VMWare avec diverses fonctions.

On the road to 1.0

Jusqu'à maintenant nous n'avions que la version "de dev" qui va devenir la 1.0. Et c'était déjà pas mal :

• 16 projets jenkins
• 2 gits, 3 avec celui pour nos dépendances de compilation. Plus les git de cyrus...
• des machines virtuelles de tests : rhel6 (rw), lucid (rw), ldap (ro), active directory (ro), hosts (rw, la machine qui nous sert à tester l'ajout de nouveau serveurs sur une infra)

Disons que chacun de nos projets jenkins a son intêret. Sortir une version maintenue implique avoir le même niveau de qualité sur cette version. Donc à minima :

• 2 branches git à créer
• 16 projets jenkins à cloner
• 3 machines virtuelles à dupliquer

Pour faire ce genre de trucs, normalement, il vaut mieux mettre la majorité de l'équipe en RTT et 1/2 personnes font le travail.

Chez Blue Mind un des principes fondateurs est de tout indistrualiser, que ce soit pour le commerce avec scenari et e-deal ou pour la R&D.

On a donc essayé de dé-dramatiser cette 1.0 à venir, et on a anticipé sur une fréquence de release rapide (le commerce dirigeant la technique, s'ils veulent une version, on doit pouvoir la fournir).

L'objectif était :

"en tant que MembreDeLequipe je peux créer une nouvelle version à volonté et bénéficier des mêmes contrôles qualité que sur la version en cours de dev"

Backoffice is born

Ce qui est cool avec tous les outils que nous avons en place :

• git
• jenkins
• redmine
• vsphere

C'est qu'ils ont tous des API java utilisables. Automatiser notre process de release est donc surement faisable. On a besoin de :

• brancher nos git
• cloner nos projets jenkins et les re-pointer sur la nouvelle branche
• cloner nos machines virtuelles de test
• faire des commits dans la nouvelle branche pour cibler les tests vers les nouvelles VMs
• mettre en ligne la version

On ne voulait pas non plus que pour faire l'opération de release 4 personnes soient nécessaires : attend c'est Sylvain qui sait comment on upload la release et fait la branche. Faut que Thomas copie et modifie nos projets jenkins, etc.

Comme quand on a un marteau, tout ressemble à un clou, on a refait toute une application de "backoffice" avec le même niveau d'industrialisation que Blue Mind.

bm-backoffice

Un web service assez proche de Blue Mind Core dans sa structure, mais avec des API différentes. Il nous sert aussi de test-bed pour des évolutions futures de Blue Mind core. Par exemple au lieu de faire du XML/RPC il utilise une API web service à base de JSON qui utilise Jackson, assez proche de celle de http://get.gaug.es/documentation/api/ 

bm-backoffice-wizard

Une wizard d'installation / upgrade pour maintenir la base de données du backoffice.

bm-console

Une console d'admin dédiée à la gestion de nos builds/sprints/releases et leur mise en ligne.

De bons outils pour un bon niveau d'industrialisation.

Cet outil sait brancher nos gits, va automatiquement cloner et mettre à jour les projets dans jenkins, et sait mettre une release Blue Mind en ligne. Le travail sur nos VMs au travers des API vsphere est en cours.


What else ?

Blue Mind beta1 (part1)

Blue Mind beta1

Lots of work went into this release. Of course we always want more, but the release should be pretty solid.

Let's have a quick look at what is really nice in our new software.

Easy setup

Installation is just 3 clicks away after you installed your packages. Basically you just need an ubuntu 10.04 setup reachable through its hostname and you're good to go.

Simply point your browser to your ubuntu server.

Your first click will setup your PostgreSQL database and reconfigure the needed services

The next screen will verify your network setup is correct. Your server must be able to contact itself through its hostname.

Next question is about your domain. Blue Mind handles mail domains.

And that should be your last click. Enjoy the progress bar while the installation wizard sets everything up for your new mail domain.

And here we are ! complete mail server setup with a working account.

Don't forget to write down your setup wizard password and you are go to go right into the admin console. Please note the private/public computer switch. This is for our offline mode support. When checking public computer, Blue Mind's calendar will not dump all your data in your browser's storage for offline use.

Now you can manage everything, setup your users and add new mail domains.

Back to business with BlueMind

BlueMind fait ses débuts et premières démonstrations au salon JRES à Toulouse.

Plus d'informations sur le Blog de Sylvain.

Jobs available

E-DEAL recrute !
4 Ingé. de Développement / Chef de projet Junior
2 Ingé. Recherche & Développement
1 Chargé(e) de Comm.
1 Responsable de la Lead Generation

En savoir plus, postuler : http://www.e-deal.com/gestion-relation-client/recrutement-crm.html

More groupware related news coming soon. My non-compete clause is over :D

You have been unsubscribed from the Obm mailing list

Nice move from my past employer, pushing me out of our free (?) software mailing list.

I guess helping people without getting anything in return does not comply with the new rules... http://obm.org/doku.php?id=tools:ml:charte

minig 2.3.10.1545

Few things fixed in this release :

• login with complicated characters in password


• html email was not correctly indexed

This should fix opush problems as opush & minig IMAP lib are shared.

The place to be

Rate your rugby referee !.

Singleton for dummies

While searching for some coding practices, I realized some people are wondering how to do a singleton in Java (I know, this is considered an anti-pattern when badly used) :

public class Controller {
 
 private static final Controller ctrl;
 
 static {
  ctrl = new Controller();
 }
 
 private Controller() {
 
 }
 
 public static Controller get() {
  return ctrl;
 }

}

That's the _only_ way to do it correctly in Java. Your static block is protected by the classloader lock, so everything is fine. This implementation _can_ bug, but that supposes you know how to subclass/implement a ClassLoader. In a webapp it's fine if you're wondering (well tomcat will leak your singleton memory if you abuse hot deploy).

I made this post as I'm reviewing some opush contributed code. What you can find on the net about java & singleton is quite fun. Nice article by someone who didn't know what are static blocks. Static blocks are "Java approved" since 2000.

Leaving my current job

I'll switch to another job in october. My work on MiniG & OPush will continue. I created those projects and I'm proud of them. tcataldo@minig.org is a working email to reach me.

I'll have some time to fix flawed things in MiniG with my next assignment ;-)

Bigger than free software

Having fun with all our co-workers

New minig feature

MiniG can now handle email priority in its conversation list. Threads with a high priority email are shown in red. They can be found using is:important.

Working late

Answered flag is now handled by minig (on conversations & messages).

BigG, GrosG, MaxiG, you name it....

First step is complete : BigG now has an ltmp compliant daemon. At least it's compliant with postfix lmtp client which is sufficient for my needs.

It does not do a lot for now, just outputs some logs while doing the LMTP dialog:

2010-03-22 22:49:59,458 RunnableExtensionLoader INFO - DeliveryFactory loaded.
2010-03-22 22:49:59,462 RunnableExtensionLoader INFO - ValidatorFactory loaded.
2010-03-22 22:49:59,462 RunnableExtensionLoader INFO - Loaded 2 implementors of org.obm.lmtp.lmtpbackendfactory
2010-03-22 22:49:59,499 LMTPDaemon INFO - Starting LMTP daemon...
2010-03-22 22:49:59,565 MinaServer INFO - mina lmtp server created
2010-03-22 22:49:59,578 MinaCodecFactory INFO - mina codec factory created
2010-03-22 22:49:59,589 MinaLmtpServer INFO - Starting listener. active: true h: org.obm.mina.MinaIoHandler@128ae45a
2010-03-22 22:49:59,589 LmtpServer INFO - server started
2010-03-22 22:49:59,589 LMTPDaemon INFO - LMTP daemon started.
2010-03-22 22:50:17,969 BiggLmtpBackend INFO - addressStatus(sga@zz.com)
2010-03-22 22:50:17,969 LdapValidator INFO - validating sga@zz.com
2010-03-22 22:50:18,039 BiggLmtpBackend INFO - deliver(org.obm.lmtp.backend.LmtpEnvelope@7a19a37a, blob: Blob: { path=/tmp/blob2739528671992765153.bin, size=null }
2010-03-22 22:50:18,040 CassandraAgent INFO - delivering to a cassandra node... in the near feature :/
2010-03-22 22:50:18,043 MinaIoHandler INFO - session closed

Postfix is happy with that:

Mar 22 22:50:17 obm23 postfix/pickup[6601]: E27B85E82: uid=0 from=
Mar 22 22:50:18 obm23 postfix/cleanup[6827]: E27B85E82: message-id=<20100322215017.E27B85E82@lenny.buffy.kvm>
Mar 22 22:50:18 obm23 postfix/qmgr[1996]: E27B85E82: from=, size=284, nrcpt=1 (queue active)
Mar 22 22:50:18 obm23 postfix/lmtp[6829]: E27B85E82: to=, relay=10.0.0.1[10.0.0.1]:2400, delay=0.48, delays=0.21/0.05/0.15/0.07, dsn=2.1.5, status=sent (250 2.1.5 Delivery OK)
Mar 22 22:50:18 obm23 postfix/qmgr[1996]: E27B85E82: removed

Next step is to parse the mail with mime4j before giving it to CassandraAgent. The agent will push everything to a cassandra node.

Coming soon to BigG (mostly empty) website.

Jabber in minig

After a first experiment, we are now committed to complete chat support in minig.

Here is a screenshot showing what already works.

Filters in MiniG for 2.3.2

After 6 month with filters code disabled in MiniG, I finally took the last steps to enable them : vacation & forward handling using OBM infos.

The finishing touch was pretty boring :

• adding APIs to OBM sync for updating forward and vacation infos in the UserObm table.
• Completing the vacation ui in MiniG
• Adding an interface to set email forwarding

Well it's done and will be available with all 2.3.2+ obm installs.

Everything is not perfect : obm vacation & forward modules must be disabled as they have no knowledge of the filters stored in minig private database. A quick fix is possible as making an HTTP POST to minig-backend/regenerateSieve.do (login/password required) will generate a correct sieve script with forward, vacation & filter rules.

Back in Toulouse

Just spent one month in Paris to meet few customers and give some training to our Paris "obm core team".

Some projects are now on track : Gilles Blanc is working on a BES (Blackberry Enterprise Server) connector for OBM. Nicolas Chauvet is deploying OBM+MiniG servers and should soon work on improving our redhat packages.

While in Paris, Thomas Chemineau gave Sylvain & I a big training session on LemonLDAP. While I hate having all my code proxied by some perl crap, LemonLDAP is pretty cool. At least it is the only SSO solution that works as a proxy and gives you clean single sign out. I did a simple diagram on how we could implement minig sso using lemon.

This week I'll work on finishing MiniG sieve filtering code :

• handle vacation settings from obm
• use email forwarding infos from the obm database

Brown paper bag release

For the first time in minig history, x.y.z.1166 seems ok. Two new bug reports so far, one of them being that minig requires 1280x1024 for real use. The other one relates to "whole folder actions".

Biggest known bug in my list is "body part selection" : identify which part of an email is the body that must be shown to the user. Separating an email sent as "send as text and html" from a "text email with an html attachment".

All your folders are belong to us

My hate for a folder tree in MiniG never was a secret :

• It requires lot of screen real estate
• very hard to manage in a web context
• a tree helps for mail triage. When I began using internet, altavista was becoming a leader and yahoo was loosing marketshare. Google didn't exist. At this time yahoo was trying to "categorize" the web while altavista understood that trying to create a tree on top of internet-scale data was a lost fight.
• Displaying a tree is the easy solution : first thing I learned while attending Algorithm101 was that in computer science, everything can be represented with a tree.

A tree was clearly not the way I wanted to go. It's fine for a small bunch of data. It's also fine when you don't have any other way to do the triage of your data.

To make things short, folder tree is out of minig, for good. As I have to live with IMAP servers, I still had to find an efficient replacement. The replacement has the following features :

• dynamic displays that only shows folders with unread emails by default
• incremental search : a text field on top pushes the search paradigm of minig to your folders. Just type "m" in the field and you'll get your "MiniG" "Mini Me" "foo/mmm/bar" folders
• dynamic color assignment : each folders gets a color. This color is used when showing search results to easily identify where they are stored (in your inbox, in a shared folder, in your sent messages)

Even some of my users that felt in the "A folder tree on the left of a mail client is mandatory" category are saying some nice things about the new display.

BaraFolder (internal name of the new folder widget) (lot's of private jokes around that name) is available in the latest stable MiniG release we did today (2.2.15.1166). This release will hit about 1500 customer's mailboxes next week.

A release candidate of this minig version was shown to one of our customers last week and he said "no, I don't need the calendar demo, I want this" (ok, he was an easy target as we deployed horde/imp for him years ago). Given the time me, david and others dedicated into this release, I hope users will enjoy it.

Next steps are not set in stone, but big minig deployments (as in 8000 mailboxes) are planned. My work items should be :

• clustered deployement : N-frontend, M-backend
• bugfixes
• showing the filter feature in the UI. MiniG can write sieve filters, but it's disabled in released build as it conflicts with some obm features (vacation, forwarding, etc)
• bugfixes
• composer. this part of minig really needs some love. Even more when you see what google wave can do (no, the plan is not to be able to insert a "poll widget" in your signature)

PS : MiniG is free software, released as GPLv3. No paying-customers-only-crap-for-the-others version.

Random links of the weekend

Still exploring what Cassandra could provide as a clustered mail storage.

Easy read on cassandra data model : http://arin.me/code/wtf-is-a-supercolumn-cassandra-data-model

Using this article, I drafted an early data model :

Conversations : { // CF
    INBOX/123-1 : { // Row key is the conv id
 timeuuid_1 : messageId1
 timeuuid_2 : messageId2
    }      
    INBOX/123-2 : { // Row key is the conv id
         timeuuid_3 : messageId3
         timeuuid_4 : messageId4
    }       
}

Folders : { // CF
     INBOX : {
           timeuuid_1 : INBOX/123-1
           timeuuid_3 : INBOX/123-2
     }
}

Messages : { // CF
   messageId1 : { // row key
       subject: bla bla
     to: xx
     cc: yy
     bcc: zz
     preview : hello world
                   body: hello world ! qdqsd
                   hasAttach: true
                   hasInvitation: false
                   attachments: attach_123, attach_456
   }
}

Attachments : {
    attch_123 : {
        filename: fileName1
 size: 123456
        content: xazeqsdqdqsd_base_64
    }
}

Article on lucene/solr + cassandra : http://nicklothian.com/blog/2009/10/27/solr-cassandra-solandra/

As I wanted to store my tests on those technologies somewhere, I created the BigG project on googlecode.