Thursday, July 30, 2009

Deliver MiniG on cd-rom ?

The problem

MiniG interface is built with GWT, that's common knowledge. GWT compiles java to javascript and generates permutations.

A permutation is a static javascript compilation unit for a given browser and a given locale. MiniG supports 2 locales (french & english). GWT 1.7 supports six browser flavours, so the compiler generates 12 javascript files (all the ABCD123.cache.html in the gwt build).

When a GWT application is loaded, a bootstrap file runs (the minig.nocache.js). This file runs checks on the user browser and determines which permutation should be used. The bootstrap javascript is only 6.2K and is similar for allmost all gwt applications. Once the correct permutation is determined, minig.nocache.js instructs the browser to load the abcd123.cache.html for your browser.

Most of the time, this file is already cached by your browser, but when a new version is deployed... the complete minig interface is downloaded.

With GWT compiler 1.7.0, the minig interface stands between 472K and 481K. Now you obviously begin to understand why this blog post is named "Deliver on cd-rom ?".

Ok, 500K is... half a second on a low cost end-user internet connection. As a french provider commercial says : "il a free, il a tout compris". But sometime, users do not even have an 8MB dsl connection. That's the case for the MiniG install running on my work mailbox. The 500K are a pain in the (insert joke about the G in MiniG) to download, as our work connection often uploads at 10K/s during work hours. This means that you cannot log into MiniG in less than 50 seconds.

Apart from getting correct network connectivity, I must have a look at what can be done on the MiniG side.

If you got me correctly, the localized messages are copied into javascript permutations. So if I add a 20K piece of text for online help in MiniG, the download size will go from 500K to 520K.

How can this be solved ?

Solution 1

Here is an easy one : a2enmod deflate

Run this command on the reverse proxy serving minig files and restart apache : gzip compression on static data files.

Before :

-rw-r--r-- 1 tom tom 479K 2009-07-30 00:41 DCEC18F66460AB2A6CB58F4EF8A75E70.cache.html


After :

-rw-r--r-- 1 tom tom 139K 2009-07-30 00:41 DCEC18F66460AB2A6CB58F4EF8A75E70.cache.html.gz


Just adding an apache module turned a 479K download to a 139K download. If the previous one was a 50sec download with our crappy bandwidth, this one is now 14sec. Pretty nice ?

Other solutions

MiniG is a pretty big application, but imagine something like Google Wave. You can guess they had to solve the same problem. They talked about it and provided solutions at the last Google I/O conference.

Let's summarize (or emphasize...) what the upcoming solutions are.

Compiler cut points

Even if minig never loads a page, all screens are not visible at the same time. There's no need to load the composer code when the frontpage only shows a list of message.

before :

addTab(Webmail.COMPOSER, new MailComposer());


after :

GWT.runAsync(new RunAsyncCallback() {
public void onSuccess() {
addTab(Webmail.COMPOSER, new MailComposer());
}
});


This tells the compiler that adding the composer to the UI is not "top priority". If this is the first usage of the MailComposer class (in MiniG case it is), all its code will be generated in a separate javascript file. The composer code will be loaded later, when the INBOX was already shown to the user.

Same applies for the progress bar shown on minig loading. We show the progress bar and can "runAsync" the XmlHttpRequest that fetches user settings on startup. Using this gives us :

  • Instant progress bar display

  • The main download is done with the progress bar on screen

  • The 500K become 50K and then 450K



With some runAsync added, I managed to display inbox with only a 170k download.

Class metadata

GWT obfuscates a lot of things (renames all your variables to a, b, c, z, a1, z26 instead of their original names). But it keeps the name of your java classes in the code. The -XdisableClassMetadata forces obfuscation of that to. MiniG uses fr.aliasource.webmail.*. This compile option gives more than 5% improvement. Easy win :-)

SOYC : Story Of Your Compile

This is a reporting tool designed to tell you which parts of your code contributes to the final JS size. This is really helpful to understand where runAsync and code refactoring are most needed.

Conclusion

All those improvements are available in GWT compiler trunk. I tried them all on MiniG code. It is pretty unstable right now, but the solutions are coming. Situation is not critical (500K for a complete application is nothing), but as MiniG will gain features, we had to investigate this. Other optimizations are coming in the compiler, Google I/O presentation is a very interesting read.

Thursday, July 16, 2009

Mobile device provisionning with o-push

I just commited mobile device provisionning to the o-push ActiveSync server. We tested it with an iPhone 3.0 and it allows pretty cool things.

What is mobile device provisionning :

When a device starts its dialog with the ActiveSync server, it sends an header "X-MS-PolicyKey : 0". The server can either ignore it or choose to enforce a security/enterprise policy. It (the server) does so by refusing the sync request and sending back a 449 http error code.

Of course you need a device that supports security policies, but the iPhone 3.0 does.

The device, when receiving the 449 error, starts a provisionning dialog :


  • it asks for the server policy

  • the server returns one with a temporary policy id

  • the device acknowledges the id by sending it back to server

  • the server respond with a "final policy id" for the device

  • the device will now use the "X-MS-PolicyKey: <final policy id>" in all its future communications

  • normal sync dialog will occur, except if the server administrator changes the policy (the servers sends another 449 error if the "final policy id" is no longer valid)



What can the administrator enforce in the policy :

Well, pretty much everything, but as the policy document is pretty explicit, lets reproduce it here :

<EASProvisionDoc>
<DevicePasswordEnabled>0</DevicePasswordEnabled>
<AlphanumericDevicePasswordRequired>0</AlphanumericDevicePasswordRequired>
<PasswordRecoveryEnabled>0</PasswordRecoveryEnabled>
<DeviceEncryptionEnabled>0</DeviceEncryptionEnabled>
<AttachmentsEnabled>1</AttachmentsEnabled>
<MinDevicePasswordLength>4</MinDevicePasswordLength>
<MaxInactivityTimeDeviceLock>900</MaxInactivityTimeDeviceLock>
<MaxDevicePasswordFailedAttempts>8</MaxDevicePasswordFailedAttempts>
<MaxAttachmentSize/>
<AllowSimpleDevicePassword>1</AllowSimpleDevicePassword>
<DevicePasswordExpiration/>
<DevicePasswordHistory>0</DevicePasswordHistory>
<AllowStorageCard>1</AllowStorageCard>
<AllowCamera>1</AllowCamera>
<RequireDeviceEncryption>0</RequireDeviceEncryption>
<AllowUnsignedApplications>1</AllowUnsignedApplications>
<AllowUnsignedInstallationPackages>1</AllowUnsignedInstallationPackages>
<MinDevicePasswordComplexCharacters>3</MinDevicePasswordComplexCharacters>
<AllowWiFi>1</AllowWiFi>
<AllowTextMessaging>1</AllowTextMessaging>
<AllowPOPIMAPEmail>1</AllowPOPIMAPEmail>
<AllowBluetooth>2</AllowBluetooth>
<AllowIrDA>1</AllowIrDA>
<RequireManualSyncWhenRoaming>0</RequireManualSyncWhenRoaming>
<AllowDesktopSync>1</AllowDesktopSync>
<MaxCalendarAgeFilter>0</MaxCalendarAgeFilter>
<AllowHTMLEmail>1</AllowHTMLEmail>
<MaxEmailAgeFilter>0</MaxEmailAgeFilter>
<MaxEmailBodyTruncationSize>-1</MaxEmailBodyTruncationSize>
<MaxEmailHTMLBodyTruncationSize>-1</MaxEmailHTMLBodyTruncationSize>
<RequireSignedSMIMEMessages>0</RequireSignedSMIMEMessages>
<RequireEncryptedSMIMEMessages>0</RequireEncryptedSMIMEMessages>
<RequireSignedSMIMEAlgorithm>0</RequireSignedSMIMEAlgorithm>
<RequireEncryptionSMIMEAlgorithm>0</RequireEncryptionSMIMEAlgorithm>
<AllowSMIMEEncryptionAlgorithmNegotiation>2</AllowSMIMEEncryptionAlgorithmNegotiation>
<AllowSMIMESoftCerts>1</AllowSMIMESoftCerts>
<AllowBrowser>1</AllowBrowser>
<AllowConsumerEmail>1</AllowConsumerEmail>
<AllowRemoteDesktop>1</AllowRemoteDesktop>
<AllowInternetSharing>1</AllowInternetSharing>
<UnapprovedInROMApplicationList/>
<ApprovedApplicationList/>
</EASProvisionDoc>

As you see, you can completly "enterprise lock" the phone :


  • You can lock wifi, infrared or bluetooth usage

  • No photos (we tested this one, the iPhone supports it)

  • No text messaging

  • No phone unlock without password (we tested this one too on the iPhone), with a strict password policy

  • No mail/groupware account except the enterprise/ActiveSync one

  • Only allow sending SMIME signed emails

  • Force storage of email/groupware data in an encrypted storage

  • etc



Right now, the policy sent to the phones is harcoded in the o-push server, but will probably develop an interface to manage those policies ;-)

Friday, July 03, 2009

We had it first ;-)

MiniG had drag & drop of conversations into folder before GMail. But my implementation clearly sucked (which was not a problem as I find drag&drop an inefficient way to move mail around).

Gmail did it better, so I mimiced their user interface for dnd.

No screenshot, but a small video ;-)