Thunderbird Developer Tools Wrapup

In my earlier two posts I showed you my work on the Google Summer of Code 2013 Project to bring the Developer Tools to Thunderbird. The method for doing so is making use of Firefox’s remote debugging protocol, allowing to use the web developer tools available in Firefox to manipulate Thunderbird. More details are covered in the earlier posts. The Summer of Code has now come to an end, so I would like to tell you about my progress, the goals I’ve reached and those my mentor and I have decided are out of scope.

First of all, let me tell you about the remaining features I have implemented since the last post. One of the features is support for the remote inspector. This was pretty easy to do, although support for it is still preliminary. There are still a few quirks, but it’s mostly usable.  You can see here I’ve changed an attribute value:

Devtools Inspector in Action

Next up is support for scratchpad, which is still work in progress on the client side but is almost complete. Here is a screenshot:

Devtools Scratchpad in Action

Also, there is the app manager. This is, as far as I’ve understood, still in a beta stadium and aims to be a central place for managing remote devices. Thunderbird is one of these “remote devices”. The app manager shows some information about Thunderbird like its resolution and allows making screenshots:

app-manager

Finally, I’ve made progress packaging the glue code required for the debugger server into an extension. This is mostly a build system change that allows packaging the code as a restartless addon which I can distribute on addons.mozilla.org. The extension has an option dialog which allows starting and stopping the remote connection. From within Thunderbird this extension is not needed, but it is helpful for other applications based on the Mozilla Platform, like those based on XULRunner. I will post an update when the extension is available on addons.mozilla.org. Here is a screenshot of the options dialog:

Devtools Server Extension Dialog in Action

In my original milestone planning there were a few features considered a bonus. Some of these were not completed. It turns out those extra features are a substantial amount of effort, possibly even worth their own Summer of Code Project.

The first of these two is adding a way to inspect IMAP connections in the network monitor. This requires providing a specific interface in the IMAP channel implementation which makes it possible to inspect the content even after the request has been sent. Also, it is needed to mimic certain aspects of a http channel, specifically the concept of request and response headers. In Thunderbird, the IMAP channel implementation is heavily cached. Hooking up the channel interface to the network monitor would cause display of cached requests as separate requests. Also, this would only fix it for IMAP connections. A better way would be to add a general mechanism in the Mozilla Platform to be able to inspect TCP connections. This requires some changes very deep down in the networking platform and is probably not easy to carry out. I have filed a bug to solve this, but it won’t be a part of the Summer of Code.

The next feature that is missing is gcli, also known as the “Developer Toolbar”, that small black bar you can open in the Web Developer menu that allows executing text commands. The problem here is that the code has a lot of dependencies to Firefox code. A substantial amount of files need to be moved from the directory containing Firefox code to a directory common to all XUL applications. Some files also need to be split up.  As this feature is a nice to have, but not considered vital functionality for Thunderbird Developer Tools, we have decided to postpone it. If you see a need for this feature, please leave a comment describing what you want to do with it. In the meanwhile you can follow the bug on bugzilla.

With this I have covered all the features I have proposed, I’d say it was a very successful Summer of Code. I have managed to reduce the code needed in Thunderbird and made most of the changes inside the Developer Tools code. This makes sure that support for Thunderbird will work in the future without needing updates. Also, new remote features will automatically work, given there is no Firefox specific code in them.

If you want to jump right in and try it, I have to appeal to your patience. Some of the patches required for functionality are still in review by my mentor and the Mozilla developer tools team. I will let you know once everything is in place. I’m pretty sure we will able to get all code into the tree by the end of the current cycle.

Executing JS Code when receiving an Email with a specific header set

I spent the last two days figuring out exactly how the Thunderbird new email notifications work. My goal was to listen in new Emails for a specific header and then run a specific task. Before I go into details a general note on this first. Please be careful when doing this and be even more careful when passing that header value to your code. The code processing the new mail is privileged and the header value should be treated as unsafe. In the best case this could cause a DoS type attack on your extension, in the worst case attackers could use it to control the user’s Thunderbird.

When I started on this I though it would be easy, a matter of hours. I started with a simple message filter listener, which is what I found on the internet first:

Components.utils.import("resource://gre/modules/mailServices.js");
Components.utils.import("resource://gre/modules/iteratorUtils.jsm");
Components.utils.import("resource://gre/modules/Services.jsm");

const CUSTOM_HEADER_NAME = "x-my-custom-header";

let listener = {
  QueryInterface: XPCOMUtils.generateQI([Components.interfaces.nsIMsgFolderListener]),
  msgsClassified: function(aMsgs, aJunkProcessed, aTraitProcessed) {
    for each (let msg in fixIterator(aMsgs.enumerate(), Components.interfaces.nsIMsgDBHdr))) {
      if (msg.getProperty(CUSTOM_HEADER_NAME)) {
        // do something
      }
    }
  }
  // other nsIMsgFolderListener methods ommitted for brevity
});

function setupListener(listener) {
  const mfn = Components.interfaces.nsIMsgFolderNotificationService;
  MailServices.mfn.addListener(listener, mfn.msgsClassified);
}

The problem that took me a while to figure out is that the “do something” block is never called. Until I found the actual soltuion, I tried different things like using a pure nsIFolderListener or creating a temporary message filter that takes care. I was convinced that the header values were just not parsed into the nsIMsgDBHdr yet. The reason for this was a very misleading comment in the nsIMsgDBHdr interface. But without some extra code, x-my-custom-header is never added to the properties of the nsIMsgDBHdr. This is done on purpose to keep the database small.

To make it work, we just need to set an extra pref:

let customHeaders = Services.prefs.getCharPref("mailnews.customDBHeaders").split(" ");
if (customHeaders.indexOf(CUSTOM_HEADER_NAME) < 0) {
  customHeaders.push(CUSTOM_HEADER_NAME);
  Services.prefs.setCharPref("mailnews.customDBHeaders", customHeaders.join(" ").trim());
}

I hope this saves you a few hours of digging. I certainly would have hoped to find a guide like this, but then again maybe I was looking at the wrong place with the wrong terms.