Executing JS Code when receiving an Email with a specific header set

I spent the last two days figuring out exactly how the Thunderbird new email notifications work. My goal was to listen in new Emails for a specific header and then run a specific task. Before I go into details a general note on this first. Please be careful when doing this and be even more careful when passing that header value to your code. The code processing the new mail is privileged and the header value should be treated as unsafe. In the best case this could cause a DoS type attack on your extension, in the worst case attackers could use it to control the user’s Thunderbird.

When I started on this I though it would be easy, a matter of hours. I started with a simple message filter listener, which is what I found on the internet first:

Components.utils.import("resource://gre/modules/mailServices.js");
Components.utils.import("resource://gre/modules/iteratorUtils.jsm");
Components.utils.import("resource://gre/modules/Services.jsm");

const CUSTOM_HEADER_NAME = "x-my-custom-header";

let listener = {
  QueryInterface: XPCOMUtils.generateQI([Components.interfaces.nsIMsgFolderListener]),
  msgsClassified: function(aMsgs, aJunkProcessed, aTraitProcessed) {
    for each (let msg in fixIterator(aMsgs.enumerate(), Components.interfaces.nsIMsgDBHdr))) {
      if (msg.getProperty(CUSTOM_HEADER_NAME)) {
        // do something
      }
    }
  }
  // other nsIMsgFolderListener methods ommitted for brevity
});

function setupListener(listener) {
  const mfn = Components.interfaces.nsIMsgFolderNotificationService;
  MailServices.mfn.addListener(listener, mfn.msgsClassified);
}

The problem that took me a while to figure out is that the “do something” block is never called. Until I found the actual soltuion, I tried different things like using a pure nsIFolderListener or creating a temporary message filter that takes care. I was convinced that the header values were just not parsed into the nsIMsgDBHdr yet. The reason for this was a very misleading comment in the nsIMsgDBHdr interface. But without some extra code, x-my-custom-header is never added to the properties of the nsIMsgDBHdr. This is done on purpose to keep the database small.

To make it work, we just need to set an extra pref:

let customHeaders = Services.prefs.getCharPref("mailnews.customDBHeaders").split(" ");
if (customHeaders.indexOf(CUSTOM_HEADER_NAME) < 0) {
  customHeaders.push(CUSTOM_HEADER_NAME);
  Services.prefs.setCharPref("mailnews.customDBHeaders", customHeaders.join(" ").trim());
}

I hope this saves you a few hours of digging. I certainly would have hoped to find a guide like this, but then again maybe I was looking at the wrong place with the wrong terms.

Disable Specific Error Messages in the Closure Linter (gjslint, fixjsstyle)

I have recently been working on ical.js, a library to parse rfc5545 calendar data. James Lal has been helping me outand has suggested to use a Javascript linter on a regular basis. We have agreed on using the Google Closure Linter.

One thing I didn’t like about the Closure Linter is that error messages are not configurable. There are some cases where I think 80 characters per line are just wrong, for example a for() statement that has 81 characters. Would you really want to break lines for just one character? What about URLs in comments that are just longer than 80 characters? Also, I’m not a fan of single quotes.

On the internet I found the Closure Linter’s feature request issue and a few discussion topics. While the feature request links a patch, I wanted a solution that does not modify the Closure Linter so new contributors don’t have to go around patching things just to check and fix the style. To do so I created a wrapper script around gjslint and fixjsstyle, as well as a common file for defining the errors to ignore.

First of all, the common file. Name it something like myerrorrules.py:

from closure_linter import errors
from closure_linter import errorrules

OriginalShouldReportError = None

def InjectErrorReporter():
 global OriginalShouldReportError
 OriginalShouldReportError = errorrules.ShouldReportError
 errorrules.ShouldReportError = MyShouldReportError

def MyShouldReportError(error):
 global OriginalShouldReportError
 return error not in (
   errors.UNNECESSARY_DOUBLE_QUOTED_STRING,
   errors.LINE_TOO_LONG,
 ) and OriginalShouldReportError(error)

Then you need a small wrapper around gjslint.py:

from closure_linter import gjslint
import myerrorrules

if __name__ == '__main__':
  myerrorrules.InjectErrorReporter()
  gjslint.main()

And quite the same for fixjsstyle.py:

from closure_linter import fixjsstyle
import myerrorrules

if __name__ == '__main__':
  myerrorrules.InjectErrorReporter()
  fixjsstyle.main()

That is all! Now you just have to start gjslint and fixjsstyle through these wrappers and you will have all the errors ignored that you like.

Über die Zukunft von Thunderbird

Tristan Nitot hatte Gelegenheit mit JB Piacentino ein Interview zur Zukunft von Mozilla Thunderbird zu führen.
Hier eine Übersetzung dieses Gespräches:
Tristan: Hi Jb, ich habe im Internet gelesen, dass Thunderbird Tot sei. Ist das wahr oder ist das Ende von Thunderbird übertrieben?

Jb: Nein, Thunderbird ist nicht am Ende. Wir ändern die Art und Weise, wie wir neue Funktionen für Thunderbird entwicklen. Nichts wird sich für den einzelnen Privatanwender oder Nutzer in Unternehmen/Organisationen ändern. Mozilla wird Thunderbird weiter unterstützen.

Tristan: Es wird also Thunderbird 14 geben?

Jb: Ja, natürlich! Wir haben einen soliden Plan, um Thunderbird bis Ende 2013 zu unterstützen und sind ebenso in der Planungsphase, was die anschließende Unterstützung angeht.

Tristan: Und was passiert bis dahin?

Jb: Thunderbird 15, 16 und 17 werden demnächst herausgebracht. Für Thunderbird 15 gibt es bereits eine Alpha-Version und auch nach Version 17 werden weiterhin neue Versionen herausgebracht. Wir werden coole, neue Innovationen sehen, wie zum Beispiel Thunderbird Chat (Instant Messaging), weitere Anbieter zum Anmelden neuer E-Mail-Adressen und für Thunderbird Filelink, sowie neue Funktionen, die vollständlig von der Community beigesteuert wurden. Zusätzlich wird Mozilla auch weiterhin die Beiträge aus der Community beachten und mit in das Produkt einfließen lassen.

Tristan: Weisst du, wer die Thunderbird-Benutzer sind und was sie wollen?

Jb: Wir haben eine Mischung aus Privatanwendern und Firmenkunden. Unter den über 20 Millionen Benutzern gibt es viele, die Thunderbird großflächig einsetzen, wie zum Beispiel Firmen, Regierungs- und Bildungseinrichtungen. Wir müssen dabei das Gleichgewicht wahren, um sowohl Privatanwender als auch Firmenkunden zufriedenzustellen.

Eine Gemeinsamkeit unter allen Benutzern ist zu aller erst die Forderung nach einem sehr stabilen und sicheren E-Mail-Produkt. Beispielsweise können in sehr großen Firmen Veränderungen an den vorhandenen Funktionen alle möglichen Probleme hervorrufen, angefangen mit Problemen bei der Zertifizierung, der Verteilung auf allen Systemen bis hin zur Schulung der Benutzer. Ebenso haben unsere Privatanwender kein großes Verlangen nach neuen Innovationen.

Dennoch glauben wir, dass das Erlebnis für den Benutzer verbessert werden kann, zum Beispiel durch die Einbindung zusätzlicher Web-Dienste ähnlich wie wir es für Thunderbird Filelink gemacht haben. Es ist schwer das richtige Gleichgewicht zu finden, aber im Moment glauben wir, dass unsere Benutzer mit den aktuellen Funktionen zufrieden sind und viel mehr Wert auf ein stabiles Produkt legen.

Tristan: Ich sehe eine große Kluft zwischen den Erwartungen der Thunderbird-Benutzer und Mozillas Schwerpunktsetzung auf das Internet und die Mobilgeräte-Sparte mit Firefox für Android und Firefox OS (aka Boot to Gecko). Gibt es einen Zusammenhang mit dieser Entscheidung über Thunderbird?

Jb: Mozilla setzt seinen Schwerpunkt sicherlich auf das Internet als Platform, da wir an dieser Stelle als Gemeinschaft mit der Community einen Unterschied machen können. Mobile Umgebungen sind weitgehend geschlossen. Wir glauben, wenn wir die Werte und Technologien hinter dem Internet auf die mobilen Geräte bringen können, dass wir dadurch einen Mehrwert für die Benutzer mobiler Geräte schaffen können. Thunderbird bleibt ein sehr gutes E-Mail-Programm für den Desktop-Bereich, was sich aber nicht wirklich mit der Strategie von Mozilla vereinbaren lässt.

Tristan: Lass mich einmal den Teufel an die Wand malen: Warum stellt Mozilla die Unterstützung für Thunderbird nicht vollständig ein?

Jb: Ich glaube, dass der Plan, den wir vorschlagen, zeigt, dass Mozilla trotz dieser verzwickten Lage zu seinem Wort steht: Wir werden Thunderbird nicht einstellen. Im Gegenteil, wir schmieden die Vorraussetzungen, damit das Produkt weiterhin sicher und stabil ist und Beiträge aus der Community integriert werden können. Um genau zu sein, ermutigt uns die frühe Resonanz aus der Community und ich bin mir sicher, es werden interessante Entwicklungen von freiwilligen Mitwirkenden und auch Firmen kommen.

Weiterhin gibt es eine Reihe von Open-Source-Beratungsfirmen, die ihre Firmenkunden beim Einsatz von Thunderbird unterstützen, und ich bin mir sicher, sie werden mithelfen, um den Wünschen ihrer Kunden nachzukommen und somit Thunderbird zu verbessern.

Tristan: Werden auch in Zukunft Mitarbeiter von Mozilla an Thunderbird arbeiten? Wie lange wird das noch der Fall sein?

Jb: Ja, Mozilla wird auch weiterhin Thunderbird mit bezahlten Mitarbeitern unterstützen, obwohl diese möglicherweise nur Teilzeit am Projekt arbeiten werden.

Tristan: Wenn du einen Aufruf an unsere Leser richten könntest, was würdest du ihnen mitgeben?
Jb: Das kommt wirklich auf deine Interessen und Fähigkeiten an:

• Hast du Interesse, über die Zukunft der Verwaltung von Thunderbird als Projekt zu disktuieren, solltest du vielleicht die Wikiseite zur Verwaltung von Thunderbird besuchen und dort den Anweisungen folgen. Wir planen, im Laufe des Sommers Fortschritte zu machen und das neue Verwaltungsmodell anfang September fertigzustellen.
• Möchtest du hingegen an der Entwickelung von Thunderbird teilnehmen, schau doch einfach mal auf dem Entwicklerportal vorbei . Ein paar Interessante Vorschläge für Innovationen findest du auf der Seite UpForGrabs. Zusätzlich braucht die Thunderbird-Hilfe immer neue Mitarbeiter, so wie die Qualitätssicherung.

Tristan: Jb, vielen Dank für deine Zeit. Da ich selbst auch Thunderbird-Benutzer bin, freue ich mich sehr, diese Information aus erster Hand zu bekommen. Ich freue mich sehr zu sehen, wie sich Thunderbird in Zukunft entwickelt.

Jb: Danke!

Keyword Bookmarks with Multiple Parameters

If you have ever looked at the properties of a bookmark, you might have seen the “Keyword” field. This is a very powerful field and makes your life easier. If you already use this field, you may have been disappointed that you cannot add multiple parameters to a keyword, since there is only one placeholder “%s”.

Inspired by the FileIt page, I wanted to create a bookmark keyword that allows me to pass in product and component for filing a new bug. Javascript to the rescue! All you need to do is to take the following code snippet and create a new bookmark as described on the Bookmark Keywords page. You might want to compress the bookmarklet using a site like jscompress.com.

(function(){
  /* First, split the passed string. You may use any delimiter you want, I chose "/" */
  var pc = "%s".split("/");
  /* Now set the base URL for the target to visit */
  var target = "https://bugzilla.mozilla.org/enter_bug.cgi?rep_platform=All&op_sys=All";
  /* If at least one parameter was passed, then add it as the product */
  if (pc.length) target += "&product=" + pc[0].charAt(0).toUpperCase() + pc[0].substr(1);
  /* If at least two parameters were supplied, then add the second as the component */
  if (pc.length > 1) target += "&component=" + pc[1].charAt(0).toUpperCase() + pc[1].substr(1);
  /* Now go to the target location */
  document.location.href = target;
})();

Afterwards, you can directly use the bookmark using the address bar:

Show a Calendar from FHEM Data

I recently bought a new router (AVM Fritzbox 7390) since my older router doesn’t support 802.11n and didn’t have an integrated dsl modem. Shortly after, I visited the tradeshow Cebit where home automation was a big topic. I’m always trying to find new ways to make life easier and home automation has been a long standing dream of mine. Visiting AVM’s booth I noticed they are also pushing towards giving their routers home automation capabilities. In the meantime, their labs firmware incorporates FHEM, an open source home automation server based on perl. Immediately I ordered the required USB device to connect wireless home automation devices and a control unit for a radiator.

Installing went fine, I was able to reach fhem via http://fritz.box:8083/fhem/. While it was pretty cool to be able to control my radiator via mobile phone, this does not relieve me of needing to control my heating schedule. Ultimately, I want my radiators to know if I am home or not, when I come back and maybe even what room I am in. This requires a lot of work so I decided to start small.

I use a FHT80b to control my radiator, which supports automatically setting the temperature in two intervals per weekday. I then created a module for fhem which reads this schedule and transforms it into recurring events in the ical (rfc5545) format. This is readable by your favorite calendar client, mine is obviously the Lightning extension to Thunderbird. The first version of this module is merely read-only, but support for actually changing the schedule is planned.

1. Get the module from my github repository. You should save it in your fhem module directory, i.e /usr/share/fhem/FHEM/. If you are also using a Fritzbox it will be here.
2. Add the line define ICS FHEMICS to your fhem.cfg
3. Make fhem reload your config (i.e kill -HUP $pid, telnet host 7072 and rereadconfig, or restart your Fritzbox)
4. Now you should be able to subscribe to http://fhemhost:8083/fhem/fhem.ics
5. The FHT80b doesn’t advertise the weekly schedule by itself, so it needs to be initialized. Use the control unit and cycle through each day’s heating schedule.
6. It takes a while until all commands are transmitted (1% bandwith usage regulation enforced by most 868Mhz devices)

If you’d like more options, please take a look at the module file itself. If you are not seeing events, the following commands are available via the telnet interface of fhem:

telnet fhemhost 7072
list FHT_1234    # lists the readings of your FHT device called FHT_1234.
                 # If you are missing readings like "mon-from1" or "fri-to2"
                 # then the FHT has not transmitted its schedule.
list             # Shows all defined devices, make sure there is a FHEMICS device
list ICS         # Show state information about the ICS device.
get ICS          # Retrieves the ics stream. This should contain more than 2 lines.

If you find a bug, please report at github.

How to Triage Mozilla Bugs

Once in a while, I get emails from users that are willing to help out with the Calendar Project. Often, these users are not developers but would still like to help. If you believe it or not, there is nothing easier than that! All you need is a cup of coffee (or any mind-soothing drink), a Bugzilla account and a nightly build of the Mozilla product of your choice. I’ll assume you’ll be using Thunderbird and Lightning for now :-)

If you’d like more information on triaging bugs, please also check out the QMO’s Unconfirmed Bug Triage Project.

Getting The Prerequisites

Coffee should be easy. If you don’t have one at home, I’m sure you’ll find one nearby.

Now you need to create a Bugzilla account, unless of course you already have one. Its a very quick signup process, but be sure to pick an email address that you don’t mind sharing with the public, it will be shown on each bug you comment on.

Next, download the latest nightly of Lightning. To use it, you must download a fitting Thunderbird version as mentioned on the Lightning nightly page. Specifically for Lightning  you should use the Lightning version that fits to the next major release of Thunderbird. For example, at the time of writing this post the next major Thunderbird version to be released is Thunderbird 3.1, so you want to download Lightning 1.0b2pre. Make sure you create a new profile for testing, I’ll explain more below.

In the next part of this post, I’ll explain what there is to triage. It makes sense to start small, just asking for more information can be very helpful for us. If you want do dive in deeper, then you may want to test on your own. Later on, I’ll tell you how to stay up to date with new bugs filed so you can quickly assist bug reporters and how to make life easier when processing bugs.

Asking for more Information

This is the easiest step you can take. To do this, you don’t even need a nightly version yourself, although it might be helpful for easy-to-reproduce bugs if you want to check yourself.

It doesn’t work! Help!

In the worst case, people file bugs like “Your application doesn’t work. Its very obvious. Please tell me what to do”. Now what should we do with that! What version is the reporter using? What operating system? Most importantly: what exactly isn’t working? If you see a bug like this, you need to be asking exactly these questions. If the bug is about Lightning, then we need to know:

• What Thunderbird version is the user using, on which operating system?
• What Lightning version does he or she have installed?
• If more extensions are involved (for example, the Provider for Google Calendar), which extension version is being used?

Some reporters turn out to be using older versions of the Product. Users tend to use stable versions of software and if something fails they report a bug. While this might work for Products with many developers, this doesn’t work for smaller projects like Lightning. Others use the latest version in their Linux distribution, which may turn out to be over 2 years old. In any case, it helps to ask: “Does this still happen with the latest Lightning nightly <version> builds? You can find them at <url> and should use Thunderbird version <version> “. This also makes sense if the last time the bug was confirmed has been quite a while back (i.e happened in the previous Lightning version).

Debug, what!? Messages? Where!?

Also, reporters usually don’t know about the extra debugging preferences, or where to look when something is going wrong. For Lightning, there are a few preferences that can be changed to give more debugging output. Its a good idea to tell the user how to find the advanced config editor. Then, you can tell them what to change to get more debugging info. For Lightning, calendar.debug.log and calendar.debug.log.verbose need to be set to true. Although not always required, it might also make sense to have the user restart the application.

Man…I’ve seen this before!

The more bugs you visit, the better picture you get of what bugs are filed. If you see a bug report that you think you’ve seen before, then search for that other one. Pick the one that has less information on it and mark it as a DUPLICATE of the other one.

Are you still out there?

If you or someone else has already asked the reporter to provide additional info and no response has been made for a couple of months, the next thing you can do is mark the bug INCOMPLETE, telling the reporter why: “No response for a while. I’m closing this bug INCOMPLETE for now, please feel free to reopen if you can still reproduce or have an answer to the above questions”. This helps keep the number of open bugs down and makes it easier to find out what issues need to be fixed for the next release.

Testing unconfirmed or new bugs

This step requires you to download a nightly build of Lightning. Assuming you are using Thunderbird/Lightning for your normal Emails and Calendar, you want to create a new profile dedicated to testing, since you might be breaking things with a nightly build. It makes sense to create a desktop shortcut or alias to directly start Thundebird with your testing profile. Adding the  “-no-remote” command line flag to the alias or shortcut allows you to run it alongside your normal Thunderbird. Also, you’ll want to download the Lightning Nightly Updater extension, which allows you to stay up to date with the nightly builds.

Now that you have things set up, you can start testing bugs. In most bugs, there will be a set of “Steps to Reproduce” in the first few comments. Just repeat exactly these steps and see if the error happens to you too. If so, leave a comment on the bug which Lightning Nightly version you are using and that the error is still happening. If not, try to be creative. Think about what the user might have done unknowingly, or change the steps a bit to see if you can make the error occur. To make it easier for the developers, you can document what you’ve changed on the bug.

If you can’t reproduce, go ahead and tell the user too. If its an obvious case, you can mark the bug WORKSFORME, otherwise you might want to ask the user additional questions, see “Asking for more information” above.

Verifying fixed bugs

These are bugs that have been marked RESOLVED and FIXED. Usually a patch has been checked in so the bug was marked FIXED. To identify if the bug is really fixed, you can test again using the original steps to reproduce. If the bug is really fixed, set it to VERIFIED. Otherwise you have two options. If the bug was just fixed recently, you can just set the bug to REOPENED and tell us what fails. This will ensure the developer takes a second look. The other option you have is to file a new bug. Note which bug you were testing and what goes wrong. This is also the better option if you notice that what fails is not directly related to the bug.

Now where do I find those bugs?

There are two different types of bugs that need triaging. The most important are UNCONFIRMED bugs. These are bugs that users have filed, but have not been confirmed by developers or bug triagers like you. We need to find out which of these are really bugs and which ones are support issues. About a week ago, we had almost 500 unconfirmed bugs. These are the bugs that can easily be processed by “asking for more information”, but are more quickly resolved if you test them yourself.

The next category is NEW bugs. These are bugs that have been confirmed before. If this has been a longer while back, then it might be that it works by now, maybe it was fixed by a different bug. These are not so important to triage, but if you see someone asking if the bug still exists, you might want to check if this is the case.

To find the bugs, use the Bugzilla search. For this task it makes sense to use the advanced search interface. There you can set the Product to Calendar, select only NEW or UNCONFIRMED bugs, and make sure nothing is selected in the “Resolution” box (use Ctrl+Click to unselect). You can also use these quick links for the queries: NEW and UNCONFIRMED.

Also, johnath made a very nice video blog post introducing you to Bugzilla, its worth listening to.

But I’m now allowed to!

When creating an account, you don’t directly have all privileges. You will be able to file bugs and comment on them, but actually setting other fields like the bug status needs some extra permission bits. I’d suggest to start out with just adding some comments to existing bugs as noted above. Once you’ve made a few comments and we see that you want to help out, contact me or others from the calendar team, asking for “canconfirm” or “editbugs” priviledges. The first lets you only set bugs from UNCONFIRMED to NEW, the second lets you change any aspect of the bug. Please note that to achieve editbugs, we need to have the feeling we can trust you. Of course we can undo bug changes, but you’ll probably understand that if an evil-doer comes along and just re-opens all calendar bugs, this is quite annoying and causes unneeded work.

I hope this interests you in helping out with a Mozilla product, and personally I hope it even inspires you to help out with the Mozilla Calendar Project. I’m looking forward to hearing from you!

All this typing is getting very annoying!

When triaging bugs, you end up typing the same stuff over and over again. Still happening? What version? Please retest! Your questions get shorter and shorter, which makes it hard for new users to understand what you really want. To help with this, I’ve created a few jetpack extension which makes work much easier. First of all, you need to install the jetpack extension to Firefox. Now you can visit my jetpacks page. You should install “Bugzilla Comments Helper” and “Close on Submit”. The first one gives you a set of predefined comments you can easily select from a dropdown, while the second adds a “Commit and Close” button, that closes the tab after your changes have been made.

Welcome to my world!

Hello reader. You are probably here because you are interested in what I am up to in the Mozilla world, more specifically how the Mozilla Calendar Project is doing. First of all I’d like to tell you a bit about myself so you can get a feeling for what to expect here.

As children often do, I’ve had a number of technical ideas in my childhood. The most related one is my wish to make organizing life as easy as possible. I wanted to have a solution where I can manage my time anywhere I am — be it via cell phone, PC or maybe even something exotic like looking in the mirror. Back then it was very hard to achieve this, but nowadays there are iPhones, laptops (that fit my budget) and mobile internet is probably getting cheaper as we speak.

After having tried Sunbird for a brief moment before, I read about a calendar testday with prizes (yes, that always works ;-). I decided to show up. After the testday was over I was one of the top testers for that day and received a prize. Afterwards I took a little closer look into the application and quickly found things I though I’d like to improve, and so I did.

After working on the Provider for Google Calendar, noticed there were still certain aspects of the Provider that I couldn’t synchronize with the application. This got me into fixing core calendar code, which soon got me into getting to know the team, and later on I was encouraged to apply for a job at Sun.

I’m having a great time at Sun and just recently I was appointed the new lead for the Calendar Project. Since especially in my role it is important to keep in touch with the community and users, I’ve decided to start this blog. I plan to introduce you to some new ideas and vision I’ve been having that might just make your life a bit easier and could bring me closer to my childhood dreams. I’ll leave the more technical posts to the Calendar Weblog, but once in a while you might also hear of an idea for an extension I have (hands on, implementation is up to you ;-) or I’ll introduce a new extension I have created.

Since the success of the product is very dependent on what those who use it think, I’d like to encourage you to tell me whats on your mind regarding Calendar. Tell me about your childhood dreams and what you think is the most important aspect of calendaring. I can’t promise that I’ll sit down right away and add your favorite feature, but your feedback will help me form the future of the Calendar Project.

So go ahead, leave a comment, write me an email, ask me on #calendar. The future of Calendar is waiting for you — get involved!