13May / 2008

Auto-Complete on Subjects

To make an initial start on our new search aspirations we need to begin testing and trying out some of our improved searching ideas.

Quick Search

Our first step in this direction is to add an auto-complete on subjects in the Quick Search entry. It’s important to get a lot of feedback on our search improvements so integrating our improvements with the current search is paramount. New behaviors need to be pushed out during our alpha releases to gain visibility and testing. This improvement doesn’t alter the current search behavior at all, everything is planned to act in parallel.

What will change?

The quick search will try to help you find message subjects by auto-completing on the subject name. The subjects it offers for auto-complete are searched from the available subjects in the folder Thunderbird displays in the current view.

For Example:

Type in the name “address” into the quick search entry
The auto-complete will give a list that match the word “address” somewhere in the subject
Selecting an item in the list will complete the whole subject name and search the message view for that name

Lo-fi mockup of Quick Search Auto-Complete on Subject

The auto-complete is only planned to work for subjects at the moment. Hopefully we’ll be able to start expanding this soon to include email addresses and names too. Once we have some experience with the auto-complete widget we can start expanding it’s scope a little.

Here’s the current design for the layout of the rich-item widget for matching message subjects.

There are a couple of other tweaks to the {meta} area that need to be improved. The light colors are a little hard to see and it might be better to brighten up the sender names. Also it could be good to add the date the message was sent. Perhaps like this:

$SENDER to $RECIPIENTS $TIME_AGO

ex: Bryan to you,david,gary 3 hours ago

What will stay the same?

The quick search should continue to search only in the current folder / view. This may change sometime in the future, but only when we have a better solution for that problem.

Also it will still work for searches that aren’t subjects, like senders. When you select a different search type, like “to or cc”, then it won’t continue to auto-complete on subjects; only when you select “subject” or “subject or sender” types.

When is this happening?

Everything is up in the air for discussion right now as we work through an incremental design that makes sense. The implementation pieces are going to come together soon when the new toolkit auto-complete widget from firefox is pulled into Thunderbird (see bug 370306 and bug 309081) and we figure out the best strategy for quickly searching a set of subjects from the current view.

Permalink 11 Comments

08May / 2008

Searching for a new find

It’s time to start looking into a new search method for Thunderbird. One of the major changes planned for Thunderbird is a new and improve search, but what does that mean?

What do we have?

First lets look at what we have for a search system. At a very simple level most search systems break down into two pieces, a search interface for filtering and a results interface for listing. Thunderbird does this in a couple places.

Quick Search

The quick search entry is always at the top right of the Thunderbird window and allows people to search over the current view. The results of a quick search fill into the current view, replacing whatever listing was previously shown.

The Quick Search defaults to searching only the Subject or Sender and will only search mail that Thunderbird has downloaded already. Messages that are not listed in the current view (like in another folder) will not be searched unless that folder is selected, otherwise a person needs to use the Advanced Search.

Advanced Search

Hidden under the Edit Menu and Find Sub-Menu is an advanced search dialog that can make use of the remote mail or news protocol to perform a full search instead of just a local search. The Search Messages dialog provides it’s own search interface as well as it’s own results view directly below the search. While the Search Messages dialog provides some more advanced search methods over the quick search it’s hard to find and difficult to use effectively.

The Search Messages dialog allows for complex search queries to be built with multiple search terms composed of a number of different field type selectors. The queries require a lot of input from the user because of the tight structure used to create them. The same search and results interface code is used for creating mail filters.

Edit -> Find -> Search Messages…

Advanced Search Dialog

What do we want?

I was lucky enough to chat with Andrew Gilmartin yesterday and he framed a future goal very well. “We’re not looking to make search an added feature box on the side of Thunderbird“, we’re looking to make search the definitive method for viewing mail.

What does “Search as the definitive viewing method for your mail” mean? That’s a good question and I’m not sure exactly what a good answer is yet. A search would help you find the message you’re looking for, and perhaps a search view never lets you lose that message in the first place. There’s a lot to explore.

Here are two important pieces of a search system and view that need to be examined and somehow exposed in the interface.

Search and Filter

An impediment of the current search system is requiring people to choose a search type (Subject or Sender) before they even enter any text. To help people hunt for the correct item you want to allow for starting their search very broad and then allow them to narrow down that broad search with filters like subject or sender.

The current search system has some speed issues that likely prevented a broad to filter system of searching to be implemented. The mail client Mail.app provides a decent filter bar when searching mail that allows people to see what the current filters are (folder, account) and change them.

Browse and Filter

The SEEK extension is an excellent example of how offering a system of browsing mail by grouped attributes from the start can help people find the item or group of items they were looking for. Instead of starting with a search term you give the person a list of attributes they might use to filter the list of messages.

An inspiring system for a similar searching, browsing, and filtering methods is things, you should try it if you haven’t already.

Getting What we Want

Moving towards a new search based paradigm will take some adventurous steps and it’s important not to disturb current usage while making those steps. Here are a number of changes to look at making.

Merging Search Interfaces

Each of the two current search interfaces provide some needed features and capabilities, however having two separate interfaces for searching is confusing and difficult to understand. We need to combine the ability to do a quick search with the ability to perform a full search into a single interface with an improved results view.

With a single search interface Thunderbird will be searching the local and remote mail (like IMAP) at the same time. However local results will be listing quickly and remote results will likely take a little more time. Both sets of results, local and remote, can be merged into the same search results view by showing local results instantly and filling in remote results as they arrive.

Offline Cached and Indexed Mail

In order to have a fast search system even while offline Thunderbird needs to do a much better job of caching and indexing mail as it encounters it. With new messages instantly cached and indexed they can be made available to search queries, filters, and views immediately.

This is an excellent time to start thinking about the data mining mail in a way that helps searching messages later. It’s also time to think about making the defaults tuned towards offline usage while still allowing people to control online / offline caching.

Auto Complete

With mail data indexed locally and quickly available Thunderbird should be able to provide a slick and fun auto-complete on search terms it knows about. Auto complete when searching for items you’re already aware exists helps with miss-spelling errors and more complete matching. The awesomebar shows how with just a little broken memory of a title or url you can easily find the page you saw once before.

Fetching Results

Our current drive is to investigate some indexing on messages (at least subjects), pull the new auto-complete into Thunderbird, and get a search bar using that fancy auto-complete on mail subjects and hopefully the addition of a couple more fun things. Leave some comments or jump on the newsgroup to participate.

A wire frame of a possible mail search auto-complete

Permalink 14 Comments

28Apr / 2008

Lightning Calendar Month View Hacks

With an hour or so of downtime during the Calendar Face 2 Face meeting this week I started a little CSS hacking of the Lightning Calendar extension. My goal was to change improve the visual effect of the month view into more of a heat map than a listing of all your events for the month. I believe the month view should only be answering the question, “What days am I busy, what days am I free?”, while avoiding answering every other possible question.

Old Calendar Month View

Here’s a screenshot of a day in the current month view. There are dark drop shadows for each event listed, as well a (visually alarming) alarm bell for every event that has an alarm. The borders are thick 2px lines with a dark color that overwhelms the board.

Effectively there is a lot of visual noise happening with all the different background and dark border shading that makes the view always appear somewhat cluttered.

Current Month Day View

Current Full Month View

Drawing the Eye

When you start shading the the areas of the calendar in a darker shade than they were before you can begin to see the visual effect created and where they eyes are drawn to. Places and things that aren’t as important as others.

Plus the extra visual information that is provided in some of the shading isn’t always necessary in the month view. For example, off month days are now indicated by a lighter shade of font color in the date text instead of a dark background color that makes them look important instead of additional.

Calendar Month Hack

In this screenshot of a single day in the new hacked month view you can see that the borders of days changed to a very light 1px color and the date numbering has been lighted up as well. I put in a slight background shade for the top border as well to give it some separation (not completely necessary). The colors and stacking of events in a day provides the necessary separation that the dark border colors were giving before.

In general everything has been flattened so the visual effect is about the colors of the calendar events drawing your eyes to the areas they exist in the most, while the empty parts of the calendar look much more open.

Designer code can be had at bug 430382 and you can see a full month view patch screenshot from there as well.

Next Steps

Start changing the timed events such that they don’t look so visually loud on the board. Time events, ones that range for a certain time during the day, should appear to be less than an all day event on the visual landscape.

Further Improvements

The text for calendar events is obviously much too verbose. You can see in the above screenshot that 08:30 AM could easily be changed into 8:30 and still provide the same information while taking less space.

Christian has much more work done for Improved Events and Tasks as well as others areas, all of which need to be merged.

Permalink 5 Comments

16Apr / 2008

Mac Address Book… try Thunderbird Nightly

The latest Thunderbird Trunk Builds are built to integrate into the Mac Address Book and we need some testing, so stop watching bug 203927 for builds.

1. Grab an Official Mac (Universal binary) from the nightly build

2. Enable the Address Book via the config editor (sorry you have to do this)

Open Preferences / Options and go to the tab Advanced -> General
Click: [Config Editor...]

Create the following: (from bug 397811)
"ldap_2.servers.osx.description" -> "System Address Book"
"ldap_2.servers.osx.dirType"     -> 3
"ldap_2.servers.osx.uri"         -> "moz-abosxdirectory:///"

3. Restart Thunderbird

4. Try out using your System Address Book integrated into Thunderbird

5. File some bugs or hop on the dev.apps.thunderbird news group to ask about issues

Some Know Issues

System address book is not turned on by default (thus the directions above)
Thunderbird doesn’t respect primary email address of OS X address book

Updates

These pref entries have to be created as they don’t already exist, see comments below.

bug 397811 has landed in the nightly Thunderbird releases, look for this menu.

Permalink 26 Comments

07Apr / 2008

Travel, Work and Other Updates

Travel

I’ll be heading out to Hamburg April 18-23rd with David and others for the Calendar project face-to-face meeting. It will be great to meet Christian in person now that we’ve been talking on the phone discussing possible Calendar and Thunderbird changes. I’m excited to make a quick overnight trip up to Copenhagen as I’ve never travelled there before.

Work

It’s great to see that Mark has started work with us. There’s lots to be done, especially on the address book, work that Joshua has developed in the Great Addressbook Rewrite. I’ve started compiling some research of other addressbook / contacts systems so we can have some ideas of what current implementaitons do.

Other

As I got back home really late after Friday, well into Saturday morning, I didn’t end up doing much on Saturday. So in my recovery time I poked around with my bugzilla link grabber extension and added a little AJAX to it. And thus I feel buzzword compliant!

Note the lovely screenshot of the bugzilla info inlined at the bottom. It might be nicer to create those elements as hovers to the bug links so they don’t take up space in the email but appear on mouse over of a bug link.

I just picked out a few things from the bug like bug number, status, number of comments, the title and the last comment text. Other information might be a bit better, but it’s all available.

I did this by using the XMLHTTPRequest to the bugzilla bugs XML version (just add “&ctype=xml” to the url) and then running the result through XPath. There’s a bit of a problem with the XML version as it gives you all the attachments as well as all the comments so things can be a bit slow when there are a lot of large attachments in a bug.

Anyway, not bad for a quick couple hour hungover hack.

Designer Code!!! eeek!!!

The code for all this is up on github in the ajax branch, check it out.

Who knew email didn’t have to be static!

Permalink 4 Comments

04Apr / 2008

Extending Bugzilla Links in Thunderbird

I took a couple hours… ok most of the day yesterday to fix a little issue that’s been bothering me for a while.

Bugzilla links inside email messages. I get countless messages where people reference bug 426175 but then don’t link to the bug. The other option is for the person to include the link in the email which is ugly and pushes the flow around https://bugzilla.mozilla.org/show_bug.cgi?id=426175 because there is a large link inside the text.

Despite emails not being HTML mail, for whatever reasons, I still want bugs to be linked in a reasonable manner when I’m looking at my mail. I couldn’t find an existing solution, though there likely is one hiding somewhere. So I started a new extension to solve my problem.

The Bugzilla Link Grabber Thing

I’m not good with names, another reason I probably shouldn’t have kids. (Offspring of Bryan Clark Jr.)

Here is a typical message that contains a mix of bugzilla urls and shorthand bug mentions. You can see the urls are long and a bit ugly, while the short hand link mentions (even though this example mentions the same bug) are not linked to the bug itself.

Before you view any mail the extension takes the long bugzilla urls and converts them into the shorthand form. It also linkifies any shorthand bugs into working urls.

Right now the extension only understands mozilla bugzilla and gnome bugzilla because that’s all I have accounts for. But others could be added and I was hoping to have a preferences dialog that allows you to add alternate bugzillas (see TODO). But otherwise it works great.

For xpi downloads, source, TODO, and more details take a look at the Bugzilla Link Grabber wiki page.

Fixing things

If you’re interested in this working for your copy of bugzilla or see some bugs in the code, don’t ask, please dive right in and fix things. Don’t forget to grab the STEEL extension or this one won’t work. Also it only works on thunderbird nightly builds right now; but maybe that’s something you can fix.

I put the extension source up at github, sorry if that’s not your RCS of choice. Anybody who creates patches to fix one of the TODO items is welcome to one of my remaining invites to github. I don’t really have anything else to give…

Permalink 13 Comments

31Mar / 2008

Lightning Nightly Builds

Since I started using the Thunderbird nightly builds I haven’t been using the Lightning plugin because the add-on version didn’t keep up with the latest build versions of Thunderbird. Luckily this morning I woke up from this reality when I found the Lightning Nightly Updater which fixes that problem.

After installing I changed it’s options to also install the gdata provider for Google Calendar, then I ran the Check for Updates menu item from the Help menu and calendaring is back.

Update: Just to note that the nightly updater should grab the proper version even if you’re running a thunderbird branch instead of trunk.

Permalink 4 Comments

20Mar / 2008

Blue STEEL!

Out of the Thunderbird status meeting this week I learned that there’s already a test extension available which gives Thunderbird Extension Developers some of the new STEEL interfaces.

What is STEEL?

STEEL stands for, Scriptable Thunderbird Easy Extension Library, and that means a simple javascript library to access your data (email, addressbook, etc.) inside Thunderbird. Like it’s Firefox counterpart FUEL, STEEL will create an easy extension development API for Thunderbird.

What can you do with STEEL?

Right now STEEL is in it’s infancy with a 0.1 release as you can see in the implementation plan. However there already exists lots of things you can do with the existing STEEL code.

Here are some examples:

You’ll need to grab the latest STEEL extension from bug 408370, luckily there are STEEL Extension Install Instructions which you can follow.

Where is STEEL going?

What happens with STEEL is up to extension developers. If you’ve already developed an extension for Thunderbird please give STEEL a try and let the developers know what you think. If you’ve been thinking of developing an extension for Thunderbird try STEEL out to see if it does what you need.

The coming API will depend a lot on the kind of feedback that can be gathered right now. Join the conversation on the #maildev IRC channel or send a message to the mozilla.dev.apps.thunderbird newsgroup.

Personally I’d still like to see a couple things happen.

Simplify extension development by relying heavily on a local cache. As a person who wants to try out a lot of different ideas inside Thunderbird via extensions I’d like to avoid network latency issues and would rather have all the information cached and indexed locally such that all calls could be fast and synchronous. The only signal I would want to worry about is when the cache has updated so I can refresh my calls.

Allow for objects to be retrieved separately and by-directionally queried. For example I’d like to be able to ask for a list of attachments and then for each attachment find out what message it was sent in, who sent it, and even other attachments they sent.

for each ( attachment in Application.attachments )
- from = attachment.message.from
  - for each ( attachment in from.attachments )

Improved Search APIs. I got to talk with the excellent David Huynh of SEEK fame and asked if he could take a look at the STEEL APIs for improvements; I’m excited to see what he has to say. Improving search is one of the major goals for Thunderbird 3 and all the great new ideas come out of extensions so development needs to be ready for that.

Malleable STEEL

A simple exercise I have been doing is to take an existing extension, even ones for outlook, and offer a Thunderbird extension developers perspective of the STEEL API . Right now I have a breakdown of the Xobni Extension which I thought was very interesting extension that I’d like to see in Thunderbird in the future. The breakdown examines different pieces of the extension with simple function calls that could enable it.

Permalink 3 Comments

21Feb / 2008

Sending the right message

Mozilla Messaging has been formed! And yes, that’s me in there; I’ve made a new home with Mozilla Messaging so look forward to much more talk about email, calendaring, and communication in general.

Luis pointed out, the name born from MailCo is Mozilla Messaging, a change that I believe brings the correct focus to the coming efforts.

Since the launch I’ve been reading lots of different articles about the launch and the related comments following peoples opinions.

lifehacker: What Do You Want to See in Thunderbird?
slashdot: Mozilla Opens Thunderbird Email Subsidiary
ars: Mozilla Messaging reveals plans for Thunderbird 3
lwn: Mozilla Messaging Starts up Operations
- For novices, fix reliability and usability
More Links at: Mozilla Messaging Launches

Here’s a mix of common threads I’m seeing from the comments.

Polish!
Calendar Integration
Better Search
Shared Calendars
Better LDAP
Conversation View (gmail mail)
Archiving Support
Encryption (make it easy and by default)
Phone Sync (Address Book, Text Messages, Phone Logs)
Better OS Integration (Windows, OSX, GNOME, KDE, etc.)
Better Address Book
Import Outlook PST, .DAT files
Exchange Server Support
Better vCard Support
Taggging (ala gmail)
Sync contacts w/ GMail, LinkedIn, Facebook, Yahoo, etc.
New Mail / Reply Templates

And a very common theme is for making Thunderbird FAST AND LEAN!!

All excellent directions to look into. But that is a big list of tough items to handle all at once. I’d like to take the approach of focusing on our core goal and iterating many of these pieces as they align and become defined by our progress.

Email… and Calendaring?

David Ascher wrote an excellent entry launching Mozilla Messaging which unveiled the current plans for Thunderbird. Many comments in his blog and other articles relayed a feeling that these new Thunderbird plans would lose sight of improvements that are sorely needed to bring regular email up to speed. When actually there is a short and medium term set of fixes that need to be done in order to get the regular email experience up to speed.

In parallel to those email fixes we’ll be working on a plan of improving personal communication. There’s no secret agenda in this communications plan we’re going to be very open, honest, and looking for lots of feedback. But it includes improving calendar support inside Thunderbird, not as an additional tool to download but as another way to help people Communicate Effectively and Get Things Done.

Getting Things Done

I’ve got to make a run to the used bookstores in Central Square and look for a copy of Getting Things Done. If that doesn’t work out I’ve added that to my wishlist just in case. But email is usually a thorn in my side, I get so much of it everyday like so many other people and yet I feel like I’m fighting the tide of emails instead of sailing the seas. I like the set of mental tools that the GTD book provides for not languishing in the meta and instead diving into the guts of things and I want to see that inside my email. I want to feel a sense of control over my email again, like I used to have when I first started using email so long ago.

Extensions and innovating beyond ourselves

Since there are so many ideas out there on how to improve communication, organize email, handle attachments, integrate calendaring, and more there is no way a small team will be able to prototype all these ideas; much less implement them all. This is where extensions have to come into play. The Thunderbird extension system has fallen far behind that of Firefox and it’s constraining our communities ability to innovate in an area where so much innovation is possible. Extensions allow everyone to try to fix this problem with email and communication and it creates a vibrant feedback loop where we can all benefit; but I’ll talk more about this later.

My starry eyed view of the future of Thunderbird is a fast and lightweight platform for wrangling all my electronic communication, a supported, active, and vibrant extensions community building cool new ideas on top of an extensible platform can be tried, tested, and rolled into future releases. The future is bright.

Permalink 18 Comments

Newer posts →

Bryan Clark

Tag Archives: thunderbird

Auto-Complete on Subjects