Muttlight is a MacOS application that improves search and preview for email files stored in MailDir format. Specifically, it allows the associated file extensions to be specified and registered via a graphical user interface, provides a Spotlight importer to extract meta data, integrates with Quick Look to provide previews and thumbnails, and allows files to be opened directly in Mutt.
Muttlight Launcher was created with Sveinbjörn Þórðarson's Platypus application. It responds to ‘open’ requests on MailDir messages by running a simple (naive) shell script to launch an Iterm2 session, or failing that, a native Terminal session, with Mutt open on the selected message.
Documentation is available elsewhere.
Pull requests are welcome.
- Install prerequisites
brew install gettext
brew install ncurses
- Clone (or link) the Mutt source code into a
src/muttsubdirectory
wget -qO- ftp://ftp.mutt.org/pub/mutt/mutt-1.8.3.tar.gz \
| tar xvz -C ./src && ln -s mutt-1.8.3 src/mutt
- Build Mutt
(cd src/mutt && ./configure && make)
- Build Muttlight
cd src
make
Install Muttlight simply by copying the Muttlight.app bundle (directory) to
$HOME/Applications.
Mutt is a text-based email client that works as well in a terminal on MacOS as on any other system. It provides an efficient, configurable, programmable, and keyboard-only interface, seamless integration of powerful texts editors like Vim, and a non-vendor-specific and easy-to-synchronize storage format, namely individual MIME files in MailDirs.
Unfortunately, mail messages stored in MailDir format are not well
integrated into the MacOS search (Spotlight) and preview (Quick Look)
features. Mails appear in Spotlight results, since they are indexed, but
their encoded filenames are all but unreadable and their contents are not
displayed. Renaming these files to have the .eml extension causes them to
be properly indexed, displayed, and previewed by exploiting plugins in the
native Mail application. But, this violates the MailDir format and renders
the files unreadable by Mutt and similar applications.
Two possible solutions suggest themselves.
-
Introduce a version of the MailDir specification that requires
.emlextensions and modify Mutt and similar applications accordingly. -
Somehow get the Mail application to also index these MailDir files without renaming them.
The first solution is possible since Mutt is open-source software, but it either complicates system configuration, since Mutt must be installed with patches, or requires upstream acceptance of the patches. It would, furthermore, be necessary to patch mail delivery agents like fetchmail and postfix. But, is it reasonably for a relatively minor issue on a particular operating system to spread to such long-standing and widely-used pieces of software?
The second solution would be ideal, but unfortunately I was not able to make
it work. I tried registering file extensions in the finder, with the
duti application, and using the UTTypeConformsTo
feature of MacOS applications. The Mail application is closed source and so
it is not possible to investigate it directly or to patch it. If you know
how to do this or you work for Apple and can influence the application,
please contact me! I would be very pleased to make
Muttlight redundant by finding a better solution to the problem described
above.
Muttlight represents a third solution: an application that registers itself as the owner of MailDir files and leverages Mutt's source code to improve their integration into the MacOS search and preview features.
A filename in the MailDir format ends
with a suffix of the form :2,DFPRST, where the six final letters are flags
that are either present or absent (giving 64 different combinations). Since
file extensions on MacOS begin with a ., it is also necessary to consider
the characters to the left of the suffix. They are sometimes mbox, but
more often they are (part of) the host name on which the file was originally
created (to ensure distinct names across multiple systems). Files in the
new subdirectory are named without a :2,DFPRST suffix but usually
include a host name. Obviously, the host name extensions will vary across
installations. Muttlight provides a GUI that searches the system for MailDir
files, summarizes the extensions that are being used, and allows users to
select those which should be treated. Note that the
DoveCot version of the
MailDir scheme cannot be handled since it embeds file size information in
the extension.
Once the MailDir filename ‘extensions’ have been registered to the Muttlight
file type (org.tbrk.muttlight.email), the system will rely on Muttlight to
extract metadata and provide preview images. Both features are provided by
plugins that exploit the existing Mutt source code to parse and display mail
messages.
Metadata is extracted from the mail header and any plain text, html, or enriched parts. Attachment names are indexed, but not their contents. The various parts are decoded before being indexed. This should improve the quality of search results, particularly for emails in character sets other than ASCII. I do not know whether the Mail application indexes mail attachments but this seems difficult to achieve using the public Spotlight interfaces (since it would be necessary to recursively call other importer plugins).
The Quick Look plugin exploits (...hacks around...) the Mutt pager to give previews that, while not as elegant as the native Mail ones, closely resemble the text-based display in the Mutt client. This style may even be preferable to some users. Rudimentary parsing is applied to colorize the header fields, attachment status lines, and quoted replies.
Altogether, these features, together with the Mutt Launcher component, allow a rapid configuration and natural integration of MailDir contents into the MacOS user interface.
There are, of course, other ways to index and search mail messages in MailDir format. For instance, I already use mairix on both MacOS and Linux, and it works well once you get the hang of it. It has the advantage of working specifically with mail files and of showing the results directly within Mutt.
The advantage of Spotlight is that the indexes are kept up-to-date more dynamically and they span all files and file types on a system. Muttlight allows more convenient browsing of mail messages when they come up amongst other results (pdfs, text files, source code, etcetera).
Run mdimport -L to see the list of installed Spotlight plugins.
The list should include
.../muttlight.app/Contents/Library/Spotlight/muttlight.mdimporter.
Run mdls on a MailDir file to check that the plugin is working correctly.
The kMDItemContentType field should be org.tbrk.muttlight.email and the
other fields (kMDItemAuthors, kMDItemDisplayName, etcetera) should be
valid. To log the importing process and to see whether the plugin is being
called, run mdimport -d 4 <filename>.
Run qlmanage -m | egrep --color '.*muttlight.*|$ to see the list of
installed Quick Look plugins (and to highlight the muttlight entry).
The list should include
org.tbrk.muttlight.email -> .../muttlight.app/Contents/Library/QuickLook/muttlight.qlgenerator.
It may be necessary to reset the plugin manager by running qlmanage -r
(and qlmanage -r cache).
Open a MailDir directory in Finder to check that thumbnails are being
generated correctly, press ⌘-Y to preview a file. The thumbnails and
previews should show the messages more or less as they appear in mutt.
The Quick Look plugin can be debugged by running
qlmanage -d 4 -p <filename>. This will show detailed logging. There may be
a delay before the preview is displayed. Add -o . to the command line to
dump the generated files to disk.
Run mdfind <keywords> to list indexed files that contain the given
keywords. Entering the same keywords into the Spotlight search box
(⌘-Space) should display the same list. When muttlight is functioning
correctly, any mail files in the search results are named according to their
subject (and not their MailDir filename) and their contents should be
previewed as in the mutt pager. It will probably be necessary to
reindex your MailDir directories
before the search results are named correctly (the mds and mds_stores
processes become active during reindexing).
The following command deletes and regenerates the Spotlight index on the root volume (regeneration may take some time).
sudo mdutil -E /
Run the following command and search for muttlight to see registered file
extensions and plugin “claims”.
/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister -dump | less
Reset the entire Launch Services database:
/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister -kill -r -domain local -domain system -domain user