MBOX-Line: From brong at fastmail.fm  Wed May 18 01:08:07 2011
To: imap-protocol@u.washington.edu
From: Bron Gondwana <brong@fastmail.fm>
Date: Fri Jun  8 12:34:45 2018
Subject: [Imap-protocol] Thoughts on keyword improvements/enhancements
In-Reply-To: <20110518011645.Horde.6sTkboF5lbhN03Jd3XNndCA@bigworm.curecanti.org>
References: <20110518011645.Horde.6sTkboF5lbhN03Jd3XNndCA@bigworm.curecanti.org>
Message-ID: <20110518100807.29d4b72c@arbre.eng.oslo.osa>

On Wed, 18 May 2011 01:16:45 -0600
Michael M Slusarz <slusarz@curecanti.org> wrote:

(Terminology: keyword, label, flag - joy)

> 1. Implement UTF7-IMAP (or similar) storage of keywords.
> Pros: Would allow keyword & label information to be kept in a single  
> location; all MUAs should be able to easily adapt since they should  
> already have code necessary to do UTF7-IMAP conversions
> Cons: Haven't thought about it fully, but doesn't seem like it could  
> be made to be backward compatible with RFC 3501 keywords.  Doesn't  
> solve problem of having one keyword meaning the same thing across all  
> mailboxes.

Lack of backwards compatibility kills this one out the gate.  You can't
break existing clients and servers.

> 2. Define a server METADATA entry (RFC 5464) to store keyword->label mapping
> Pros: Works with old RFC 3501 servers.  Leverages an existing IMAP extension
> Cons: Doesn't solve problem of having one keyword meaning the same  
> thing across all mailboxes.  Doesn't really seem to be the kind of  
> thing that should necessarily be a metadata entry - flags/keywords are  
> a core part of the IMAP spec, so to me it just "feels" like something  
> else should be used instead (really useful description, I know).

IMAP is a twisty little maze of hacks on top of the original spec already.

> 3. New commands/response codes
> Pros: Works with old RFC 3501 servers.  Would be able to handle issue  
> of one keyword meaning the same thing across all mailboxes.
> Cons: As with all extensions, adds complexity and more commands to  
> implement on the client side.

And on the server side too - unless you only want to support a couple of
servers.  Cyrus would add it, and Dovecot.  But selling it to everyone else
might be tricky.  And actually, storage in Cyrus would be slightly tricky
too, but I can figure that one out.

> 4. Combo of #2 and #3
> Leverage #2 to store label lookup information, with #3 to define  
> system-wide keywords.
> Pros: See #3.  Only adds commands necessary that can't be done through  
> other means.
> Cons: Requires client/server to support METADATA, even though it would  
> probably not be that much more work just to add the needed lookup bits  
> to a new extension.

I think once you're requiring the client and server to support a brand new
command anyway, requiring METADATA isn't a big stretch.  I believe you can
actually store a server-level METADATA item, as well as per-mailbox ones.
Makes sense to use that.  It does mean that the admin needs to add new
keywords to the system though... mapping to basic flag names underneath.

> Hopefully I have provided a coherent explanation of my perceived  
> limitations with the current specs.  Coming from a webmail developer,  
> this is an important issue to me since it is quite common to be  
> dealing with IMAP servers that are being accessed by other MUAs.   
> Synchronization of keywords would go a long way towards making  
> cross-usage of MUAs smoother.

Absolutely.  While you're at it, can you look into standardising
$NotSpam $NonSpam NotSpam NonSpam Spam Junk NotJunk, etc - all the
various magic flags used by different clients to label UBE.

This boat may well have sailed...

> As a last little bit, just to get it out there, I will mention that  
> Timo also briefly mentioned that he has thought about commands to  
> manage PERMANENTFLAGS  
> (http://dovecot.org/list/dovecot/2011-May/058894.html)  Figured I  
> would throw it out there while on the topic of flags/keywords in order  
> to ensure all topics are covered.  Hope you don't mind Timo :)

I've been thinking about that for Cyrus.  We run a "repack" on the mailbox
every so often - and we could count up the usage of flags during that, and
remove any named user flags from PERMANENTFLAGS that had no message currently
using them after a repack.

FYI: Cyrus has a hard limit of 128 user flags due to the fixed-width binary
nature of the cyrus.index format.  This makes removing stale flags slightly
more pressing for Cyrus than for other servers which don't have this limit...

Bron.

