MBOX-Line: From mrc+imap at panda.com  Thu May 19 08:20:21 2011
To: imap-protocol@u.washington.edu
From: Mark Crispin <mrc+imap@panda.com>
Date: Fri Jun  8 12:34:46 2018
Subject: [Imap-protocol] Thoughts on keyword improvements/enhancements
In-Reply-To: <4DD4FAAC.3090101@gentoo.org>
References: <20110518011645.Horde.6sTkboF5lbhN03Jd3XNndCA@bigworm.curecanti.org>
	<BANLkTikVbKdXg01nnajJ-=XLNfTpjjjG0Q@mail.gmail.com>
	<alpine.OSX.2.00.1105181108180.24932@hsinghsing.panda.com>
	<20110518210716.GA12636@brong.net>
	<alpine.OSX.2.00.1105181519330.24932@hsinghsing.panda.com>
	<20110519045851.GA23466@brong.net>
	<alpine.OSX.2.00.1105182203320.24932@hsinghsing.panda.com>
	<4DD4FAAC.3090101@gentoo.org>
Message-ID: <alpine.OSX.2.00.1105190737550.24932@hsinghsing.panda.com>

On Thu, 19 May 2011, Jan Kundr?t wrote:
> Please advise me on a way to better show information about the total
> number of unread messages in a mailbox. I can't use STATUS on a SELECTed
> mailbox, and I can't do anything when I see an EXPUNGE of a message for
> which I haven't got flags before. Shall I decrement my unread count, or
> not? To my knowledge, there's absolutely no way of doing that correctly
> except knowing FLAGS of all messages in advance.

SEARCH will always be less data than fetching all the messages in advance.

It can be argued that you're still in a world of hurt in a 100K mailbox
where most messages are unseen (or there's an unseen message near the
start).  But it is less of a world of hurt than fetching all the flags.
What's more, this is an edge case, not the normal use case.

[And it's not all that bad.  I've loaded threads for mailboxes that large,
via WiFi, on a bitty device.  It took a while, but not forever.
Considering that most bitty device clients toss their cookies at much less
than 1000 messages, the fact that it handled 100K messages at all was
amazing and the modest delay was excusable.]

Unseen messages in the normal use case are clustered at the top of the
INBOX.  That's not going to hurt.  This is the case that you care about,
not the edge case.

You even have warning about the edge case; the UNSEEN value from SELECT!
If you see:
 	* 100000 EXISTS
 	* OK [UNSEEN 1]
that indicates that there may be as many as 100000 unseen messages (and
even suggest that all messages are unseen).

Conversely, if you see:
 	* 100000 EXISTS
 	* OK [UNSEEN 99982]
Then you know that there are, at most, 19 unseen messages.

Depending upon how much you care about the edge case, there are some
creative things that you can do.  For example, you can do
 	tag SEARCH 1:50 UNSEEN
and found that all of the first 50 messages were unseen, that might be an
indicator that all messages are unseen.

In fact, if you do
 	tag FETCH 1 FLAGS
or
 	tag SEARCH 1 RECENT
and find that the first message is recent, that is a sure indicator of
that case.  You don't care any further.

You may also want to say that showing an unseen count is only useful if it
is under some value.  Thus if you do:
 	tag SEARCH 99900:100000 UNSEEN
and see that they are all unseen, then maybe just report "99+" as the
unseen value.  As an end user, once we get into three digit unseen counts,
we're in the world of "lots of unseen messages" that I'm probably not
going to read on a bitty device screen.

It is limited only by your imagination (a problem for some Gen-Xers) and
the number of RTTs that you're willing to spend.

There are lots of other tricks that you can do with message sequence
numbers and the very useful property of strict ascendency.  A client that
does not use MSNs at all will suck.  It can't help but suck; it is denied
most of the magic of IMAP.

Last but not least, there is an extension called ESEARCH that solves this
problem.  Unfortunately, there is ambiguity in how to interpret the
ESEARCH specification.  But for this purpose, if the server offers ESEARCH
you are home free.

In conclusion, let me emphasize that there is quite a bit of magic in IMAP
that you'll never learn if you just look at the examples and start
programming without considering the interactions.  IMAP is a (too) subtle
protocol; yet every arcane aspect has a purpose - at times quite deep and
magical.

Learn as well to distinguish between edge cases and common cases.  Don't
say "there's no point to doing the common case well because the edge case
will suck"; that will lead you to sucking in the common case.  Let the
edge case suck; you want to excel in the common case.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
