MBOX-Line: From mrc at CAC.Washington.EDU  Wed Apr 12 10:41:47 2006
To: imap-protocol@u.washington.edu
From: Mark Crispin <mrc@CAC.Washington.EDU>
Date: Fri Jun  8 12:34:37 2018
Subject: [Imap-protocol] LIST Clarification
In-Reply-To: <web-35046695@mail.stalker.com>
References: <443A7A2D.2070708@consilient.com> <web-35034698@mail.stalker.com>
	<Pine.OSX.4.64.0604101053530.2906@pangtzu.panda.com>
	<web-35035906@mail.stalker.com>
	<Pine.WNT.4.65.0604101441160.4904@Tomobiki-Cho.CAC.Washington.EDU> 
	<443C0286.60200@att.com>
	<Pine.WNT.4.65.0604111228310.3332@Tomobiki-Cho.CAC.Washington.EDU>
	<syMWpRdWCviH+GmTLGxvWQ.md5@libertango.oryx.com>
	<Pine.WNT.4.65.0604111418130.3332@Tomobiki-Cho.CAC.Washington.EDU>
	<web-35044345@mail.stalker.com>
	<Pine.WNT.4.65.0604111814490.3332@Tomobiki-Cho.CAC.Washington.EDU>
	<web-35045943@mail.stalker.com>
	<Pine.OSX.4.64.0604112206540.2906@pangtzu.panda.com>
	<web-35046695@mail.stalker.com>
Message-ID: <Pine.OSX.4.64.0604120854390.2906@pangtzu.panda.com>

On Wed, 12 Apr 2006, Vladimir A. Butenko wrote:
>>> correctly matching the pattern the client has provided - matching 
>>> according to the rules known to the server only, not the client?
>>> then the client HAS ABSOLUTELY NO RIGHT to expect that all returned names 
>>> will start with "ZZZ/", and thus - can be displayed as "ZZZ" "subtree".
>> I don't understand why.  There's no M-UTF7 here. 
> Who cares? You've just said that I (being a client) cannot assume ANYTHING 
> about your (server's) responses, so I cannot plan my life depending on you.

But that's no different than doing a wildcard in the shell on a system. 
The response is according to that system's rules.

I had to learn (very painfully I might add) that
 	mv *.c *.c.bak
on UNIX did not do what I expected it to do.

Mac OS X is particularly bizarre, although it makes sense when you 
consider how it is implemented.  A non-wildcard ls matches a name 
case-insensitive, but a wildcard ls is case-sensitive.

IMAP never tried to define an ideal mail store which had one (and only 
one) set of rules.  Rather, IMAP tried to be a means to export many 
different types of existing mail stores, often with very different rules.

>> The only thing that may different is that with a case-insensitive server 
>> that it may return names that start with zzz/ or Zzz/ etc.
> As we saw with non-ASCII names, it's more complex than that, and the number 
> of variations is endless (though it's a countable set, not a continuum :-).

Yes, but your example was ASCII.  I agree that once we get into non-ASCII 
names things become much messier.  But case-sensitivity is not the 
solution.  You have to be completely octet-sensitive if that's the goal.

> There are some freeware IMAP servers using "." as the path separator. At the 
> same time, "." is part of "Modified-UTF-7 BASE64" encoding.

"." can not be a hierarchy delimiter if it is within an M-UTF7 sequence. 
Perhaps you can't use "." as a hierarchy delimiter with M-UTF7.  Note that 
M-UTF7 did get out of the way of "/".  But yes, a server that uses "." as 
a hierarchy delimiter has a problem with M-UTF7.

Most people agree that it was a mistake for IMAP to have server-defined 
hierarchy delimiters; and that "/" should be the one and only hierarchy 
delimiter.  Hindsight is always easier.

Everybody agrees that M-UTF7 was unfortunate.  It was necessary at the 
time to abolish "just send 8-bits".  Too bad that punycode didn't exist at 
the time, since punycode is far superior.

Also too bad that people did "just send 8-bits" for IMAP mailbox names; 
had that not happened, we might have been able to define UTF-8 back then.

> So, some name can 
> look as &ADB.r985-, because it was "言語" (it's just an example, not a real 
> encoding of this japanese word). So, what would
>   LIST "%.%" command return?
> I'd say that the client really cannot know here, and the hierarchy it will 
> present to the user would be quite different from what some users expect.

I agree.  Life is tough for client authors some times.  Life is tough for 
server authors some times too.

We don't live in a perfect world.  I don't think that it's useful to 
complain about imperfections because perfection is unattainable.  It's 
more useful to try to determine what is the most reasonable thing to do 
in the fact of imperfections that exist and that other people will 
continuously create.

Most of these particular problems will be presently overtaken by events. 
At some point, we will adopt UTF-8 mailbox names, and M-UTF7 will be first 
a compatibility mode for old broken clients and later a bad memory of the 
past.

>> IMAP makes no such promises.  It's implementation dependent, as are the 
>> semantics of the naming hierarchy.
> And that's exactly the problem.

I don't consider that problem to be solvable.  The more restrictive that a 
protocol becomes in the face of implementation options, the less suitable 
that protocol is for various applications.

> a) I'm trying to point to the problem with the current design.

We all know these problems.

> b) Because I'm getting a feeling that some people, including Mr.Crispin MAY 
> find time to do some revision of the IMAP protocol in the nearest future

You are too optimistic.

So far, IMAP has been a 20 year process, the past 10 of which have 
supposedly been refining the current revision.  A new revision would open 
up a Pandora's box, as *everybody* will see it as their opportunity to 
get their own personal hot-button issue address.

I'm not volunteering to open that box.  I would be one of the people 
fighting tooth and nail to keep that box slammed shut.

IMAP is not the only IETF protocol which is having problem with 
internationalized identifiers.  It is not the only protocol in which the 
first attempts at internationalized identifiers have proven flawed.

There are a great many people who are wrestling with internationalization 
problems; and I am happy to await their guidance.

Be thankful that, unlike NNTP, IMAP took an early stand against "just send 
8-bits".  NNTP is a mess, and the proposed new specification isn't doing 
anything to solve that mess; it defends every behavior of every server 
forever.

> It all depends on the amount of time you have. If I were you (if I may), I'd 
> drop all concerns about the "compatibility issues" - and would create a clean 
> and nice IMAP5 (NOT IMAP4revX).

Compatibility with IMAP2 isn't the problem; in fact, hardly anyone worries 
about that any more.  Creeping featurism is the problem.

As attractive as it may sound to build an IMAP5 that removes the current 
known warts, I can guarantee that such a protocol would have much more 
added than removed.  It would not be completed before I retire.  It 
probably would not be completed during my lifetime.

And...it would have brand new warts, that won't be generally-recognized as 
warts until years later.

> But - that's just my not so humble opinion, this protocol is your kid, it's 
> up to you to decide if it's a time for a Revolution, or smooth evolution is a 
> better thing to do.

My goal is for IMAP to evolve to a state where it can be called "done", 
and then let the next generation worry about its successor.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

