wasm-demo/demo/ermis-f/imap-protocol/cur/1600095110.22807.mbox:2,S

MBOX-Line: From mrc+imap at panda.com  Fri Jun 11 14:17:56 2010
To: imap-protocol@u.washington.edu
From: Mark Crispin <mrc+imap@panda.com>
Date: Fri Jun  8 12:34:43 2018
Subject: [Imap-protocol] IMAP MOVE extension
In-Reply-To: <201006112113.56572.witold.krecicki@firma.o2.pl>
References: <201006110854.37969.witold.krecicki@firma.o2.pl>
	<201006111953.13490.witold.krecicki@firma.o2.pl>
	<alpine.OSX.2.00.1006111111320.662@hsinghsing.panda.com>
	<201006112113.56572.witold.krecicki@firma.o2.pl>
Message-ID: <alpine.OSX.2.00.1006111214020.662@hsinghsing.panda.com>

On Fri, 11 Jun 2010, Witold Kr?cicki wrote:
>> Maildir is not a particularly good store for IMAP.
> Yet it is the very most popular store for small-to-medium servers.

And your evidence for this is????

You don't know this.  You're guessing, and making completely unfounded
assertations based upon thos guesses.

>> MOVE doesn't solve those problems in maildir; it makes them worse.  MOVE
>> doesn't solve those problems in other formats either.
> No, MOVE keeps the problems in maildirs at the same level. It does not make
> them worse.

Yes, because MOVE introduces requirements and semantics that currently DO
NOT EXIST in IMAP.

By its definition, MOVE must fufill the requirement of removing from the
source.  There is no way that this can not be done; otherwise it would be
a COPY and not a MOVE.

IMAP has no guaranteed means of removing a message.  Study the semantics
of STORE and EXPUNGE carefully.  MOVE, on the other hand, requires that
guarantee; and does not have the semantics of STORE and EXPUNGE.

>> You're still thinking in terms of maildir as being a prototypical IMAP
>> store.
> And you're still thinking in terms of one and only implementation as the only
> correct one.

I am thinking about at least four very different types of mail stores, and
various subdivisions of these.  I have implementation experience with all
four and their subdivisions.

>> MOVE provides no real benefit to such stores; yet those stores are the
>> only ones that can actually implement it properly.
> Database-backed stores can implement it properly and can have performance
> boost.

So, you postulate a database which is designed to have a performance boost
in the case of this extension, yet can not implement the base
specification without a performance cost.

>> Think about what happens when some server implements XAOL-MOVE in a way
>> that is different than Thunderbird expects (since it has only ever seen an
>> AOL server do it).
> That would mean that the specification is not clear enough. As I can see this
> is pretty often in IMAP.

It is "pretty often" in every IETF specification ever written.

Even clarity in specification isn't enough.  IMAP has incredibly rigid
syntax, carefully spelled out in the formal syntax, that is designed to be
readable by a syntax parser.

Yet time and time again we see people (at some Very Large Vendors) who
don't use the formal syntax (and thus do not use any syntax parsing code
generation tools) but instead hand-code parsers using examples as a guide
as if we were still back in the 1970s and using BASIC.

>> And in such a case MOVE has no benefit.  There's no quota cost to COPY.
> Yes, there is if there is only one 'mail label' per mail. Copying in this case
> is equivalent to inserting new record and, if handled inproperly, can be non-
> atomic (comparing to fully atomic move)

Not "can be non-atomic"; rather "is non-atomic".

Any failure anywhere in the chain leads to an indeterminate state on the
server from the client's perspective.

>> That is one of many reasons why Maildir is not a particularly good store
>> for IMAP.  Timo's server had to extend Maildir.
> Again, yet it is the very most popular store for small-to-medium servers.

Prove it.

> You are pursuing the goal of all IMAP operations to be atomic, not me.

So you, a newcomer to the IMAP world, purpose to undo 25 years of IMAP
history by repealing the requirement that all IMAP operations are atomic?

Are you ready to face the torches and pitchforks when client authors
suddenly discover that you've turned IMAP into CrapMAP; a protocol where
it is completely indeterminate on the client what happened in the server?

> But
> read few lines above - I have written client case (specifically on weak
> connection clients such as GPRS/EDGE).

I've used such environments (and far worse) for well over 15 years.

We do not move forward by adding new capabilities to solve yesterday's
problems tomorrow.

2G (and 1G before that) radio is yesterday's problem.  And it wasn't that
much of a problem.  The extra RTTs can be eliminated by pipelining, and it
is extremely rare that you have a sparse enough sequence that the byte
count becomes significant.

Considering all the crap that gets transmitted over radio data, I find it
very hard to be sympathetic to claims that it is important to save 50
bytes in a user-initiated transaction.

>> So far, you've been talking about server-based benefits, which I (and
>> others) have shown are more illusory than real.  Adding MOVE will/can not
>> fix a slow server implementation of COPY.
> No. Only You have tried to show me that server-based benefits are
> illusory, and have shown no arguments to that except for examples from
> exisiting implementation.

Huh?  Where did that come from?

I discussed the impact on multiple major classes of store implementations.

If you don't understand something, please ask questions instead of
insisting that you are right.  There are severalpeople here with decades
of experience and some amount of wisdom.  As tempting as it is to handle
complex situations with simple solutions, more often than not those simple
solutions backfire and lead to disaster.

>> Multiple client code paths depending upon server is not a good thing.
> Well, it sounds pretty weird hearing it from You. You have created a protocol
> that, by design, supported extensions. And the result is different capabilities
> of different implementations

Stop claiming that you know what other people think and do.

If you look at the history of IMAP, you will discover that I did not
design IMAP with extensions, and that I opposed the addition of
extensions.

>> If AOL uses a database that permits that kind of operation, then there is
>> no reason why COPY can not also work that way.
> In this example COPY would create a need to insert new records, MOVE is just
> update. How could COPY work that way?

If MOVE can be done atomically by an update, there is no reason why COPY
can not be done that way too.

Even if the database doesn't work that way, that doesn't mean that a new
functionality has to be added in IMAP for it.

>> There is no server benefit.  MOVE does not solve the problem of servers
>> with slow COPY.  It does create the temptation for server implementors
>> with slow COPY to create an unsafe, non-atomic, implementation of MOVE
>> (e.g., using rename()).
> So there is server benefit (as shown above)

No.  We don't want unsafe, non-atomic, implementations.

Nor do client implementors.
 	"Hey Mr. Client Implementor, we're going to give you
 	this wonderful MOVE command.  But it won't always
 	actually do what you expect, and you won't know what
 	was actually done."

>> This is an "attractive nuisance".  It's one of those ideas that sounds
>> really good on paper until you study the unintended consequences.
> What consequences?

A facility that is unimplementable with some stores, some of which
(notably Maildir) appear to be implementable albeit by a dangerous
implementation that can (and will!) cause indeterminate results to
clients.

A facility that, on the stores where it is implementable, provides no
compelling advantage.  You have to presuppose a store that is designed so
that COPY must be slow and MOVE must not be yet is atomic.

IMAP has generally not added features whose sole purpose is to provide
support for a particular type of store.  Far more common is that a
particular type of store has vetoed a proposed addition.

> And that sounds like total inability to listen to your counterpart on Your
> part.

This is not the first time this discussion has taken place.

Every few years, a newcomer comes along and says "I have this simple
solution that you all should adopt."  Then more experienced people try to
explain the complexities of the situation.

There are several of these recurring discussions in IMAP.

I first considered a MOVE command in the original IMAP protocol in 1986.
1986!!  There were reasons then not to do it, even though it made other
tasks more difficult for me.

Those reasons remain valid today; and additional reasons have come about
since then.

I understand perfectly well why a MOVE command is desirable.  You do not
need to repeat those reasons.  I have known these reasons for nearly 25
years.  For the first 5 or so of those years I was the only implementor,
and it still annoyed me that it wasn't there.

Unfortunately, the reasons that a MOVE command is undesirable are more
compelling.

Don't assume that I (or anyone else) somehow don't understand your
brilliant insights as to why it would be nice to have MOVE, and that we
aren't listening when we tell you that there are problems with it.

> I have gave you a good example of mail store (that I have seen) in which
> MOVE is atomic and in which MOVE is much faster than COPY - and you have
> ignored it.

You offered an example of a poorly-designed database.

Does IMAP have to change every time someone wants to deploy a poor
database in IMAP?

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.