MBOX-Line: From mrc+imap at panda.com  Mon Jun 14 22:34:39 2010
To: imap-protocol@u.washington.edu
From: Mark Crispin <mrc+imap@panda.com>
Date: Fri Jun  8 12:34:44 2018
Subject: [Imap-protocol] MOVE is a pipeline
In-Reply-To: <4C17017A.2090305@aol.com>
References: <201006110854.37969.witold.krecicki@firma.o2.pl>
	<201006150012.43889.witold.krecicki@firma.o2.pl>
	<alpine.OSX.2.00.1006141528400.662@hsinghsing.panda.com>
	<201006150103.52513.witold.krecicki@firma.o2.pl>
	<alpine.OSX.2.00.1006141604420.662@hsinghsing.panda.com>
	<4C16C4E8.7010008@aol.com>
	<alpine.OSX.2.00.1006141713260.662@hsinghsing.panda.com>
	<4C17017A.2090305@aol.com>
Message-ID: <alpine.OSX.2.00.1006142133200.662@hsinghsing.panda.com>

On Tue, 15 Jun 2010, John Snow wrote:
> If the client sends a copy 1:100 saved
> and the server crashes when only 50 have been copied what happens?  The
> server managed to copy 50, but didn't finish so the client doesn't know
> any were copied yet.  In this case I 'd expect that the client
> reconnected and retried the copy resulting in the first 50 being
> duplicated in the saved folder.

That does not happen with my mix store in any of the servers that use it;
nor will it happen in the new store that I am designing.

In mix, a crash while a copy is in progress simply causes lost space which
will get recovered the next time the mailbox is opened.  In the new store,
not even that consequence will happen.

It may happen on a crash with legacy flat file stores; but only on a crash
and not on non-crash failures.

>> Do you think that client implementors will be comfortable with that?  "It
>> may or may not move all the requested messages, and it may leave you in a
>> state where you can't move forward (complete the operation) or backward
>> (undo it)."
> As noted above, I think this can already happen.

Not with COPY on any modern store; and not in non-crash situations even
with legacy stores.

If your store has that problem with COPY then you ought to fix it; since
such a bug would render your server non-compliant with the IMAP
specification.

> If you look at a move as a macro form of copy/store/delete, then I think
> that an untagged expunge response should fall under the ambiguity rule
> of RFC3501 section 5.5. and should not be allowed.

You can't simultaneously define MOVE as a pipeline and as a command.  RFC
3501 section 5.5 refers to a pipeline, not to a command.

With a command, the rules on RFC 3501 7.4.1 apply.  MOVE is not a FETCH,
STORE, or SEARCH; therefore untagged EXPUNGE is permitted.

However, as in COPY, the only way that untagged EXPUNGE in MOVE can work
is as I stipulated; having effect AFTER the execution of a MOVE and not
impacting the sequence as processed by MOVE.  Since MOVE does its own
expunging, this in turn means that any untagged EXPUNGE has to refer to
the MSN state after the expunges by MOVE.

> A per message response would remove the ambiguity.

No.  The effect of any external initiated EXPUNGE MUST take effect after
the execution of a MOVE.  It can not be before or during.

If the messages expunged by MOVE were announced with untagged EXPUNGE
(which results in considerable unnecessary chattiness), that would still
not change the requirement that external initiated EXPUNGE has to happen
after the MOVE expunges.  Don't forget the UIDPLUS return value, which
is carried in the tagged OK and thus is after any of these.

Thus, announcing the effect of MOVE with untagged EXPUNGE adds chattiness
and leads server implementors to think that they can do a defective
implementation.  Better not to have it at all and make it be impossible to
do it wrong.

This stuff is complicated.  Getting it done right requires working out all
the possible scenarios and multi-access timings.

> I don't think it's a kludge, and I don't think it's only a benefit to a
> client by saving RTTs.  I think it puts message management more in line
> with the way a user would think it should work.

A user doesn't give a damn whether the "move" button does one IMAP command
or 69 IMAP commands.

The only people who care are server authors, who want to kludge around the
deficiencies of their message stores, and perhaps client authors who want
to save an RTT.

There is no reason, other than a careless mail store implementation, that
a MOVE should be faster than a COPY.

A client implementor would be better off with improvements to the
pipelining facility instead of one-off kludge commands that duplicate a
pipeline but not really since nudge-nudge-wink-wink it's trying to sneak
in file renames for Mail.

> Most clients I've tried have a Move command. When I move a message to a
> new folder, the client does a copy and store, leaving the expunge for
> later.  The result is I still have the message in the folder I just
> tried to "move" it out of.

Perhaps that is something that you should take up with the maintainer of
the client.

In the case of the "save" command with delete in Pine/Alpine, that
behavior is quite intentional.  It is not documented as a move, and the
normal way for a person to do a "save without delete" is to do a save and
then an undelete.

I know of at least one other client in which that behavior is intentional;
the whole point being that nothing other than an expunge can ever
permanently remove a message from the mailbox.

But that is all documented in those clients; and if there was an IMAP MOVE
command that did the expunge those clients would not use it.  If they
wanted an expunge they would do it.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

