146 lines
6.6 KiB
Plaintext
146 lines
6.6 KiB
Plaintext
MBOX-Line: From mrc+imap at panda.com Fri Jun 11 22:53:55 2010
|
|
To: imap-protocol@u.washington.edu
|
|
From: Mark Crispin <mrc+imap@panda.com>
|
|
Date: Fri Jun 8 12:34:44 2018
|
|
Subject: [Imap-protocol] IMAP MOVE extension
|
|
In-Reply-To: <201006112352.05642.witold.krecicki@firma.o2.pl>
|
|
References: <201006110854.37969.witold.krecicki@firma.o2.pl>
|
|
<201006112113.56572.witold.krecicki@firma.o2.pl>
|
|
<alpine.OSX.2.00.1006111214020.662@hsinghsing.panda.com>
|
|
<201006112352.05642.witold.krecicki@firma.o2.pl>
|
|
Message-ID: <alpine.OSX.2.00.1006112111250.662@hsinghsing.panda.com>
|
|
|
|
On Fri, 11 Jun 2010, Witold Kr?cicki wrote:
|
|
>> IMAP has no guaranteed means of removing a message. Study the semantics
|
|
>> of STORE and EXPUNGE carefully. MOVE, on the other hand, requires that
|
|
>> guarantee; and does not have the semantics of STORE and EXPUNGE.
|
|
> I really don't get the point, please be more specific.
|
|
|
|
Neither STORE, nor EXPUNGE, promise that they will store or expunge
|
|
anything. They do, or not do, whatever seems fit according to their
|
|
discretion. Generally, what they decide to more corresponds to what the
|
|
user intended. But there's no promises.
|
|
|
|
Because of this, both STORE and EXPUNGE send back untagged responses
|
|
stating what they actually did. The client uses this to update its cached
|
|
state of what is on the server. If it were not for these untagged
|
|
responses, the client wouldn't have a clue as to what is going on.
|
|
|
|
There are certain circumstances, such as when closing a mailbox in a
|
|
client that does not cache offline state, that the client does not care
|
|
what the server did beyond a vague presumption that that the server did
|
|
more or less reasonably was what the user wanted. For this reason, there
|
|
STORE's .SILENT form, and the CLOSE command, exist.
|
|
|
|
A successful MOVE, on the other hand, is an unconditional guarantee that
|
|
all the requested message(s) were moved. But it it? Similarly, a failed
|
|
MOVE ought to be an unconditional guarantee that none of the requested
|
|
messages(s) were moved. But, once again, is it?
|
|
|
|
What happens when there is an error in the middle of the move? Is there a
|
|
state in which some messages were moved and others were not? Is there a
|
|
state in which some messages appear in both places (so it was a copy after
|
|
all)? Is there a state in which some messages vanished completely?
|
|
|
|
The only way that you can prevent all of these states is with a
|
|
simultaneous commit in which a failure of either side (remove from source,
|
|
insert in destination) completely undoes everything. Locking by itself
|
|
isn't good enough; locking only prevents external races. You still have
|
|
a chicken-and-egg situation in which neither side can safely commit
|
|
without confirmation that the other side has committed.
|
|
|
|
There are ways to do it. But these are not simple nor commonly
|
|
understood. If I talk about this stuff with most programmers their eyes
|
|
glaze over.
|
|
|
|
Not even the (much simpler) synchronization timing requirements of IMAP
|
|
are well-implemented. Timo's server and my Panda server are the only ones
|
|
that do it correctly in all circumstances (I'm hoping to add my new server
|
|
to the list).
|
|
|
|
So let's say that we punt and say that an error is an error and we'll
|
|
break the all-of-nothing guarantee of IMAP.
|
|
|
|
There's now the problem of how what was actually done is communicated to
|
|
the client. The untagged responses from FETCH and EXPUNGE won't do it.
|
|
|
|
So now you have to add a whole new framework of new untagged responses to
|
|
your "simple" MOVE command to convey all the possible error conditions.
|
|
And it's just not what happens in your server. It's what can happen in
|
|
any server.
|
|
|
|
That is the problem with response to complex situations with simple
|
|
solutions; it backfires.
|
|
|
|
> COPY is by definition a slow operation. Using hardlinks etc. is a performance
|
|
> hack, not a normal method.
|
|
|
|
Nowhere in the definition of COPY does it say that it is a slow operation.
|
|
I should know. I wrote it.
|
|
|
|
Furthermore, your calling the use of hardlinks a "performance hack" is a
|
|
personal insult to me. I anticipated the use of hardlinks in IMAP from
|
|
the very beginning, influenced by a pioneering mail store of the early
|
|
1980s. I am designing a new store that carries that technique to a new
|
|
level. I already added stubbing to my 2006 store design last year; but
|
|
the new design is far more advanced.
|
|
|
|
Last but not least, it betrays a lack of understanding of how modern
|
|
filesystems work. Rename uses hardlinks internally when the target is on
|
|
the system filesystem (when it isn't, it has to copy+delete). Rename is a
|
|
world of hurt on a filesystem without hardlinks; and ultimately the
|
|
problem is left for the disk auditor to resolve after a crash.
|
|
|
|
> COPY creates NEW entity, MOVE only changes properties of an EXISTING entity.
|
|
|
|
Now you are presupposing that the message itself knows where it is
|
|
located. That's fine if you are developing Gmail. But all the world is
|
|
not Gmail.
|
|
|
|
Nor is that how modern filesystems work. The inode has no clue where it
|
|
is in the filesystem tree.
|
|
|
|
The next step is the abolish the filesystem.
|
|
|
|
> If COPY is implementable then MOVE is implementable. period.
|
|
|
|
Wrong. There are many mail stores in which MOVE is completely
|
|
unimplementable. The only thing that you can do is simulate the effect of
|
|
MOVE by using COPY then delete.
|
|
|
|
> In Maildir it can (and will!) cause indeterminate results to client only in
|
|
> case of catastrophic failure.
|
|
|
|
Maildir has numerous failure modes short of catastrophic failure. Timo
|
|
can go into this in quite a bit of detail.
|
|
|
|
> So will COPY in case of catastrophic failure.
|
|
|
|
Not at all. A mail store can be designed so that there is only a single
|
|
operation: commit updated index. It's much easier to do safe commit of a
|
|
single object than two simultaneous objects.
|
|
|
|
> MOVE creates advantage for any kind of single-labeled flat store.
|
|
|
|
So all the world is all designed around Gmail?
|
|
|
|
> Currently You are the only one more experienced that is trying to explain the
|
|
> complexities of the situation. Most of the people here (I'm silently assuming
|
|
> that they are not all newcomers) agreed that it is in fact a needed feature.
|
|
|
|
Now you're talking like the cretins who changed the formerly predictable
|
|
and deterministic behavior of the motion keys (e.g., CTRL/A, CTRL/E,
|
|
CTRL/N, and CTRL/P) in emacs 23. Apparently it was a "needed feature" to
|
|
make emacs more like a word processor instead of a text editor.
|
|
|
|
It is a typical "response to complex situations with simple solutions that
|
|
backfires." The backfire here is that 34 years worth of scripts no longer
|
|
work correctly, with resulting subtle file corruption depending upon the
|
|
window width.
|
|
|
|
-- Mark --
|
|
|
|
http://panda.com/mrc
|
|
Democracy is two wolves and a sheep deciding what to eat for lunch.
|
|
Liberty is a well-armed sheep contesting the vote.
|