155 lines
6.6 KiB
Plaintext
155 lines
6.6 KiB
Plaintext
MBOX-Line: From mrc+imap at panda.com Fri Jun 11 11:44:11 2010
|
|
To: imap-protocol@u.washington.edu
|
|
From: Mark Crispin <mrc+imap@panda.com>
|
|
Date: Fri Jun 8 12:34:43 2018
|
|
Subject: [Imap-protocol] IMAP MOVE extension
|
|
In-Reply-To: <201006111953.13490.witold.krecicki@firma.o2.pl>
|
|
References: <201006110854.37969.witold.krecicki@firma.o2.pl>
|
|
<201006111817.39761.witold.krecicki@firma.o2.pl>
|
|
<alpine.OSX.2.00.1006110938590.662@hsinghsing.panda.com>
|
|
<201006111953.13490.witold.krecicki@firma.o2.pl>
|
|
Message-ID: <alpine.OSX.2.00.1006111111320.662@hsinghsing.panda.com>
|
|
|
|
On Fri, 11 Jun 2010, Witold Kr?cicki wrote:
|
|
>> Options are not a good thing
|
|
> Looking at number of possible extensions to imap and number of ones supported
|
|
> by popular IMAP servers
|
|
|
|
The vast majority of which were mistakes and should not have happened.
|
|
|
|
Yes, there are open source servers which implement every goofy extension
|
|
under the sun. But when you look at the Real World and which extensions
|
|
have a business case, the set becomes very small indeed.
|
|
|
|
> Do any of IMAP servers implement COPY as atomic?
|
|
|
|
All compliant IMAP servers do.
|
|
|
|
> You're saying that you can undo a failed COPY by deleting copied messages and
|
|
> you can't undo MOVE by moving everything back. Situations in which COPY using
|
|
> maildirs is not atomic:
|
|
> 1. During COPYing filesystem error causes switch to read-only mode. Several
|
|
> messages are copied, and cannot be deleted.
|
|
> 2. During COPYing server crashes.
|
|
|
|
Yes, maildir has a problem complying with IMAP's atomicity requirements.
|
|
Maildir is not a particularly good store for IMAP.
|
|
|
|
> I am seeing Your point on how MOVE can fail to be atomic, but those are
|
|
> exactly the same cases in which COPY can also fail to be atomic, so where's
|
|
> the point?
|
|
|
|
The point is that it is a mistake to assume that maildir is a prototypical
|
|
store for IMAP. Any implementation using maildir is going to have
|
|
problems.
|
|
|
|
MOVE doesn't solve those problems in maildir; it makes them worse. MOVE
|
|
doesn't solve those problems in other formats either.
|
|
|
|
>> Thus, the attractive idea of using rename for a maildir type store doesn't
|
|
>> work. You have to use something like hardlink+delete. But even then you
|
|
>> have error conditions. What if, for some reason, the delete doesn't work?
|
|
> Again, what if after copying half of the messages, for some reason, delete
|
|
> doesn't work?
|
|
|
|
You're still thinking in terms of maildir as being a prototypical IMAP
|
|
store.
|
|
|
|
In an indexed store, the copied messages do not exist until the index is
|
|
updated, and the index update is the final step and is atomic. The worst
|
|
consequence of a crash prior to this is unindexed data occupying
|
|
inaccessible space that gets recovered in the next audit. Some stores
|
|
don't even have that problem.
|
|
|
|
MOVE provides no real benefit to such stores; yet those stores are the
|
|
only ones that can actually implement it properly.
|
|
|
|
>>> Also, the implementation of MOVE method in clients is, in most cases, not
|
|
>>> a problem at all.
|
|
>> "[It is], in most cases, not a problem at all" is the earmark of
|
|
>> inexperience and not having thought all these issues through. Every
|
|
>> separate code path for the same functionality adds 2^n complexity in
|
|
>> overall code path.
|
|
> It is already implemented in, at least, Thunderbird (for XAOL-MOVE).
|
|
|
|
Think about what happens when some server implements XAOL-MOVE in a way
|
|
that is different than Thunderbird expects (since it has only ever seen an
|
|
AOL server do it).
|
|
|
|
Experience says that this is a real (and quite expensive) problem.
|
|
|
|
>> MOVE is unimplementable as an atomic operation unless the DB works that
|
|
>> way.
|
|
> In any mail store in which mailbox membership is treated as 'mail label' it is
|
|
> atomic.
|
|
|
|
And in such a case MOVE has no benefit. There's no quota cost to COPY.
|
|
|
|
>> The first thing is to realize that rename can't work. You have to use
|
|
>> hardlink+delete. And that's non-atomic and has danger points where an
|
|
>> error can mean that you can't go forward or back.
|
|
> In this way COPY is also non-atomic, as mentioned before.
|
|
|
|
That is one of many reasons why Maildir is not a particularly good store
|
|
for IMAP. Timo's server had to extend Maildir.
|
|
|
|
>> Not easily. It's possible with a database if the database is defined in a
|
|
>> way to make it possible. But in such databases it's a pointless tweak
|
|
>> over pipelining.
|
|
> In such databases it guarantees atomicity - after MOVE and dropped connection
|
|
> client is SURE that message is either in source or destination folder, and not
|
|
> in both at the same time.
|
|
|
|
What is the client use case? What clients have expressed a need for move
|
|
atomicity (which is the only reason not to use pipelining)?
|
|
|
|
So far, you've been talking about server-based benefits, which I (and
|
|
others) have shown are more illusory than real. Adding MOVE will/can not
|
|
fix a slow server implementation of COPY.
|
|
|
|
Consider, in particular, a commercial client product in which implementing
|
|
this option costs real money and adds a support window of vulnerability
|
|
(suppose the server advertises XAOL-MOVE but the server implementation is
|
|
buggy?).
|
|
|
|
These are REAL problems. Right now, as we speak, the IMAP world faces
|
|
client support issues because of servers that sends broken ENVELOPE and
|
|
BODYSTRUCTURE data. And, invariably, the answer is "it must be a bug in
|
|
the client since it works fine with Outlook."
|
|
|
|
Multiple client code paths depending upon server is not a good thing.
|
|
|
|
>> John is a smart guy, but I'll bet that I can discover atomicity and/or
|
|
>> error recovery flaws in AOL's move functionality.
|
|
> UPDATE messages set folder='Trash' where uid in ... - where is the atomicity
|
|
> flaw here (assuming that database is ACID)?
|
|
|
|
If AOL uses a database that permits that kind of operation, then there is
|
|
no reason why COPY can not also work that way.
|
|
|
|
There is no server benefit. MOVE does not solve the problem of servers
|
|
with slow COPY. It does create the temptation for server implementors
|
|
with slow COPY to create an unsafe, non-atomic, implementation of MOVE
|
|
(e.g., using rename()).
|
|
|
|
The only benefit is client benefit, and specifically that of an atomic
|
|
operation. But no use case has been given for it.
|
|
|
|
This is an "attractive nuisance". It's one of those ideas that sounds
|
|
really good on paper until you study the unintended consequences.
|
|
|
|
>> It's easy enough to say "that's alright, we don't care about these edge
|
|
>> cases" in a private implementation.
|
|
>> The problem is that it isn't alright, and you have to care, in a public
|
|
>> specification.
|
|
> You are assuming that it's impossible to implement this operation properly
|
|
> (eg. atomic), and this is obviously false statement.
|
|
|
|
That sounds like inexperience on your part.
|
|
|
|
-- Mark --
|
|
|
|
http://panda.com/mrc
|
|
Democracy is two wolves and a sheep deciding what to eat for lunch.
|
|
Liberty is a well-armed sheep contesting the vote.
|