180 lines
8.1 KiB
Plaintext
180 lines
8.1 KiB
Plaintext
MBOX-Line: From witold.krecicki at firma.o2.pl Fri Jun 11 12:13:56 2010
|
|
To: imap-protocol@u.washington.edu
|
|
From: Witold =?utf-8?q?Kr=C4=99cicki?= <witold.krecicki@firma.o2.pl>
|
|
Date: Fri Jun 8 12:34:43 2018
|
|
Subject: [Imap-protocol] IMAP MOVE extension
|
|
In-Reply-To: <alpine.OSX.2.00.1006111111320.662@hsinghsing.panda.com>
|
|
References: <201006110854.37969.witold.krecicki@firma.o2.pl>
|
|
<201006111953.13490.witold.krecicki@firma.o2.pl>
|
|
<alpine.OSX.2.00.1006111111320.662@hsinghsing.panda.com>
|
|
Message-ID: <201006112113.56572.witold.krecicki@firma.o2.pl>
|
|
|
|
On Friday 11 of June 2010 20:44:11 Mark Crispin wrote:
|
|
> On Fri, 11 Jun 2010, Witold Kr?cicki wrote:
|
|
> > You're saying that you can undo a failed COPY by deleting copied messages
|
|
> > and you can't undo MOVE by moving everything back. Situations in which
|
|
> > COPY using maildirs is not atomic:
|
|
> > 1. During COPYing filesystem error causes switch to read-only mode.
|
|
> > Several messages are copied, and cannot be deleted.
|
|
> > 2. During COPYing server crashes.
|
|
>
|
|
> Yes, maildir has a problem complying with IMAP's atomicity requirements.
|
|
> Maildir is not a particularly good store for IMAP.
|
|
Yet it is the very most popular store for small-to-medium servers.
|
|
|
|
> > I am seeing Your point on how MOVE can fail to be atomic, but those are
|
|
> > exactly the same cases in which COPY can also fail to be atomic, so
|
|
> > where's the point?
|
|
>
|
|
> The point is that it is a mistake to assume that maildir is a prototypical
|
|
> store for IMAP. Any implementation using maildir is going to have
|
|
> problems.
|
|
>
|
|
> MOVE doesn't solve those problems in maildir; it makes them worse. MOVE
|
|
> doesn't solve those problems in other formats either.
|
|
No, MOVE keeps the problems in maildirs at the same level. It does not make
|
|
them worse.
|
|
|
|
> >> Thus, the attractive idea of using rename for a maildir type store
|
|
> >> doesn't work. You have to use something like hardlink+delete. But
|
|
> >> even then you have error conditions. What if, for some reason, the
|
|
> >> delete doesn't work?
|
|
> >
|
|
> > Again, what if after copying half of the messages, for some reason,
|
|
> > delete doesn't work?
|
|
>
|
|
> You're still thinking in terms of maildir as being a prototypical IMAP
|
|
> store.
|
|
And you're still thinking in terms of one and only implementation as the only
|
|
correct one.
|
|
|
|
> In an indexed store, the copied messages do not exist until the index is
|
|
> updated, and the index update is the final step and is atomic. The worst
|
|
> consequence of a crash prior to this is unindexed data occupying
|
|
> inaccessible space that gets recovered in the next audit. Some stores
|
|
> don't even have that problem.
|
|
>
|
|
> MOVE provides no real benefit to such stores; yet those stores are the
|
|
> only ones that can actually implement it properly.
|
|
Database-backed stores can implement it properly and can have performance
|
|
boost.
|
|
|
|
> >>> Also, the implementation of MOVE method in clients is, in most cases,
|
|
> >>> not a problem at all.
|
|
> >>
|
|
> >> "[It is], in most cases, not a problem at all" is the earmark of
|
|
> >> inexperience and not having thought all these issues through. Every
|
|
> >> separate code path for the same functionality adds 2^n complexity in
|
|
> >> overall code path.
|
|
> >
|
|
> > It is already implemented in, at least, Thunderbird (for XAOL-MOVE).
|
|
>
|
|
> Think about what happens when some server implements XAOL-MOVE in a way
|
|
> that is different than Thunderbird expects (since it has only ever seen an
|
|
> AOL server do it).
|
|
That would mean that the specification is not clear enough. As I can see this
|
|
is pretty often in IMAP.
|
|
|
|
> >> MOVE is unimplementable as an atomic operation unless the DB works that
|
|
> >> way.
|
|
> >
|
|
> > In any mail store in which mailbox membership is treated as 'mail label'
|
|
> > it is atomic.
|
|
>
|
|
> And in such a case MOVE has no benefit. There's no quota cost to COPY.
|
|
Yes, there is if there is only one 'mail label' per mail. Copying in this case
|
|
is equivalent to inserting new record and, if handled inproperly, can be non-
|
|
atomic (comparing to fully atomic move)
|
|
|
|
> >> The first thing is to realize that rename can't work. You have to use
|
|
> >> hardlink+delete. And that's non-atomic and has danger points where an
|
|
> >> error can mean that you can't go forward or back.
|
|
> >
|
|
> > In this way COPY is also non-atomic, as mentioned before.
|
|
>
|
|
> That is one of many reasons why Maildir is not a particularly good store
|
|
> for IMAP. Timo's server had to extend Maildir.
|
|
Again, yet it is the very most popular store for small-to-medium servers.
|
|
|
|
> >> Not easily. It's possible with a database if the database is defined in
|
|
> >> a way to make it possible. But in such databases it's a pointless
|
|
> >> tweak over pipelining.
|
|
> >
|
|
> > In such databases it guarantees atomicity - after MOVE and dropped
|
|
> > connection client is SURE that message is either in source or
|
|
> > destination folder, and not in both at the same time.
|
|
>
|
|
> What is the client use case? What clients have expressed a need for move
|
|
> atomicity (which is the only reason not to use pipelining)?
|
|
You are pursuing the goal of all IMAP operations to be atomic, not me. But
|
|
read few lines above - I have written client case (specifically on weak
|
|
connection clients such as GPRS/EDGE).
|
|
|
|
> So far, you've been talking about server-based benefits, which I (and
|
|
> others) have shown are more illusory than real. Adding MOVE will/can not
|
|
> fix a slow server implementation of COPY.
|
|
No. Only You have tried to show me that server-based benefits are illusory, and
|
|
have shown no arguments to that except for examples from exisiting
|
|
implementation.
|
|
|
|
> Consider, in particular, a commercial client product in which implementing
|
|
> this option costs real money and adds a support window of vulnerability
|
|
> (suppose the server advertises XAOL-MOVE but the server implementation is
|
|
> buggy?).
|
|
>
|
|
> These are REAL problems. Right now, as we speak, the IMAP world faces
|
|
> client support issues because of servers that sends broken ENVELOPE and
|
|
> BODYSTRUCTURE data. And, invariably, the answer is "it must be a bug in
|
|
> the client since it works fine with Outlook."
|
|
>
|
|
> Multiple client code paths depending upon server is not a good thing.
|
|
Well, it sounds pretty weird hearing it from You. You have created a protocol
|
|
that, by design, supported extensions. And the result is different capabilities
|
|
of different implementations
|
|
> >> John is a smart guy, but I'll bet that I can discover atomicity and/or
|
|
> >> error recovery flaws in AOL's move functionality.
|
|
> >
|
|
> > UPDATE messages set folder='Trash' where uid in ... - where is the
|
|
> > atomicity flaw here (assuming that database is ACID)?
|
|
>
|
|
> If AOL uses a database that permits that kind of operation, then there is
|
|
> no reason why COPY can not also work that way.
|
|
In this example COPY would create a need to insert new records, MOVE is just
|
|
update. How could COPY work that way?
|
|
|
|
> There is no server benefit. MOVE does not solve the problem of servers
|
|
> with slow COPY. It does create the temptation for server implementors
|
|
> with slow COPY to create an unsafe, non-atomic, implementation of MOVE
|
|
> (e.g., using rename()).
|
|
So there is server benefit (as shown above)
|
|
|
|
> The only benefit is client benefit, and specifically that of an atomic
|
|
> operation. But no use case has been given for it.
|
|
Read above.
|
|
|
|
> This is an "attractive nuisance". It's one of those ideas that sounds
|
|
> really good on paper until you study the unintended consequences.
|
|
What consequences?
|
|
|
|
> >> It's easy enough to say "that's alright, we don't care about these edge
|
|
> >> cases" in a private implementation.
|
|
> >> The problem is that it isn't alright, and you have to care, in a public
|
|
> >> specification.
|
|
> >
|
|
> > You are assuming that it's impossible to implement this operation
|
|
> > properly (eg. atomic), and this is obviously false statement.
|
|
>
|
|
> That sounds like inexperience on your part.
|
|
And that sounds like total inability to listen to your counterpart on Your
|
|
part.
|
|
I have gave you a good example of mail store (that I have seen) in which MOVE
|
|
is atomic and in which MOVE is much faster than COPY - and you have ignored
|
|
it.
|
|
--
|
|
Witold Kr?cicki
|
|
|
|
Grupa o2 Sp??ka z o.o., ul. Jutrzenki 177, 02-231 Warszawa,
|
|
KRS 0000140518, S?d Rejonowy dla m.st. Warszawy,
|
|
Kapita? zak?adowy 377.298,00 z?., NIP 521-31-11-513
|
|
|