MBOX-Line: From mrc+imap at panda.com  Mon Jun 14 14:29:05 2010
To: imap-protocol@u.washington.edu
From: Mark Crispin <mrc+imap@panda.com>
Date: Fri Jun  8 12:34:44 2018
Subject: [Imap-protocol] MOVE is a pipeline
In-Reply-To: <201006142256.29781.witold.krecicki@firma.o2.pl>
References: <201006110854.37969.witold.krecicki@firma.o2.pl>
	<alpine.OSX.2.00.1006141208460.662@hsinghsing.panda.com>
	<alpine.OSX.2.00.1006141310330.662@hsinghsing.panda.com>
	<201006142256.29781.witold.krecicki@firma.o2.pl>
Message-ID: <alpine.OSX.2.00.1006141411190.662@hsinghsing.panda.com>

On Mon, 14 Jun 2010, Witold Kr?cicki wrote:
> COPY (in Maildir) can fail partially (not all messages copied).

A raw Maildir implementation without extensions has that flaw, yes.

That's why IMAP servers don't use raw Maildir.  Raw Maildir simply does
not have the semantics to support IMAP.  IMAP servers that offer Maildir
use some extension, typically an index, that is not in the Maildir
specification.

That index solves the COPY problem.  The MOVE problem, however, is quite a
bit more complex.

> EXPUNGE can
> fail that way (not all messages expunged).

Wrong.  EXPUNGE is defined so that any operation (including no messages
expunged) is a success.

What EXPUNGE does, or does not do, is defined to be completely
non-deterministic by the client in advance.  Otherwise, there would be no
need for the untagged EXPUNGE response.

> It's all a matter of proper
> implementation to not get a chance of a partial failure.

You can't do it with flat files.

You can't do it with Maildir.

You can do it with the right type of database.  But then you can fix the
database so that it does not have a performance proble with COPY.

> You are forcing an opinion (on everyone) that this pipeline always exists AND
> that MOVE command cannot be made atomic, and it's certainly not true.

It can not be made atomic with flat files or Maildir.  Both require
semantics from the host filesystem that do not exist.

It can be done with a database architecture that supports the needed
semantics.  But by the same argument the database could be designed so
that move becomes pointless.

> You are creating those error cases,  based on Your interpretations. I'm able
> to think of an implementation (using Maildirs) in which MOVE command is either
> full success or full failure, and it's fully atomic. You can't.

Either you are lying, or you are so inexperienced that you do not yet
understand the types of timing races that are possible in filesystems.

The only way to do it in Maildir is to use a database that has
simultaneous index update.  But then it is not Maildir; it is a database
that happens to have files that look like a Maildir.  And any access of
the Maildir as a Maildir will encounter the races and potentially
invalidate the database.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
