MBOX-Line: From witold.krecicki at firma.o2.pl  Sat Jun 12 08:10:36 2010
To: imap-protocol@u.washington.edu
From: Witold =?utf-8?q?Kr=C4=99cicki?= <witold.krecicki@firma.o2.pl>
Date: Fri Jun  8 12:34:44 2018
Subject: [Imap-protocol] IMAP MOVE extension
In-Reply-To: <849822B6-69A5-4FEF-8865-BF8DD0D01957@iki.fi>
References: <201006110854.37969.witold.krecicki@firma.o2.pl>
	<201006120958.13649.witold.krecicki@firma.o2.pl>
	<849822B6-69A5-4FEF-8865-BF8DD0D01957@iki.fi>
Message-ID: <201006121710.36845.witold.krecicki@firma.o2.pl>

On Saturday 12 of June 2010 16:58:17 Timo Sirainen wrote:
> On 12.6.2010, at 8.58, Witold Kr?cicki wrote:
> >> So let's say that we punt and say that an error is an error and we'll
> >> break the all-of-nothing guarantee of IMAP.
> > 
> > This guarantee is a myth that is not working in eg. Maildirs.
> > The same problems that appear with COPY or EXPUNGE in those critical
> > situations but nobody is thinking about removing COPY or EXPUNGE from the
> > protocol.
> 
> Since you keep talking about maildirs..:
> 
> Assumptions: Maildir can be accessed mostly without locks. Reading mails
> should never get any locks, and that's also how Dovecot works. Changing
> flags or expunging messages would be possible to do without locks,
> although currently Dovecot doesn't. Assigning UIDs to new messages (i.e.
> making new messages visible) pretty much requires some kind of locking.
> 
> The way I've implemented COPY is that it first link()s files to the
> destination Maildir/tmp/ directory. Then it locks the maildir, assigns
> UIDs to messages, rename()s files from tmp/ to new/ and unlocks. Messages
> won't be visible to other connections until unlock is complete. This means
> that as long as the system doesn't crash, there is a single atomic event
> (unlock) when the messages either become visible or the copy operation
> gets rollbacked. This can fail only if there are non-Dovecot MUAs
> accessing the maildir that aren't using Dovecot's locking, but in normal
> Dovecot-only situations this works perfectly.
> 
> Now, if you wanted to implement MOVE operation using rename(), there is a
> problem: Immediately after you do a rename() the message is gone from the
> source mailbox. If another session tries to open the mail (remember, no
> locks while reading), or for some other reason scans the maildir for what
> messages it contains, it sees that the message is gone and issues EXPUNGE
> to client. There is no way to revert that! And why would the MOVE
> operation fail? 1) COPY fails if one of the messages has already been
> expunged, so MOVE should probably too, and this won't be known beforehand.
> 2) Filesystem quota can cause rename() to fail if it would increase the
> destination directory's size.
> 
> So the difference is that COPY can under normal operating conditions work
> as intended, while MOVE implemented with rename() can't.
Maybe MOVE implemented with rename() can't, but move implemented with link & 
delete (so 'internally' as copy + expunge with a single lock) can.

> > What if the server deletes 3 out of 10 messages and then is struck by a
> > filesystem failure (cannot delete remaining 7, cannot undelete of the
> > first 3)? This situation is (formally) not covered in the protocol,
> > server cannot send untagged EXPUNGE responses for those 3 messages
> > (because then it would have to send OK) and it cannot send OK because
> > EXPUNGE after a success removes ALL messages with \Deleted flags...
> 
> I don't think that's a correct assumption.
>
> > It should send NO but then the client would
> > think that no messages were deleted permanently...
> 
> I don't think that's a correct assumption either.
> 
> My server replies to EXPUNGE always with OK, even when there are \Deleted
> messages that weren't expunged because ACLs didn't allow that (it used to
> send NO, but some clients tried to send EXPUNGE internally and kept giving
> popup windows saying the EXPUNGE failed). Clients should use the untagged
> EXPUNGE replies to figure out what happened, the OK reply from EXPUNGE
> doesn't tell anything.
"The EXPUNGE command permanently removes *all* messages that have the \Deleted 
flag set from the currently selected mailbox.". It is not my assumption, it's 
stated in the protocol.

-- 
Witold Kr?cicki

Grupa o2 Sp??ka z o.o., ul. Jutrzenki 177, 02-231 Warszawa,
KRS 0000140518, S?d Rejonowy dla m.st. Warszawy, 
Kapita? zak?adowy 377.298,00 z?., NIP 521-31-11-513

