MBOX-Line: From MRC at CAC.Washington.EDU  Tue Mar 27 10:57:02 2007
To: imap-protocol@u.washington.edu
From: Mark Crispin <MRC@CAC.Washington.EDU>
Date: Fri Jun  8 12:34:39 2018
Subject: [Imap-protocol] MULTIAPPEND practice
In-Reply-To: <673436100.23171175014561192.JavaMail.root@dogfood.liquidsys.com>
References: <673436100.23171175014561192.JavaMail.root@dogfood.liquidsys.co m>
Message-ID: <alpine.WNT.0.83.0703271036260.5820@Tomobiki-Cho.CAC.Washington.EDU>

On Tue, 27 Mar 2007, Dan Karp wrote:
> Are there any limits which might prevent a client from sending a single 
> 1GB MULTIAPPEND request (for example, 100,000 10K messages to be added 
> to the same quotaless mailbox), especially if LITERAL+ is available? 
> With the "atomic transaction" requirement, this could potentially be a 
> headache.

There are no limits in the protoocol.

Servers are free to impose any limit they want:
 	C: tag APPEND blurdybloop {2345765221}
 	S: tag NO Our policy prohibits messages that large

 	C: tag APPEND blurdybloop {10000}
 	S: + Ready for message
 	C: [10,000 bytes of message...] {10000}
 	S: + Ready for another message
 	C: [10,000 bytes of message...] {10000}
 	S: tag NO Disk quota exceeded

Note that in both cases, nothing is appended.

I don't think that a client should use LITERAL+ with MULTIAPPEND.  Both 
LITERAL+ and MULTIAPPEND are optimizations to reduce RTTs, but MULTIAPPEND 
is better for the APPEND case.  Although it sounds attractive to eliminate 
all per-message RTTs by using both simultaneously (and IMAP specifically 
does NOT preclude doing so), any failure will cause both client and server 
to go to a lot of work for nothing.

Put another way, the "headache" you note is real, but only if the client 
puts itself in that position.  IMHO, one RTT/message is OK since that RTT 
actually provides a benefit.  The problem with base specification APPEND 
is that there were two RTTs/message plus a mailbox open/lock operation, 
and no way to undo an extended operation if there is a failure.

> In practice, do real-world IMAP clients attempt to break such large 
> potential MULTIAPPENDs into smaller batches, or do they rely on the 
> server to process the APPENDed messages as they go?

Neither, I hope.

In general, it is quite unlikely that any client is going to APPEND 
100,000 10K messages other than some process for migrating mailboxes 
(which is a different issue from client operation).  I don't think that a 
client should attempt such a thing without external knowledge that the 
server will accept it.

Breaking up an APPEND like that is acceptable only if you do not want 
atomicity; that is, you *want* a partial APPEND on failure *AND* perhaps 
other intervening APPENDs happening (such as new mail delivery).

The problem is, a lot of kidlets aren't likely to understand the issue, 
and specifically why atomicity is a feature and thus should not be 
sabotaged.  There are a lot of kidlets out there who think that they are 
programmers; and alas a lot of companies are hiring them.  I just wasted 
hours yesterday because McAfee hired an obvious kidlet...  ;-(

> Is it acceptable for a server to queue the APPENDed messages and process 
> them once the end of the MULTIAPPEND is reached?

Yes.  That's how most servers work.  Remember that the final OK determines 
success/failure for the entire MULTIAPPEND.

> Imap-protocol mailing list
> Imap-protocol@u.washington.edu
> https://mailman1.u.washington.edu/mailman/listinfo/imap-protocol
>

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

