MBOX-Line: From mrc+imap at panda.com  Sun Aug 29 11:31:58 2010
To: imap-protocol@u.washington.edu
From: Mark Crispin <mrc+imap@panda.com>
Date: Fri Jun  8 12:34:45 2018
Subject: [Imap-protocol] "Multiple Commands in Progress" real life support
	and ambiguity questions
In-Reply-To: <4C7A2AB6.6070405@lists.grepular.com>
References: <4C7A2AB6.6070405@lists.grepular.com>
Message-ID: <alpine.OSX.2.00.1008291038500.370@hsinghsing.panda.com>

Few, if any, IMAP servers will execute commands out of order.

Nor has there ever been much of a use case for executing commands out of
order.  The claimed use case ~25 years ago was that SEARCH may take a
while because it may require the operator to mount a 9-track archive tape
to get the data on large archives.

That claim was demonstrable nonsense even back then.  But, even back then,
the silly "IMAP will die unless it has this feature I want" argument (it's
like calling a political opponent "bigot" or "mean-spirited") was babbled
just as much as it is today.

The valid use case for multiple commands in progress was, and remains,
command pipelining in order to save RTTs.  Pipelining is particularly
important in SMTP, but there are some cases where IMAP clients can benefit
from it.  In pipelining, commands are executed in the order in the
pipeline.

There is no particular requirement for an IMAP server to do anything to
support pipelining other than it MUST NOT clear its command input buffer
between commands.  [SMTP, on the other hand, has a PIPELINING extension;
and a server which does not have that extension MAY do such clearing.
Some SMTP servers do that as an anti-spam measure, as most spamware SMTP
blasters assume a pipelining server and do not check the extension.]

An IMAP client, on the other hand, MUST follow certain rules if it decides
to pipeline (many clients do not, and thus need not worry about these
rules).  These rules boil down to: "don't do anything whose operation may
differ depending upon the operation of a previous command."

For example, the "FETCH + NOOP + STORE" example that you mentioned is
non-pipelineable because an EXPUNGE event can happen during the NOOP, thus
altering the interpretation of the message sequence in the STORE.

Most pipelining rules involve the possibility of an EXPUNGE event.  A
command that allows an EXPUNGE event must be the last command in any
pipeline that involves message sequences in any command.

Even if a server implemented execute pipelined commands out of order (a
possibility that IMHO was stillborn 20+ years ago), it must nonetheless
result "as if" they were executed in order.  This quickly makes the entire
exercise pointless once you work out the few pipelined cases that can be
done out of order with "as if" results and see that there is no benefit to
doing those cases out of order.

Similarly, command tagging should properly be seen as a way of ensuring
synchronization in pipelined response processing (as opposed to just
counting responses ala SMTP) and NOT as a way to execute commands out of
order.

Although some clients open multiple sessions to the same mailbox on the
same server to do multiple commands at once, there is no benefit to this
practice.  If something causes an IMAP command to bottleneck, that same
something will affect a simultaneous IMAP session from that client to that
same server+mailbox.

Bottom line:

As a server implementor, you can disregard section 5.5 entirely, with the
exception that you MUST NOT clear the command input buffer after reading a
command (because there may be a pipelined command following the current
command).  Section 5.5 should be considered to be solely implementation
advice for clients that wish to pipeline.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

