MBOX-Line: From Lynn at BusCom.net  Sun Jun  5 12:58:45 2011
To: imap-protocol@u.washington.edu
From: "Lynn W. Taylor" <Lynn@BusCom.net>
Date: Fri Jun  8 12:34:46 2018
Subject: [Imap-protocol] Where to start?
In-Reply-To: <201106051924.p55JOwpG003543@mxout12.cac.washington.edu>
References: <4DEBB242.2090200@BusCom.net>
	<4DEBB58D.3090101@logicprobe.org>	<4DEBBAB3.2030301@BusCom.net>
	<4DEBBEC1.3080600@BusCom.net>
	<201106051924.p55JOwpG003543@mxout12.cac.washington.edu>
Message-ID: <4DEBDFF5.9000301@BusCom.net>

Pete,

Thank you very much.

I'm not starting with a deep understanding of IMAP.  I do know how to 
spell it.  This also isn't my first time at the rodeo.

The mailstore is in RFC-2822 format.  There is a UID, and it's cast in 
jello -- it can be changed so that the POP3 server uses an IMAP 
compatible UIDL.  I think I can keep the UIDs unique and ascending as 
required in the RFC.

Don't have the parser, so #1 is very useful.

On #3, I'm not sure how the flags are used, so it's hard to make an 
intelligent decision (other than to try and design something pretty 
open).  Something in my head keeps telling me that they should be stored 
as a set of booleans and translated to ASCII when/as needed.  The other 
thing in my head keep saying that I don't understand it enough to know that.

On #4, I have memory constraints, especially if I need a lot of folders 
open at once.

I have a good command parser, the listener is fairly trivial, and I 
think most of the authentication code is done or nearly so in other 
parts of the system.

On #9, in my environment making COPY atomic isn't a big deal, but the 
fact that it needs to be fully atomic is pretty useful.

Search (#10) is the one that worries the most.  Some of that is the same 
reason that COPY doesn't bother me.  Glad it can be left to last, and 
wish it could be left out.

I'm not in a position to specify clients.  I'll likely do my initial 
testing with Thunderbird unless someone warns me that it's a poor choice.

-- Lynn

On 6/5/2011 12:24 PM, Pete Maclean wrote:
> At 01:37 PM 6/5/2011, Lynn W. Taylor wrote:
>> P.S. I'm not looking for a complete roadmap, I can fill in most of 
>> the detail.
>>
>> Tags and UIDs are not a big deal.
>>
>> What I want to do is get enough IMAP working that I can start testing 
>> with a client, even if the server isn't 100% practical, instead of 
>> implementing some critical command last, and finding out that I made 
>> some critical mistake very early in the process.
>
> Here are 10 tips based on my own experience.  From what you say, some 
> of these may not be relevant to your case but I will be inclusive.  I 
> am also assuming both that you are not starting out intimately 
> familiar with the protocol and that messages are stored in 
> RFC2822-MIME format.  A general guideline is that, while books may 
> help you get a feeling for IMAP -- they did for me, when actually 
> implementing it is vital to pay full attention to the relevant RFCs.
>
> 1). You will need a fairly sophisticated message parser, notably for 
> extracting MIME parts and generating BODYSTRUCTURE information.  If 
> you do not start out with such a parser ready-made then you might well 
> choose to begin by building one.
>
> 2). If your mailstore does not have UIDs (or something that can be 
> suitably mapped into UIDs) then decide how you will create and store 
> UIDs.  Incidentally IMAP specs refer to a UID command; I find it 
> easier to view this as a command qualifier.
>
> 3). Decide what permanents flags to support and how.  Decide what 
> hierarchy delimiter to use; I believe that most servers use slash, 
> backslash or dot but I have not seen a problem using even a 
> non-graphic character such as Rubout (ASCII 127).  By the way, I think 
> that RFC 3501 does not make this clear but the hierarchy delimiter 
> must be a 7-bit ASCII character.
>
> 4). Decide what data to keep in memory for each message in an open 
> mailbox.  (By "open", I mean one for which a session is in progress, 
> that is opened as the result of a SELECT or EXAMINE command.)  A 
> likely minimum would be UID, IMAP message flags, message size and, of 
> course, a reference to the message itself.  I am assuming that this 
> information is indexed by message number and hence that that is not 
> explicitly stored.  You may find reason to add more data such as a 
> flag indicating if the message is matched in a current SEARCH.
>
> 5). I don't recall if I did this when I wrote my server, but I would 
> probably start now by implementing the LIST command.  This involves a 
> number of elements including modified UTF-7 encoding and understanding 
> the distinction between the two arguments of the command.  I wrote 
> unit test programs for my backend LIST code (and that of several other 
> commands) and I highly recommend doing the same.  These were simple 
> console programs each of which took a set of IMAP command arguments as 
> input and emitted the results as output.  In the early stages, testing 
> with actual IMAP connections (be they from a client or a test program) 
> is unnecessarily awkward.  (I am assuming here that there are other 
> means, outside of the IMAP server, to create mailboxes; if this does 
> not hold then you may want to implement CREATE first.)
>
> 6). Next I would implement SELECT and EXAMINE.  There are intricacies 
> here such as checking what rights the user in question has to the 
> mailbox and ensuring coordination among threads (and, if relevant, 
> with other programs) that are simultaneously accessing the mailbox.
>
> 7). Next I would implement FETCH.  This is a biggie because of the 
> range of data items, macros, dealing with literals, and so on.
>
> 8). Perhaps at about this point, I would work on the command processor 
> (recognising commands, checking mode, parsing arguments, etc.) and, 
> unless I had a ready-made one, the listener.  This allows one to start 
> testing online.  I suggest implementing the command processor at a 
> fairly late stage because, for some commands, argument-parsing is 
> tricky and you will likely find it easier once you have implemented 
> the commands themselves and have a good understanding of how they work 
> -- although the critical thing to do here is to follow the RFC's 
> grammar very closely.  AUTHENTICATION can be done at any stage since 
> you can fake it until such point as you choose to made it real.
>
> 9). Next I would take care of the "minor" commands:  CREATE, LSUB, 
> NOOP, CLOSE, STORE, CAPABILITY, APPEND, etc.  But do not think of COPY 
> as minor -- it can be a substantial job because of the required 
> atomicity.
>
> 10). I would leave SEARCH for last.  When I wrote my server, I left it 
> to last because I feared it would be the most difficult to do.  I was 
> wrong; it proved much easier than I had expected but it is, 
> nevertheless, a hefty amount of work.  It may also be fair to say that 
> it is the least critical command but, all the same, it has to be done 
> and done completely (unless of course your client is intended to work 
> with a strictly limited set of clients that do not use it.)  And 
> testing SEARCH can be hard because few clients implement it well; 
> indeed, some do not offer it at all.  Of course, if your mailstore 
> manager already has search capabilities then you may be able to 
> leverage those and make the job a snap.
>
> _______________________________________________
> Imap-protocol mailing list
> Imap-protocol@u.washington.edu
> http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol
>

