MBOX-Line: From tss at iki.fi  Mon Oct 31 23:40:57 2011
To: imap-protocol@u.washington.edu
From: Timo Sirainen <tss@iki.fi>
Date: Fri Jun  8 12:34:47 2018
Subject: [Imap-protocol] BODY.PEEK[section]<origin.size> FETCH response
In-Reply-To: <20111101063140.GA5353@brong.net>
References: <8F0DA5FA-FB07-4AFA-9C58-8F0927998343@ghoti.org>
	<alpine.OSX.2.00.1110301823210.9034@hsinghsing.panda.com>
	<CABa8R6uyrHJ6AqQoGfMcLztG-zovaVK4FEFep8ibpYc2-6b7ig@mail.gmail.com>
	<alpine.OSX.2.00.1110311352000.9034@hsinghsing.panda.com>
	<CABa8R6ugaAADsrvaS1Nnau5bi7k9728b+x00NcG1Nbxfk9xwsA@mail.gmail.com>
	<alpine.OSX.2.00.1110311455120.9034@hsinghsing.panda.com>
	<CABa8R6vJGTa+VH-TOgXhCSpX5N1HZTBBqisDhPY9Metz5864JQ@mail.gmail.com>
	<D96D7BAA-894A-41DC-8349-752E737C7A3B@iki.fi>
	<20111101063140.GA5353@brong.net>
Message-ID: <19901681-2C1A-410F-93EE-51F2AF707382@iki.fi>

On 1.11.2011, at 8.31, Bron Gondwana wrote:

> On Tue, Nov 01, 2011 at 08:06:29AM +0200, Timo Sirainen wrote:
>> Dovecot also stores messages with LFs and has no trouble exporting them as if they were CRLFs. I think the only actual (performance) problem with it is .. well, actually the topic of this thread :) A partial fetch from a non-zero offset requires some scanning to find out the LF-only-offset. But luckily all clients just fetch the blocks in increasing order from zero offset, so this isn't such an important problem.
> 
> How do you handle a message with a mix of LF and CRLF in the original?

"Correctly." :)

Basically everywhere there are message (part) sizes, I store the "physical size" (exactly as it is stored in disk, with or without CRs) and the "virtual size" (all LFs converted to CRLFs). If physical size equals to virtual size, I'll do some extra optimizations like being able to seek to wanted offset immediately or use sendfile() to send the message.

Although a mix of LFs and CRLFs in the same message shouldn't normally appear in mail files. Whenever saving messages via Dovecot all of the newlines are changed to either LFs or CRLFs (mail_save_crlf setting, or forced sometimes by a storage backend).


