MBOX-Line: From jkt at gentoo.org  Thu May 19 03:48:06 2011
To: imap-protocol@u.washington.edu
From: =?UTF-8?B?SmFuIEt1bmRyw6F0?= <jkt@gentoo.org>
Date: Fri Jun  8 12:34:46 2018
Subject: [Imap-protocol] GMail IMAP: returning BODYSTRUCTURE for embedded
	messages
Message-ID: <4DD4F566.2070607@gentoo.org>

Hi,
I'm glad to see someone from Google on this list. It's really relieving
to be able to report issues directly without resorting to proxying via
friends-at-Google in order to file a decent bugreport.

I'd appreciate if you could investigate your compliance with the format
of BODYSTRUCTURE you generate when working with certain messages, as
described in this messages. The same issue was reported to you through
an internal issue tracker in January 2011 by Henner Zeller on my behalf.

Here's the relevant part of my report, as I sent it on 13th January:

I've been playing with a Qt IMAP client [1] for a few years, and at this
point I'm trying to debug GMail's idea about structures of certain
messages. It would help me tremendously if you could pass this
information to someone working on the GMail IMAP team. I'm absolutely
sure that the behavior I'm seeing is in fact violation of the RFC3501.
In short, it looks like GMail IMAP would occasionally substitute "NIL"
for a proper form of "envelope" (see RFC3501 for details), and also will
not provide BODYSTRUCTURE for a message/rfc822 embedded into an e-mail
and won't allow access to such parts for download. In short, this breaks
all IMAP clients which rely on server-side message parsing, which are
basically any mobile clients or those which support the "download
without attachment" option.

I'm attaching full logs, as well as a source of a sample message which
triggers this behavior (bzip-ed for safe journey via e-mail), but as a
quick visual indication of what is going on, here's a (reformatted for
readability) short summary of the corrupted response:

* 12 FETCH (BODYSTRUCTURE (
            (
             ("TEXT" "PLAIN" ("CHARSET" "iso-8859-2") NIL NIL
              "QUOTED-PRINTABLE" 28 2 NIL NIL NIL)
             ("TEXT" "HTML" ("CHARSET" "iso-8859-2") NIL NIL
              "QUOTED-PRINTABLE" 1707 65 NIL NIL NIL)
             "ALTERNATIVE" ("BOUNDARY"
"----=_NextPart_001_0078_01CBB179.57530990") NIL NIL
            )
            ("MESSAGE" "RFC822" NIL NIL NIL "7BIT" 641 NIL
             ("ATTACHMENT" NIL) NIL)
            ("MESSAGE" "RFC822" NIL NIL NIL "7BIT" 50592 NIL
             ("ATTACHMENT" NIL) NIL)
           "MIXED" ("BOUNDARY"
"----=_NextPart_000_0077_01CBB179.57530990") NIL NIL
           ))


In comparison to GMail's response, this is what Dovecot returns:

* 4 FETCH (BODYSTRUCTURE
        (
         (
          ("text" "plain" ("charset" "iso-8859-2") NIL NIL
           "quoted-printable" 28 2 NIL NIL NIL NIL)
          ("text" "html" ("charset" "iso-8859-2") NIL NIL
           "quoted-printable" 1707 65 NIL NIL NIL NIL)
          "alternative" ("boundary"
"----=_NextPart_001_0078_01CBB179.57530990") NIL NIL NIL
         )
         ("message" "rfc822" NIL NIL NIL "7bit" 641
          ("Sat, 8 Jan 2011 14:16:36 +0100" "Subj 2"
           (("Some Name, SOMECOMPANY" NIL "recipient" "example.com"))
           (("Some Name, SOMECOMPANY" NIL "recipient" "example.com"))
           (("Some Name, SOMECOMPANY" NIL "recipient" "example.com"))
           (("Recipient" NIL "example" "gmail.com"))
           NIL NIL NIL NIL
          )
          ("text" "plain" ("charset" "iso-8859-2") NIL NIL
           "quoted-printable" 185 18 NIL NIL ("cs") NIL)
          31 NIL ("attachment" NIL) NIL NIL
         )
         ("message" "rfc822" NIL NIL NIL "7bit" 50592
          ("Sat, 8 Jan 2011 13:58:39 +0100" "Subj 1"
           (("Some Name, SOMECOMPANY" NIL "recipient" "example.com"))
           (("Some Name, SOMECOMPANY" NIL "recipient" "example.com"))
           (("Some Name, SOMECOMPANY" NIL "recipient" "example.com"))
           (("Recipient" NIL "example" "gmail.com"))
           NIL NIL NIL NIL
          )
          (
           ("text" "plain" ("charset" "iso-8859-2") NIL NIL
            "quoted-printable" 4296 345 NIL NIL NIL NIL)
           ("text" "html" ("charset" "iso-8859-2") NIL NIL
            "quoted-printable" 45069 1295 NIL NIL NIL NIL)
           "alternative" ("boundary"
"----=_NextPart_000_0073_01CBB179.57530990") NIL ("cs") NIL
          ) 1669 NIL ("attachment" NIL) NIL NIL)
         "mixed" ("boundary"
"----=_NextPart_000_0077_01CBB179.57530990") NIL ("cs") NIL
        ))

The first issue is that the whole envelope field in both embedded
rfc/822 objects is substituted by NIL. The second issue is much more
serious, though -- the response is completely missing the actual
structure of these two attached messages (yes, the BODYSTRUCTURE should
really include something very similar to the BODYSTRUCTURE for the
embedded message). The most serious problem, though, is that GMail's
IMAP server won't allow fetching the attached messages via standard IMAP
FETCH command -- please see the relevant part of the log I'm attaching.

As a side note, it looks like the web interface doesn't display messages
which are forwarded as an attachment (ie. the original message is an
attachment, most clients would show that as an .eml file). Maybe it
would be worth fixing that, too.

If you are able to pass this report to someone who could help and fix
the problem, please feel free to ask them to contact me if they need
more information.

Have fun,
Jan

[1] http://trojita.flaska.net/

-- 
Trojita, a fast e-mail client -- http://trojita.flaska.net/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gmail-testcase-bodystructure.eml.bz2
Type: application/x-bzip2
Size: 1579 bytes
Desc: not available
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20110519/03469609/attachment.bz2>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: communication.log
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20110519/03469609/attachment.log>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 270 bytes
Desc: OpenPGP digital signature
URL: <http://mailman13.u.washington.edu/pipermail/imap-protocol/attachments/20110519/03469609/attachment.sig>
