167 lines
7.6 KiB
Plaintext
167 lines
7.6 KiB
Plaintext
MBOX-Line: From asuth at mozilla.com Tue Sep 3 03:53:00 2013
|
|
To: imap-protocol@u.washington.edu
|
|
From: Andrew Sutherland <asuth@mozilla.com>
|
|
Date: Fri Jun 8 12:34:51 2018
|
|
Subject: [Imap-protocol] Best way to support HTTP PUT based 'push'
|
|
notifications from IMAP?
|
|
Message-ID: <5225BF8C.7060602@mozilla.com>
|
|
|
|
On devices that either have difficulty maintaining persistent TCP
|
|
connections or have power concerns about doing so, maintaining an IMAP
|
|
connection for IDLE/NOTIFY purposes is undesirable and it would be great
|
|
if the IMAP server could generate a notification via other means.
|
|
|
|
For the Firefox OS e-mail client, one of the Web APIs the Firefox OS
|
|
platform makes available and that Mozilla and other contributors are
|
|
trying to standardize is the "Push API" (http://www.w3.org/TR/push-api/)
|
|
under the auspices of the webapps WG (http://www.w3.org/2008/webapps/).
|
|
The API is somewhat decoupled from the actual notification mechanism.
|
|
Mozilla's reference server protocol is documented at
|
|
https://wiki.mozilla.org/WebAPI/SimplePush/Protocol.
|
|
|
|
From the IMAP server's perspective, the process is that the client uses
|
|
the Push API to get a URL and then the client tells the IMAP server the
|
|
URL. To notify the client, the IMAP server issues a PUT request to the
|
|
URL. The only data payload is the 'version' which is a monotonically
|
|
increasing 64-bit integer and allows for a degree of idempotency. The
|
|
only key thing for the version is that if the server doesn't increase
|
|
the value since the last time the client heard it, the client won't
|
|
receive a new notification.
|
|
|
|
Our application-level goal is that rather than checking every N minutes,
|
|
after each synchronization attempt we refresh our notification request
|
|
before disconnecting from the IMAP server. The IMAP server would then
|
|
send a ping notification when new messages are received for the
|
|
account/inbox/whatever.
|
|
|
|
Details/thoughts from my investigation are found below. In general, I'm
|
|
very interested in what server implementers think and would be willing
|
|
to adopt, especially for things that might be enabled-by-default in
|
|
deployments. It's been indicated to me that we (Mozilla) have
|
|
engineering resources that could be used to help provide patches to open
|
|
source implementation to help make this happen.
|
|
|
|
Note that any solution that satisfies our needs could probably also
|
|
satisfy similar needs. For example, the Firefox OS Calendar App makes
|
|
use of the Google Calendar API's push notifications mechanism as
|
|
documented at https://developers.google.com/google-apps/calendar/v3/push
|
|
|
|
|
|
# Existing Work
|
|
|
|
## XAPPLEPUSHSERVICE
|
|
|
|
The only thing I've seen in the wild seems to be XAPPLEPUSHSERVICE
|
|
advertised by Yahoo's IMAP server and Apple's dovecot fork. The most
|
|
useful information I've found on this is
|
|
http://www.dovecot.org/list/dovecot/2012-August/067682.html which leads
|
|
us to the implementation of XAPPLEPUSHSERVICE at
|
|
http://www.opensource.apple.com/source/dovecot/dovecot-239.8/dovecot/src/imap/cmd-x-apple-push-service.c
|
|
which tells us the syntax of the command. While the idea is similar,
|
|
XAPPLEPUSHSERVICE does not seem to support alternate endpoints to
|
|
Apple's Push Notification Service and so is not useful for our purposes.
|
|
|
|
## sieve/enotify
|
|
|
|
The sieve enotify extension (http://tools.ietf.org/html/rfc5435) seems
|
|
like the only standadized mechanism capable of implementing such logic.
|
|
Unfortunately, there is no HTTP/URL notification listed in the registry
|
|
at
|
|
https://www.iana.org/assignments/sieve-notification/sieve-notification.xhtml.
|
|
'mailto' is a defined mechanism that has some support (ex: Pigeonhole on
|
|
Dovecot), and it's possible for Mozilla to stand-up a proxy to help
|
|
bridge this gap.
|
|
|
|
The main problem with sieve is that adoption does not seem particularly
|
|
wide-spread and discoverability when supported is limited, although I'm
|
|
hoping that http://tools.ietf.org/html/rfc6785 could help improve that.
|
|
|
|
|
|
# Potential Workarounds
|
|
|
|
I think the following options exist to make this work today:
|
|
|
|
## sieve mailto bridge
|
|
|
|
Stand up an SMTP-receiving proxy so that sieve/enotify/mailto can be
|
|
used to trigger instead of HTTP PUT. Use sieve to set up a rule to
|
|
generate mail notifications every time a message is received. No
|
|
message contents would be sent. This might result in a large number of
|
|
notifications since it's not obvious to me how to write a stateful sieve
|
|
script.
|
|
|
|
## Hosted IMAP Proxy (Undesirable due to inherent credential/data exposure)
|
|
|
|
A proposal from those working on the back-end of Mozilla's push
|
|
notification service is an HTTPS-accessible proxy. Instead of having
|
|
the device maintain the long-lived IMAP connection, the device has a
|
|
server do that on its behalf. The very undesirable aspect of such a
|
|
solution is that while the proxy need not persist the credentials to
|
|
disk, it does need to know them to establish the connection using
|
|
traditional LOGIN. And even if a challenge-response authentication
|
|
mechanism was used in such a way that the proxy server could not
|
|
establish additional connections without device support, the proxy (or
|
|
whatever pretends to be the proxy) would still have a fully authorized
|
|
IMAP connection to use as it chooses until the connection is dropped.
|
|
|
|
|
|
# Potential Standards Approaches
|
|
|
|
## Create a new capability, ex: PUSH-NOTIFY-HTTP=PUT
|
|
|
|
Create a "PUSH-NOTIFY-HTTP PUT <URL> <BODY PAYLOAD>" capability with
|
|
very simple semantics:
|
|
- Each request is one-shot. The request lives until it is fired.
|
|
- At once most one request per URL; new requests replace the old request.
|
|
- An HTTP PUT is triggered to the given URL with content-type
|
|
application/x-www-form-urlencoded with the provided body payload.
|
|
- Body payload is limited to something really small; we only need to be
|
|
able to encode "version=<64bitnumber>"
|
|
- Delivery to the Inbox or maybe just delivery to the account in general
|
|
in case a filter files the message somewhere. Messages filed to the
|
|
junk folder would not trigger the notification.
|
|
|
|
## sieve enotify HTTP PUT extension
|
|
|
|
Since it's not likely for adoption to happen immediately anyways and
|
|
sieve is extremely powerful it seems like a reasonable choice to avoid
|
|
introducing a new feature which is doomed to feature creep.
|
|
|
|
Because sieve is not particularly stateful the main issue would be how
|
|
to achieve single-shot semantics. This might be best addressed by doing
|
|
something like the servermetadata example at
|
|
http://tools.ietf.org/html/rfc5490#section-4.1 where the client would
|
|
use metadata to control the process and the URL notification function
|
|
would have built-in idempotent semantics. Each time the client
|
|
connected it would refresh the 'verson' it uses and which is used in the
|
|
notification invocation. Then when the sieve rule next fires, the
|
|
URL/payload changes so that the notify action actually goes through.
|
|
|
|
Multiple consumers of such notifications would be handled during
|
|
rewrites of the sieve rules. A new device setting up for push
|
|
notifications would identify its rule based on the URL the push API
|
|
tells it. It would write its rule to the sieve store along with the
|
|
current timestamp for the rule. It would also cull any other rules of
|
|
the same variety whose timestamp more than twice as old as the
|
|
rule-rewrite-interval based on the timestamp.
|
|
|
|
## custom sieve extension
|
|
|
|
Realize the above state management logic is too crazy. Internalize the
|
|
one-shot state management, still using metadata somehow.
|
|
|
|
## Extend NOTIFY somehow
|
|
|
|
This seems like it would be a mismatch since the NOTIFY subscription is
|
|
inherently associated with the connection lifetime and any other
|
|
lifetime would be weird.
|
|
|
|
## Define CONTEXT=pushnotify
|
|
|
|
Similar to NOTIFY, this seems like a mismatch.
|
|
|
|
|
|
Thanks for any/all responses,
|
|
Andrew Sutherland
|
|
|