[Imap-protocol] MOVE is a pipeline

[Mark Crispin]

On Mon, 14 Jun 2010, John Snow wrote:

>> No.  MOVE takes a message set argument.  That makes its behavior

>> client-deterministic.

> How is that different than a UID expunge?  A client specifies which

> specific message it wants expunged.


That's not what UID EXPUNGE does.

UID EXPUNGE indicates the message ranges that MAY be expunged as a result
of that command (or, alternatively, indicates that messages outside that
range should not be expunged as a result of that command).

It does not require that the server expunge any of those messages.  It
won't expunge messages that aren't deleted (read RFC 4315) or were
undeleted in another session just before the UID EXPUNGE was issued (so
there was no opportunity for an untagged FETCH to report the undeletion).

Nor does it forbid an expunge that happened in some other session from
being reported.

These are all significant differences from "[in UID EXPUNGE] a client
specified which specific message it wants expunged."

There is no command in IMAP to say "expunge these, and precisely these,
messages".  I would find such a command to be utterly appalling, but I
would have no argument against it.

I do not find MOVE to be appalling.  MOVE is a desirable functionality.
The problem with MOVE are technical objections which have been known for
25 years.

My judgement is not based upon what I find to be desirable or appalling.
They are based upon technical matters.

So ideas are appalling and technically flawed.  They are the easiest to
oppose, although some people argue for them anyway.

MOVE is desirable, but not as a chatty kludge with unpredictable behavior.
MOVE must either fully succeed, or fully fail.  If it succeeds, then ALL
messages MUST be inserted in the target and MUST be removed from the
source.  If it fails, then messages MUST NOT be inserted in the target and
MUST NOT be removed from the source.  This also obviates the need for an
untagged expunge or any other response other than a MULTIAPPEND-form
APPENDUID (not COPYUID) data.


> Why doesn't a move have the same option?


So you are saying that the sequence set argument to MOVE ought to be a
guideline, like UID EXPUNGE; and not a specific client request for a
precise set of messages, as in FETCH, STORE, COPY, SEARCH?

Do you think that client implementors will be comfortable with that?  "It
may or may not move all the requested messages, and it may leave you in a
state where you can't move forward (complete the operation) or backward
(undo it)."

As I noted earlier, the requirement for atomicity in COPY did not
originate with me.  Both the original IMAP and IMAP2 had an untagged COPY
response for partial success.  Client implementors rejected those
semantics circa 1991.  As a result, the COPY response was deprecated in
RFC 1730, and removed entirely in RFC 2060 (see RFC 2062).

Non-atomic MOVE violates that precedent for COPY, and with a far more
dangerous operation that destroys data.

This will be the first operation in IMAP that may fail irreversibly.

The only definition of MOVE that I consider acceptable is:

 	MOVE and UID MOVE must either fully succeed, or fully
 	fail.  If it succeeds, then ALL requested messages MUST
 	be inserted in the target and MUST simultaneously be
 	removed from the source.

 	If it fails, messages MUST NOT be inserted in the target
 	and MUST NOT be removed from the source.

 	There MUST NOT be any intermediate state in which some,
 	but not all, messages appear in the target.

 	There MUST NOT be any intermediate state in which some,
 	but not all, messages are removed from the source.

 	A successful MOVE and UID MOVE returns data equivalent
 	to the APPENDUID response for a MULTIAPPEND.  There is
 	no untagged EXPUNGE or any other indication of the
 	messages that were removed from the source.  This means
 	that a client which does UID MOVE must either have the
 	UID/sequence map in advance to calculate the affected
 	messages or it can not use message sequence numbers.

 	An untagged EXPUNGE response resulting from some activity
 	in another session MAY occur with a MOVE and UID MOVE
 	command.  The message sequence number in such responses
 	reflects the source mailbox after the effect of the MOVE
 	or UID MOVE.  Example:
 		tag MOVE 2:5 test
 		* 2 EXPUNGE
 		tag OK [MOVEUID 48394733 3345:3348] all moved
 	The message that was expunged was the former message 6.
 	If this was a UID MOVE command, the client would not know
 	what message got expunged without having a UID/sequence
 	map.

 	All other sessions which have the source mailbox open
 	will receive untagged EXPUNGE responses at the next
 	permitted point in the session (RFC 3501, section 7.4.1).
 	This means that there is an intermediate state in which
 	the messages exist in both source and target, that exists
 	only in sessions which have the source mailbox open at
 	the time of the MOVE.

This all creates quite a few requirements on a server.  But the point is
the benefit to the client: the saving of RTTs.  It is not to create a
chatty kludge for certain mail stores that presupposes that the client
doesn't care that it can get halfway and be unable to move forward or
back.

If you agree to ALL of the above conditions, then I will drop my
objections.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.