[Imap-protocol] IMAP MOVE extension
witold.krecicki at firma.o2.pl
Sat Jun 12 08:10:36 PDT 2010
On Saturday 12 of June 2010 16:58:17 Timo Sirainen wrote:
> On 12.6.2010, at 8.58, Witold Kręcicki wrote:
> >> So let's say that we punt and say that an error is an error and we'll
> >> break the all-of-nothing guarantee of IMAP.
> > This guarantee is a myth that is not working in eg. Maildirs.
> > The same problems that appear with COPY or EXPUNGE in those critical
> > situations but nobody is thinking about removing COPY or EXPUNGE from the
> > protocol.
> Since you keep talking about maildirs..:
> Assumptions: Maildir can be accessed mostly without locks. Reading mails
> should never get any locks, and that's also how Dovecot works. Changing
> flags or expunging messages would be possible to do without locks,
> although currently Dovecot doesn't. Assigning UIDs to new messages (i.e.
> making new messages visible) pretty much requires some kind of locking.
> The way I've implemented COPY is that it first link()s files to the
> destination Maildir/tmp/ directory. Then it locks the maildir, assigns
> UIDs to messages, rename()s files from tmp/ to new/ and unlocks. Messages
> won't be visible to other connections until unlock is complete. This means
> that as long as the system doesn't crash, there is a single atomic event
> (unlock) when the messages either become visible or the copy operation
> gets rollbacked. This can fail only if there are non-Dovecot MUAs
> accessing the maildir that aren't using Dovecot's locking, but in normal
> Dovecot-only situations this works perfectly.
> Now, if you wanted to implement MOVE operation using rename(), there is a
> problem: Immediately after you do a rename() the message is gone from the
> source mailbox. If another session tries to open the mail (remember, no
> locks while reading), or for some other reason scans the maildir for what
> messages it contains, it sees that the message is gone and issues EXPUNGE
> to client. There is no way to revert that! And why would the MOVE
> operation fail? 1) COPY fails if one of the messages has already been
> expunged, so MOVE should probably too, and this won't be known beforehand.
> 2) Filesystem quota can cause rename() to fail if it would increase the
> destination directory's size.
> So the difference is that COPY can under normal operating conditions work
> as intended, while MOVE implemented with rename() can't.
Maybe MOVE implemented with rename() can't, but move implemented with link &
delete (so 'internally' as copy + expunge with a single lock) can.
> > What if the server deletes 3 out of 10 messages and then is struck by a
> > filesystem failure (cannot delete remaining 7, cannot undelete of the
> > first 3)? This situation is (formally) not covered in the protocol,
> > server cannot send untagged EXPUNGE responses for those 3 messages
> > (because then it would have to send OK) and it cannot send OK because
> > EXPUNGE after a success removes ALL messages with \Deleted flags...
> I don't think that's a correct assumption.
> > It should send NO but then the client would
> > think that no messages were deleted permanently...
> I don't think that's a correct assumption either.
> My server replies to EXPUNGE always with OK, even when there are \Deleted
> messages that weren't expunged because ACLs didn't allow that (it used to
> send NO, but some clients tried to send EXPUNGE internally and kept giving
> popup windows saying the EXPUNGE failed). Clients should use the untagged
> EXPUNGE replies to figure out what happened, the OK reply from EXPUNGE
> doesn't tell anything.
"The EXPUNGE command permanently removes *all* messages that have the \Deleted
flag set from the currently selected mailbox.". It is not my assumption, it's
stated in the protocol.
Grupa o2 Spółka z o.o., ul. Jutrzenki 177, 02-231 Warszawa,
KRS 0000140518, Sąd Rejonowy dla m.st. Warszawy,
Kapitał zakładowy 377.298,00 zł., NIP 521-31-11-513
More information about the Imap-protocol