[Alpine-info] answered flag updates close other alpine sessions

[Mark Crispin]

On Thu, 27 Mar 2008, Benjamin R. Haskell wrote:

> I couldn't FTFM (Find The Fine Manual) for 'mbx'. Is there any reference for 

> it outside of the code?


There is the information in the FAQ.


> And is there a good reference for the Maildir problem "All of those open()s 

> and close()s really add up to major filesystem thrashing." on the page you 

> pointed us to, Mark? Or is that something that servers deal with by indexing 

> things? (IOW, Why do (or how can) so many big sites seem to use Maildir if 

> this is such a big problem?)


Sigh.

The problem with describing maildir issues is that invariably some member 
of the maildir church will come back with a rebuttal that usually takes 
the form of the issue not applying on some Linux variant that he uses.

The general problem of using one file per message is that this consumes 
inodes and depends upon the filesystem for the access semantics.

How well or poorly this works depends upon the filesystem used.  In my 
test over a decade ago, I was using AdvFS on Digital UNIX; just 1300 
messages were enough to lead to horrible mailbox open times.  Some Linux 
filesystems do quite a bit better.

Just ask anyone who maintains a large USENET spool...

maildir keeps metadata in filenames and is not indexed.  Both of the major 
IMAP servers which support maildir extend it by adding an index file that 
also holds metadata.  Doing this defeats the main point of maildir, which 
is to be NFS-safe by doing everything through the filesystem and NOT 
having such files.

maildir is mostly NFS-safe -- its chief benefit and attraction -- but 
there are cases where that fails due to timing races in NFS.  One server 
works around these issues with non-standard extensions; another pretends 
that the problem does not exist and leaves the IMAP client to puzzle out 
the strange behavior that occasionally results.

The Wikipedia article on maildir has quite a battle between the people who 
point out the real issues in maildir (including a major implementor) and 
the true believers in the church.  I'm staying out of it.

maildir has some real problems from the IMAP point of view.  IMAP assumes 
that certain operations are quite fast, but these operations are costly to 
do in maildir unless you have a non-standard mechanism for indexing and 
metadata (see above about this and NFS safety).

There are third party drivers for UW imapd that add maildir support. 
None of them seem to work very well.  I find it amusing that the maildir 
church has tried, and failed, to produce good maildir support in UW imapd; 
yet somehow I (a maildir skeptic) am evil for not producing maildir 
support in UW imapd myself.

If I were to implement maildir in UW imapd, it would strictly follow the 
maildir specification, including the requirements for NFS safety, with no 
extensions.  Furthermore I refuse to compromise on IMAP compliance.  No 
other IMAP server that does maildir does so; they either compromise on 
maildir or on IMAP or on both.

Consequently, I am convinced that any maildir code that I write for UW 
imapd would suck at least as badly (perhaps worse) as the maildir church's 
efforts.  The difference is that I would have to support it; and instead 
of being evil for not doing it, I would be evil for "deliberately making 
it bad."

No thanks.

UW imapd's new mix format is an indexed format that consolidates multiple 
messages into data files, although not at the extreme of a single flat 
file for all messages (thus it seeks out a middle ground).  Like mbx, mix 
does not pretend to be NFS-safe; the use of NFS as an IMAP backend is 
pointless (although the reasons why this is the case aren't always obvious 
at first).

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.