[Alpine-info] answered flag updates close other alpine
mrc at Washington.EDU
Thu Mar 27 11:08:11 PDT 2008
On Thu, 27 Mar 2008, Benjamin R. Haskell wrote:
> I couldn't FTFM (Find The Fine Manual) for 'mbx'. Is there any reference for
> it outside of the code?
There is the information in the FAQ.
> And is there a good reference for the Maildir problem "All of those open()s
> and close()s really add up to major filesystem thrashing." on the page you
> pointed us to, Mark? Or is that something that servers deal with by indexing
> things? (IOW, Why do (or how can) so many big sites seem to use Maildir if
> this is such a big problem?)
The problem with describing maildir issues is that invariably some member
of the maildir church will come back with a rebuttal that usually takes
the form of the issue not applying on some Linux variant that he uses.
The general problem of using one file per message is that this consumes
inodes and depends upon the filesystem for the access semantics.
How well or poorly this works depends upon the filesystem used. In my
test over a decade ago, I was using AdvFS on Digital UNIX; just 1300
messages were enough to lead to horrible mailbox open times. Some Linux
filesystems do quite a bit better.
Just ask anyone who maintains a large USENET spool...
maildir keeps metadata in filenames and is not indexed. Both of the major
IMAP servers which support maildir extend it by adding an index file that
also holds metadata. Doing this defeats the main point of maildir, which
is to be NFS-safe by doing everything through the filesystem and NOT
having such files.
maildir is mostly NFS-safe -- its chief benefit and attraction -- but
there are cases where that fails due to timing races in NFS. One server
works around these issues with non-standard extensions; another pretends
that the problem does not exist and leaves the IMAP client to puzzle out
the strange behavior that occasionally results.
The Wikipedia article on maildir has quite a battle between the people who
point out the real issues in maildir (including a major implementor) and
the true believers in the church. I'm staying out of it.
maildir has some real problems from the IMAP point of view. IMAP assumes
that certain operations are quite fast, but these operations are costly to
do in maildir unless you have a non-standard mechanism for indexing and
metadata (see above about this and NFS safety).
There are third party drivers for UW imapd that add maildir support.
None of them seem to work very well. I find it amusing that the maildir
church has tried, and failed, to produce good maildir support in UW imapd;
yet somehow I (a maildir skeptic) am evil for not producing maildir
support in UW imapd myself.
If I were to implement maildir in UW imapd, it would strictly follow the
maildir specification, including the requirements for NFS safety, with no
extensions. Furthermore I refuse to compromise on IMAP compliance. No
other IMAP server that does maildir does so; they either compromise on
maildir or on IMAP or on both.
Consequently, I am convinced that any maildir code that I write for UW
imapd would suck at least as badly (perhaps worse) as the maildir church's
efforts. The difference is that I would have to support it; and instead
of being evil for not doing it, I would be evil for "deliberately making
UW imapd's new mix format is an indexed format that consolidates multiple
messages into data files, although not at the extreme of a single flat
file for all messages (thus it seeks out a middle ground). Like mbx, mix
does not pretend to be NFS-safe; the use of NFS as an IMAP backend is
pointless (although the reasons why this is the case aren't always obvious
-- Mark --
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
More information about the Alpine-info