[Imap-uw] Mix filesize

[Mark Crispin]

On Fri, 14 Sep 2007, Per Foreby wrote:
> I have changed MIXDATAROLL in src/osdep/unix/mix.c, and was a bit surprised 
> when I used mixcvt and got the default filesize.
> If this definition was moved to a common include file, say c-client/mix.h, 
> all mix applications could share the same setting.

mixcvt intentionally does not use any c-client information for mix; the 
standalone mix tools follow their own rules and not c-client rules.  Note 
that mixdfix and mixrbld don't even link with c-client at all, and the 
only reason for mixcvt to link with c-client is to read non-mix mailboxes.

If you think about tweaking things, you'd see why you'd want mixcvt to 
have its own independent idea.  But perhaps it should be a command line 
option.

> Has anyone made a serious attempt to see how large the files can be before 
> they start to affect backup performance, or is everybody guessing?

For the most part, everybody is guessing, but there seems to be some 
concensus that a "good" value is between 1MB and 10MB for an "old" data 
file.  That is likely to change over time as the average message size 
grows.  The idea is to have an approximately two order of magnitude 
reduction of number of files over maildir, news, mh, Cyrus, etc.; while 
keeping the occasional expunge in an "old" file to something manageable.

I think though that we're all groping in the dark.  I agree that we 
probably don't want it to be less than 1MB, so any tweaks to the number 
will be to make it larger.  However, 500MB is probably way too large.

I'll be interested to hear how Nancy Lin does with 20MB.  I'd think that 
"most of the .mix files get touched every day" would indicate that the 
size is too large, and needs to be scaled back.

But the only way is to play with lots of values in a real user and 
production setting, and collect enough data that you can graph the various 
values.  Sheesh, this is almost like multivariate calculus (a class I 
still have nightmares about over 30 years later...).

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.