[Imap-uw] How to manage c-client memory management?

[Mark Crispin]

On Tue, 14 Jul 2009, Shawn Walker wrote:

> I'm trying OP_SHORTCACHE, but this seems to slow the download messages since 

> now each message has to be fetched from the server.


Indeed it does.  If you want to use early 1990s memory models, you must 
suffer early 1990s performance problems.


> there 

> are countries that have very low bandwidth (Australia has 180k) and to 

> download huge files can take a long time.


That is why you should cache.  Aggressively.  Memory is cheaper than 
bandwidth.


> The issue with calling mail_free_cache is that the "nmsgs" get reset back to 

> 0 when I'm trying to get the header, envelope, body or attachment as the 

> message is being downloaded or the user took some action that I need to get 

> the text of the message or attachment.


I did not see that you were using mail_free_cache().  That is an internal 
function that you should never call.  The appropriate function, which is 
documented in internal.txt, is mail_gc().


> The issue is how the users is storing their messages on the server, some 

> users has 30,000, 100,000 or some ridiculous amount of messages in a folder. 

> Caching all of those messages consume the memory.


Memory is cheaper than bandwidth.  I routinely deal with mailboxes of that 
size.  I would not think of slowing things down to save a few pennies of 
bandwidth.


> And not everybody has a computer that has 4 GB of memory, some people in 

> eastern Europe is still using computers from the mid nineties running Windows 

> 95!


14 year old computers are suitable for tasks of 14 years ago.  They are 
not suitable for modern tasks.  At today's netbook prices, there is no 
excuse to continue using an obsolete, power-wasting, dinosaur.

Try running Outlook on that 100,000 message mailbox on that Windows 95 
machine and see how well it does.

4GB of memory costs less than an espresso at Starbucks.  However, you 
should not need anywhere near this.  I regularly play with large mailboxes 
on a Nokia N800 which only has 128MB.


>From your use of the word "download", you are not using the c-client 

library effectively.  If you were, nobody would ever download 30,000 
messages in a session, much less 100,000 messages.

Look at the Alpine source code for an example of how it is done properly.

-- Mark --

http://panda.com/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.