[Imap-uw] speeding searches by pre-indexing

[Andrew Daviel]

On Fri, 12 Jun 2009, Mark Crispin wrote:


> On Fri, 12 Jun 2009, Andrew Daviel wrote:

>> 129,634 messages in 23 hours

>

> Yikes!  What is that?  Indexing time?  Search time?


Indexing time.

Sorry, stripped off the original detail. I was trying to get a global 
search for all my mailboxes by pointing htdig at my webmail application.
So what I end up with is a web search engine that returns links into 
webmail. Works, friendlier than "grep whatever mail/.mix*", but indexing 
too slow for real world use.

The slowness is almost all in htdig (the indexing robot). It's 
single-threaded, and when it was written hammering websites at >> 
2 requests/minute was considered rude.

-- 
Andrew Daviel, TRIUMF, Canada
Tel. +1 (604) 222-7376  (Pacific Time)
Network Security Manager