[collectd] patch for email stats from postfix and amavisd-new, dns stats from powerdns

[Luke Heberling]

Florian Forster wrote:
>
> I don't want to delve into hashing functions now, I have too much stuff
> going on for that :/ For such non-obvious stuff, please put a comment
> there. 
Done.
>
> Why do you use hash tables in the first place? There's an AVL-tree which
> is used for caches in various places. The depth (i. e. steps it takes
> for a negative answer, which equals the worst case positive answer) is
> 29 for 491e6 entries, i. e. not very much. Also, hash tables are not
> sorted, but an MRU-cache implies some sorting, doesn't it? Is the
> performance gain you get for a very large number of entries really worth
> all the code and possible bugs? Or is this not the point?
>
The mru cache is both a hash and a linked list.  It needs to be quickly 
accessible by key, and the least recently accessed item should also be 
quickly accessible. This could be implemented with dual avl trees.  I 
will implement the amavis plugin this way later this week and submit it 
to see what you think.  I'm rather embarrassed to say that hadn't even 
occurred to me. :-(

> I like that idea. The utils_tail code could start a new thread which
> calls a callback with each new line read and the line reading, the
> buffer allocating and all the other annoying details would be completely
> hidden from the other code.. I'll look into that as soon as I have time
> for it..
>
Perhaps the thread creation could be an option, because in some cases 
such as mine, it's not really necessary. The amavis/postfix plugins just 
need to catch up before each reading, they don't need to be kept up to 
date continually. That just adds the thread overhead plus the need to 
synchronize access to the variables which hold the counters et al.  
There's also error handling. If the thread created by the tail utility 
module is unable to open or read the file, can it properly handle 
errors? Does the callback function serve this purpose? These issues 
might be avoided if the tail utility module does not have to create its 
own thread.

Luke