[collectd] patch for email stats from postfix and amavisd-new, dns stats from powerdns

[Luke Heberling]

Florian Forster wrote:
>
> I agree totally. It'd be best if you could first create a patch that
> puts the tail-like stuff into a module in collectd and send it here for
> discussion. This might save some work on the plugins later on..
>
You beat me to it :-)
Here's the mrucache reimplemented per the advice from the list which can 
hopefully serve the same purpose.
The attachment contains the mrucache header and source file as well as a 
reasonably comprehensive test program.

As far as the main plugins, the refactoring was a little deeper than 
simply moving code around, so I'm testing the new plugins on a small 
domain.  My plan is that by about the end of the week I should be 
satisfied enough to replace the original version of the plugins that is 
in production on two much larger domains.  When that looks good I'll 
create the patches.  If someone would like the code in the interim, let 
me know. I hope to receive from the mrucache code that is attached 
opinions on how to further conform to the preferred formatting for your 
project which I can then apply to my postfix/amavisd-new/powerdns plugins.

I've looked at the tail utility module and have a few items to submit 
for discussion:

It appears that when the plugin processing returns you close the file 
handle, save the position and re-open the file when the plugin is next 
called, attempting to seek back to the same position. This will cause a 
small amount of lost data during rotation as any lines written to the 
log between the time that it was last read and the time it is rotated 
cannot be processed. It's unlikely to be more than a few lines as 
rotation typically occurs during off-peak hours, but it has to include 
only one important line to affect the statistics. A worst case scenario 
might be the postfix plugin missing the delivery notification of a large 
message and its byte totals would fail to accumulate several megabytes. 
Is keeping a file handle open between calls to the plugin a fair trade 
for avoiding this kind of situation? Clearly the lock could not be held 
between calls, so it would have to be obtained and released with each 
call (See below).

Is file locking necessary? I wrote my tail code using as a guide tail.c 
from coreutils. It does not appear to use any kind of locking. It seems 
that if one is to tail a file that is modified in ways other than 
appending and rotating, you can ensure a consistent read with the lock, 
but between reads you're still pretty hosed. It also seems that locking 
a file like syslog or mail.log for a short time every collection 
interval is kind of rude. Without the locking the code also could 
probably become more portable to less unixish systems.

Let me know whether I'm on the right track here or just nitpicking.

Luke Heberling

-------------- next part --------------
A non-text attachment was scrubbed...
Name: mrucache.patch
Type: text/x-diff
Size: 23041 bytes
Desc: not available
Url : http://mailman.verplant.org/pipermail/collectd/attachments/20071128/4aac60d5/attachment.patch