[collectd] patch for email stats from postfix and amavisd-new, dns stats from powerdns
collectd at c-ware.com
Tue Nov 20 20:12:57 CET 2007
Alexander Wirt wrote:
> Luke Heberling schrieb am Sonntag, den 18. November 2007:
>> amavisd-new and postfix statistics are gathered by parsing the
>> log file (i.e. /var/log/mail.log). Unfortunately there does not
>> appear to be a better way of doing this.
> Please don't do this. At least not for amavisd-new. Use amavisd-nanny or the
> bdb stats databases. Alternativly you can also call the snmp agent by hand.
> But do _not_ scan logfiles. The format is adaptable and will offen be
> changed. and also scanning logs is pervert and something that should never be
Presumably you mean that the *automated* scanning of logs is perverse. I
would like to respectfully take issue with that. I may not have an
entirely objective point of view due to a penchant for tools like
logcheck and fail2ban, but I think I have some strong arguments.
Implicit in your first statement is that scanning the logs for postfix
messages is acceptable, though probably far from ideal. Is this a
correct interpretation? I've yet to see an alternative and programs like
pflogsumm and mailgraph seem pretty popular.
Log formats are as you say, adaptable, but I would say that very few
that I've run across change 'often'. Amavisd-new log format in
particular has changed slightly but is quite structured, especially
compared to postfix.
You also mention the bdb stats maintained by amavisd-new (and the snmp
agent which uses the bdb). This is a valid point. This is a good way to
get detailed statistics, particularly in the recent versions of
amavisd-new as it has matured. Modifying amavis-agent into a perl plugin
for collectd would probably be pretty simple. But it too has the same
pitfalls as the logfile: the format changes incompatibly. in the 2.5.0
release earlier this year such a change was made, and the amavis-agent
program had to be changed. This would have broken any collectd plugin
using the bdb in the same way as changing the log format would break my
I encourage you to take a look at the sheer number of log parsers that
are linked directly from http://www.ijs.si/software/amavisd/#contrib. I
count seven different log parsers that do various things. At least two
of them are specifically for creating rrd files (though not nearly as
efficient I daresay as collectd). I see no links to programs which use
the bdb database. The amavis-agent program is entirely within the
distribution, so changes to the bdb format could be more easily
justified as having less impact on third party programs. If the snmp
plugin for collectd were used to communicate with amavis-agent this
would remove the
dependency on the bdb format but require the amavisd-new snmp agent to
be running all the time.
Given that parsing the logs is clearly endorsed by the distribution and
the sheer volume of programs that rely on the structured format, it
seems that they would weigh heavily any further changes. Should they
decide to go forward with such changes, this plugin would have to be
updated. I think that if the community had a use for an updated version
of the plugin, updating it would be a pretty trivial exercise. Plus I'd
be perfectly willing to make such updates into the foreseeable future.
On the other hand, if you guys would like the powerdns plugin by itself
I'd be happy to create that patch. Maybe the amafix plugin could find
its way into the contrib/ section or something.
More information about the collectd