[collectd] Proposal: Make collectdmon start several instances of collectd based on configuration file

[Sebastian Harl]

Hi Yann,

On Tue, Feb 23, 2010 at 04:31:27PM +0000, Yann Hamon wrote:
> > On Tue, Sep 08, 2009 at 10:45:35AM +0200, Peter Fischer wrote:
> > > I was playing with the 1wire plugin lately und of cource came about the
> > > "EXPERIMENTAL!" paragraph.
> > > I think there would be a relatively simple solution for that:
> > >  * In every plugin's stanza in collectd.conf an interval can be given. If
> > > there is none, the global default is being used.
> > >  * During startup, collectdmon would enumerate every plugin's desired
> > > interval, group these, and would start in collectd instance for every single
> > > interval in the group.
> > > for example:
> > >  * soil temperature and humidity is measured every 30min
> > >  * barometric pressure is read every 5 min
> > >  * ambient temperature and humidity is measured every 2 min
> > >  * everything else uses the default 10 seconds interval
> > > in that case collectdmon would start four instances of collectd running with
> > > different intervals
> > 
> > In fact, support for a similar feature has already been implemented
> > in collectd. However, currently, there is no way to actually use
> > that since it cannot be configured so far.
> > 
> > The problem is that we'd probably have to touch any single plugin to
> > be able to make that configurable, so this is going to be a fair bit
> > of rather boring coding work. Another issue, we're currently a bit
> > unsure about is how to reflect this feature in the config file, i.e.
> > what kind of config option should be introduced. I guess, it's the
> > best to introduce a config option (e.g. "Interval") for each and
> > every plugin (as you suggested as well), but I'm not a 100% sure
> > about that yet. Any comments would be appreciated.

> In an ideal world, you would set a DefaultInterval for all plugins
> (20secs?) and then potentially overwrite that value in the
> configuration of every plugin. That would be perfect. 

Yes, that is the idea, basically.

> But maybe there are restrictions, like a plugin could only have an
> interval that is a multiple of the base interval?

There are no such restrictions (from a collectd-core architectural point
of view). The question is, whether we want to introduce a config block
for each and every plugin where most plugins will support a single
config option ("Interval") only. This would, e.g., look like this:

  Interval 10 # default

  LoadPlugin cpu
  LoadPlugin load
  LoadPlugin postgresql
  LoadPlugin rrdtool

  <Plugin cpu>
      Interval 5
  </Plugin>

  <Plugin postgresql>
      Interval 60
      # more postgresql configuration
  </Plugin>

  # more configuration

Then, cpu would read its values using a 5 seconds interval, load would
use the default of 10 seconds and postgresql would use 60 seconds. As I
said before, the "downside" of this approach is that this feature has to
be implemented in every single plugin.

Another approach might be to do something like this:

  Interval 10

  <LoadPlugin cpu>
    Interval 5
  </LoadPlugin>
  LoadPlugin load

  # more configuration

This could be implemented in the daemon itself, so it would not require
a modification of every plugin. Plugins could still support a more
specific "Interval" option (e.g., the "postgresql" plugin might want to
support different intervals for querying different databases). However,
that would introduce three different kinds of intervals: a global
interval, a plugin specific interval overwriting the global
configuration and a plugin instance (or whatever) specific interval
overwriting the plugin specific interval. That might be confusing to
users but it would probably require the least code changes and the most
consistent behavior (as in: it's not possible to accidentally have some
plugin not implementing the plugin specific interval option). Also, I'm
not yet a 100% sure how to implement that [*].

Any comments about that?

Cheers,
Sebastian

[*] One approach might be to keep a look-up table in the daemon, storing
    (plugin name, interval) pairs. When a plugin registers a read
    callback without specifying an interval, the daemon would try to
    look up the interval in that table and fall back to the global
    interval. This would require all plugins to strictly follow a naming
    convention when registering read callbacks, though (e.g.
    "<plugin_name>-<instance>").

-- 
Sebastian "tokkee" Harl +++ GnuPG-ID: 0x8501C7FC +++ http://tokkee.org/

Those who would give up Essential Liberty to purchase a little Temporary
Safety, deserve neither Liberty nor Safety.         -- Benjamin Franklin

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
Url : http://mailman.verplant.org/pipermail/collectd/attachments/20100223/f1eb0f76/attachment.pgp