[collectd] Filter mechanism in collectd

[Florian Forster]

Hi,

I'm currently working on implementing a filter mechanism for collectd,
which I plan to include in version 4.6. Since I'm not at the point where
the concept becomes kind of stable, I'd appreciate any feedback on this.

You can find the documentation on the mechanism here:
  <http://verplant.org/temp/collectd.conf.5.html#filter_configuration>
(That's in my temporary directory and will be deleted automatically in
30 days!)

In a nutshell, it works as follows:
- There are `matches' and `targets'. A `match' selects certain values,
  `targets' perform some action on or with the values.
- Matches and targets are coupled using `rules'.
- A `chain' is a list of rules.

As you may notice, it's all very similar to the way iptables filters
network packets. This is of course no accident - that concept has proven
successful and extensible.

These chains are used in collectd like this:

- Every value is sent to the `Main' chain.
- Every rule is tried in turn and the targets of applicable rules are
  executed.
- You can jump to other chains, return from those chains, stop
  processing entirely and send values to write plugins. And, of course,
  you can use more matches and targets provided by some plugins.
- The entire functionality is absolutely optional. If you specify
  nothing at all, it will work just like it used to, i. e. all values
  will be sent to all write plugins.

So, this all sounds mighty complicated. Why all this fuzz? - Well, here
are some examples what you can do with that (all that already works!:)
(All examples are untested. Beware of typos!)

Ignore certain values:

  LoadPlugin "match_regex"
  <Chain "Main">
    # Ignore all mysql-*/mysql_commands-show_*
    <Rule "ignore_mysql_show">
      <Match "regex">
        Plugin "^mysql$"
        Type "^mysql_commands$"
        TypeInstance "^show_"
      </Match>
      Target "stop"
    </Rule>

    # Default target
    Target "write"
  </Chain>


Write certain values to local RRD files, send others via network:

  LoadPlugin "match_regex"
  <Chain "Main">
    # Write all `irq' values locally.
    <Rule "irq">
      <Match "regex">
        Plugin "^irq$"
      </Match>
      <Target "write">
        Plugin "rrdtool"
      </Target>
      Target "stop"
    </Rule>

    # Default target
    <Target "write">
      Plugin "network"
    </Target>
  </Chain>


Sanity checking:

  <Rule "no_fqdn">
    # Match and ignore hostnames without a dot in them. Those are
    # certainly no FQDN and may not be desired.
    <Match "regex">
      Host "^[^\.]*$"
    </Match>
    Target "stop"
  </Rule>


Re-set certain parts of the value's identifier:

  LoadPlugin "match_regex"
  LoadPlugin "target_set"
  
  # Change sensors-coretemp-isa-000[0123]/temperature-temp1
  # to     sensors-coretemp/temperature-core[0123]
  <Chain "sensors">
    <Rule "temp0">
      <Match "regex">
        PluginInstance "^coretemp-isa-0000$"
      </Match>
      <Target "set">
        PluginInstance "coretemp"
        TypeInstance "core0"
      </Target>
      Target "return"
    </Rule>
    :
    :
  </Chain>
  
  <Chain "Main">
    <Rule "sensors">
      <Match "regex">
        Plugin "^sensors$"
      </Match>
      <Target "jump">
        Chain "sensors"
      </Target>
    </Rule>
  
    Target "write"
  </Chain>


As you can see, the `regex' match and the `set' target are already
available, together with the built-in targets `return', `stop', `jump'
and `write'. We, i. e. Sebastian and I, plan to integrate theshold-
checking as a match, log and notifications as targets and whatever else
comes to our minds.

What ideas have you got? Do you see any problem with the above concept?
Any feedback is really welcome :)

Regards,
-octo
-- 
Florian octo Forster
Hacker in training
GnuPG: 0x91523C3D
http://verplant.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mailman.verplant.org/pipermail/collectd/attachments/20081124/09ca2c66/attachment.pgp