[collectd] Generic "data" plugin (was: couchdb plugin rename?)

[Sebastian Harl]

Hi,

On Thu, Aug 20, 2009 at 10:21:19PM +0200, Mariusz Gronczewski wrote:
> 2009/8/20 Florian Forster <octo at verplant.org>:
> > In the future, we might be able to add other parsing / matching modules,
> > for example for XML or other structured data.
> I'd love curl_table module :) Would be useful for some servers that
> can return CSV data like haproxy (atm im using simple exec script to
> do that)

Hrm … going further into that direction, it might make sense to intro-
duce a very generic "data" (could not come up with a better name so far)
plugin. My idea is that this plugin would be able to load further
"drivers" for input and data processing [1]. The plugin would then
define a format (probably plain text) to pass on the data from the input
to the data processing driver. A user could then use any combination of
available drivers without having a specific plugin for each of them.

The following input drivers come to my mind:

 * curl
 * file (could be done by curl as well, though, but would not require an
   external dependency)
 * tail (possibly a special mode of "file") (this would, obviously, make
   the "tail" plugin obsolete)
 * socket
 * memcache* (which would probably make the "memcachec" plugin obsolete)
 * <you name it> (I'm sure once that's implemented a lot of new ideas
   will pop up)

(Side note: This is not necessarily limited to plain text data as long
as we've got data processing drivers that support other formats as
well.)

I'm not entirely sure about how to do the data processing yet. It might
make sense to further split that up into something like preprocessing,
matching, selecting, whatever and / or make it possible to link multiple
drivers (somewhat similar to the chain mechanism). Anyway, in general it
should take care of parsing, e.g. XML, tabular, CSV, JSON, etc. data and
somehow making that available to the user.

This is just a very rough idea so far and I did not think about any
details yet. Any comments, suggestions, beating me up, … would be
appreciated :-)

Cheers,
Sebastian

[1] It might even make sense to implement that in the daemon …

-- 
Sebastian "tokkee" Harl +++ GnuPG-ID: 0x8501C7FC +++ http://tokkee.org/

Those who would give up Essential Liberty to purchase a little Temporary
Safety, deserve neither Liberty nor Safety.         -- Benjamin Franklin

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
Url : http://mailman.verplant.org/pipermail/collectd/attachments/20090827/ad7663b5/attachment.pgp