[collectd] Using an output plugin to produce CSV files usableby Windows Performance Monitor

[mclellan_dave at emc.com]

Hi Sebastian.   Now that's an interesting possibility I hadn't
considered.  Let me think on this one.  Thanks for the suggestion! 

Dave

+-+-+-+-+-+-+-+
Dave McLellan/Cell=978-500-2546/Email=mclellan_dave at emc.com
+-+-+-+-+-+-+-+

-----Original Message-----
From: Sebastian Harl [mailto:sh at tokkee.org] 
Sent: Tuesday, March 17, 2009 11:30 AM
To: mclellan, dave
Cc: collectd at verplant.org
Subject: Re: [collectd] Using an output plugin to produce CSV files
usableby Windows Performance Monitor

Hi,

On Tue, Mar 17, 2009 at 10:45:43AM -0400, mclellan_dave at emc.com wrote:
> I'm exploring the possiblity of using the Windows system Perfmon tools
> to display data captured in Linux.   
[...]
> There are limitations of the internal tool, and we think they might be

> overcome by using collectd as the collector [...]

It's very interesting to see what broad range of use-cases people keep
coming up with over time ... ;-)

> Has anyone done this?   I know there is a .csv plugin existing, but
it's
> not producing perfmon formatted csv files.   I'll probably start there
> to build a new plugin, but I'm looking to see if anyone has done this 
> nutty thing I'm proposing.

I have never heard anybody talking about anything like that, so, I
guess, it's up to you to come up with a solution.

I'm not sure if it makes sense to come up with a new plugin that spills
out some different CSV output though. Instead, I just had the idea to
come up with a new plugin that outputs arbitrary formatted text where
the format is user-defined. I'm not sure how, exactly, that should be
designed but something like the following configuration example comes to
my mind right now:

  <Plugin "text"> # I couldn't come up with a better name yet ;-)
    BaseDir "/var/lib/collectd/text"
    FileName "%h/%p-%i/%t-%j-%Y-%M-%D.txt"
    Header "epoch;%n"
    Content "%e;%v\n"
    ValueSeparator ","
    Rotation daily
  </Plugin>

... where each config option is described as below:

 * BaseDir: Self-explanatory. The base directory for all files, just
   like in the csv and rrdtool plugins.

 * FileName: Name of the file to be created for a dispatched value list
   (see below for an explanation of the placeholders).

 * Header: Format of the first line of each file.

 * Content: Format of the content of each file.

 * ValueSeparator: Separator used to concatenate the data-sources of
   one data-set.

 * Rotation: When to start a new file, like "hourly", "daily",
   "monthly", "yearly", "never", ...

... and, at least, the following placeholders:

 * %h: hostname

 * %p: plugin name

 * %i: plugin instance

 * %t: type name

 * %j: type instance

 * %Y: year

 * %M: month

 * %D: day

 * hour, minute, second, ...

 * %n: data source names, separated by ValueSeparator

 * %e: epoch

 * %v: data source values, separated by ValueSeparator

 * ...

I could image one might come up with a lot of useful placeholders, so
using something else than %X might make sense - this is just a first
draft ...

Does that make sense? Any comments about that?

Dave, what do you think? Does that sound reasonable enough so you might
want to give it a try (after further discussion on the list)?

Frankly, my idea sounds rather crazy (even to myself) and I'm not sure
if it will be useful to anybody else - however, I'd be really powerful
... ;-)

Cheers,
Sebastian

--
Sebastian "tokkee" Harl +++ GnuPG-ID: 0x8501C7FC +++ http://tokkee.org/

Those who would give up Essential Liberty to purchase a little Temporary
Safety, deserve neither Liberty nor Safety.         -- Benjamin Franklin