[collectd] What are "interesting results" mentioned in FAQ?

[Piotr Hosowicz]

Florian Forster pisze:
> Hi Piotr,
> 
> On Wed, Dec 24, 2008 at 03:22:04PM +0100, Piotr Hosowicz wrote:
>> I am running a Python script that checks some stats and sends them to
>> stdout and further to the collectd server via exec plugin. I watched
>> the graphs and I saw very strange results. When I dumped collected RRD
>> files to XML i saw that data was not normal at all, I expectd it to be
>> numerial thousends / tens of thousands. Instead in the dumpo I saw a
>> lot of NaN's , fractional "almost zeroes" and very hight numbers in
>> the order of millions. Is this said interesting result? - I foubnd in
>> the FAQ:
> 
> sounds weird, right enough ;) Can you be a bit more specific what you're
> feeding the exec plugin? Some lines printed by your script, for
> instance?

The daemon that I watch produces on TCP port:

something:value
something2:value2

... each time it receives a newline. Values are integers. My Python
script connects to the daemon, then sends newline, receives values, then
waits 60 seconds and so on. Each time it reads values it makes PUTVAL
lines and sends them to the standard output. This works, as I see that
the data collected locally as CSV is perfectly OK.

>> #
>> Can I adjust the interval in which data is collected?
>> Yes, since version 3.9.0 this can be set at compile-time. Keep in mind, 
>> though, that this will change the layout of the generated RRD-files. 
>> Also, clients and servers should have the same setting here to avoid 
>> interesting results.
>> Version 4.0 allows this setting to be adjusted in the configfile.
> 
> If you let collectd create RRD files and then change the interval in
> which data is collected, the already generated RRD files are not
> changed. So the RRD files will be fed with data at a different rate than
> they'd expect. This may work if you changed the interval in the
> ``right'' direction, i. e. if you're adding values more often than the
> file expects. If you do it the other way around, if may still work,
> depending on your XFF and heartbeat settings. What really happens is
> hard to tell without going deep into the details of RRDtool, so it's
> left unspecified here so that the point comes across: If you change the
> interval, you have to delete and re-create the RRD files! (or know what
> you're doing ;)

I tried this, see below.

>> Server: FreeBSD, as far as I cansee in the /usr/ports/... subdirs it
>> is collectd 4.1.2 and I bet it was built with default settings, I see
>> in the collectd.conf Interval is set to 60 (seconds, I suppose).
> 
> Please note, that we have fixed quite some bugs since 4.1.2, so
> upgrading to a newer version would be a good idea - if only to 4.1.6.

Sorry, I was wrong, the version on the server was not installed from 
ports collection but straight from the source and it is 4.5.0 or 4.5.1.

>> Clients: Linux, collectd 4.5.1, built without any interval settings, I
>> even do not see them in ./configure --help . There's a commented out,
>> I suppose the default setting of the interval set at 10. It might be
>> that the problem is that my script passes data to collectd specifying
>> interval=60 , but I had started running it with intervals of 5 or 10
>> seconds, specified in the stdout output.
> 
> That's okay - the interval in which data is collected is included in the
> data structure, sent over the network and used by the rrdtool plugin and
> so forth. The problem is when you let the client run for a while and
> then decide that maybe an interval of 20 seconds would be more
> appropriate. If you do that, the server will already have created files
> that expect data every 10 seconds. You will have to delete those files
> then (collectd will automatically recreate them with the now-right
> settings).

This is what I tried - allowing collectd to recreate the files. Still 
there is garbage in the rrd files.

[snip]

I think that trying to solve the problem is not morth the effort - these 
servers will be reinstalled soon. I hope that the new OS will have 
collectd and its configuration that will not give such strange results. 
If the problem will persist I will ask again.

Regards,

Piotr Hosowicz