[collectd] collectd storing data in a db without data condensing (like rrd)?

[Andreas Maus]

Hi.

On Tue Dec 20, 2016 at 18:02:13PM +0000, Andreas Schuldei wrote:
> What are the recommended ways of storing data in a database, where the data
> is not condensed, like rrd does it?
> 
> I am recoding mostly temperature data of ca 60 sensors, over the time frame
> of several years/decades, and I need to be able to compare between years.
> So it won't be a high volume of data coming in per hour, but it will be
> some data accumulating over the years.
> 
> this system will run on a resource constrained server, so something with a
> modest memory footprint  would be appreciated. (i expect to upgrade that
> system as the hardware dies, but i would prefer not to doing migrations
> between databases every time i switch hardware.
> 
> what database do you recommend?
I've tested:

Graphite (Plugin: write_graphite)
OpenTSDB (Plugin: write_tsdb)
InfluxDB (has native support for collectd format)

Own test showed InfluxDB as the best solution for my environment
(~900 systems, total of about 40k metrics sampled every 10 sec.
on a 8 year old HP BL460c G1 system using a RAID1 of two 5400 RPM SATA
disks,filesystem: XFS and 16 GB RAM).

Graphite doesn't seem to scale well, OpenTSDB was somewhere between
Graphite and InfluxDB.

Changing data or metadata for old sets will introduce a significant
performance loss as "cold shards" must be decompressed,
changed and recompressed. But in almost all cases
your are appending data to existent sets.

> What would be a frontend (for plotting the data) to go with that?
Grafana. Supports a lot of backend formats.

All alternatives don't convert COUNTER/DERIVE types, so you have
to apply some functions (e.g. derivative/non_negative_derivative for
InfluxDB) in the graph configuration.

HTH,

Andreas.

-- 
"Things that try to look like things often do
 look more like things than things. Well-known fact."
Granny Weatherwax - "Wyrd sisters"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mailman.verplant.org/pipermail/collectd/attachments/20161220/0fb17ebf/attachment-0001.sig>