[collectd] collectd & rrdcached scalability issue

[Stefan Wiederoder]

Hello list,

my current collectd (v4.10) installation is using rrdcached (rrdtool
1.4.4), which worked fine
so far. But with the number or hosts growing beyond 1000 it seems to have a
negative impact
on my central collectd server.

I´ve 37 GB of data (residing in a RAM-disk), plus 12 GB of journal data.
Each hosts has a
minimum of nine plugins activated, but max is only two or three more on
some special boxes.

the problem is that collectd data seems to lag "behind" the longer
collectd/rrdcached is running.
this accumulates up to three hours a day until the daily restart kicks in.
An example, I see data
from 12h when I look at graphs at 15h.

this was not a problem with fewer hosts (< 900), it´s been getting worse
with more and more hosts.

I´ve only specified the number of write_threads so far (=16), but no other
parameters.

do you have any suggestion for me?

thanks
,
Stefan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.verplant.org/pipermail/collectd/attachments/20120907/4793db62/attachment.html>