[collectd] IO Performance

[Pavel V.]

Hi, Andreas.


> Am 11.03.2013 10:08 schrieb ml ml:
>> i have collectd 4.10.3 running and i have a high IO Load.
> just to compare with myself: howmany rddfiles does your collectd handle?
> or general to the list: when is the use of rrdcache recommended?

I have not so big setup, 5500 rrd files after some cleanup of unneeded metrics, and I got IO
perfomance problem on my collectd host.
All symptoms shows that - I got  high disk queue on write and _all_ host disk IO blocked,
even with cache enabled rrdtool plugin. Version is the same, 4.10, Debian-stable package.

Solution is simple for me - update to a new version.
I have updated to collectd-5.2 this week and I'm greatly impressed how perfectly it works now.
New version cache works much better in compare to 4.10, disk IO highly reduced, no so much CPU
in Wait-IO state as it was before.
At new versions rrdcached plugin is much more usable as it has CacheFlush capability since
collectd-5.x, you can try it. I don't know how anybody can use rrdcached without CacheFlush capabilty... 
I tried to use rrdcached on 4.10, but stopped due to this reason. Anyway, my try shows me main rrdcached
benefits:

+ rrdcached has a journal,
  + which is placed at one file so writes are very fast
  + and received data is written to disk and saved from crash
  + and rrdcached restart can be done via fast shutdown
+ you can freely reload collectd when needed


But now I'm using rrdtool with following config:

<Plugin rrdtool>
        DataDir "/var/lib/collectd/rrd"
        CacheTimeout 1200
        CacheFlush 1800
        WritesPerSecond 40
        RandomTimeout 300
</Plugin>

and this is enough for my needs. Drives are 2x generic SATA drives at RAID1, nothing special.
After a while (I have just upgraded to 5.2 and want to collect
some stats for this setup), I will try to use rrdcached again. 

My thanks to Collectd developers for their efforts!

-- 
Regards,,
 Pavel                          mailto:pavel2000 at ngs.ru