[collectd] v5upgrade target causing high IO wait, no RRD updates

[GRAY Andrew G (SPARQ)]

Not sure about debian packaging ... but 5.4 rpm's on centos package the collectd rrdcached plugin separately.
Check it's there. Check the rrdcached daemon and collectd config are referencing the same pipe. 
One thing I did notice ... new rrd db files are created with the root owner, rrdcached is running as low priv, implying that collectd is creating the rdd dirs, and then rrdcached cannot write to them. Cron re-chown fix is not very elegant but does work. 

My 5.4 + rrdcached 4.7 is now running pretty smoothly and I'm really happy about all the IO wait going away :)


Regards,

Andrew Gray.
RHCSA, Professional Unix Administration.


-----Original Message-----
From: collectd-bounces at verplant.org [mailto:collectd-bounces at verplant.org] On Behalf Of Jesse Reynolds
Sent: Tuesday, 8 April 2014 4:52 PM
To: collectd at verplant.org
Subject: [collectd] v5upgrade target causing high IO wait, no RRD updates

Hi all, 

I've just upgraded one of our collectd servers from 4.10.1 to 5.4.1 with rrdcached (1.4.7). Woo! One thing that surprised me was that if I had the v5upgrade target enabled, then RRDs weren't being updating (or not that I could discern) and top was saying 99% IO wait (compare about 10 to 20% without v5upgrade). So I'm currently running it without the v5upgrade target. 

Has anyone seen anything like this? Or have any idea what might be going on to cause this? I didn't see this behaviour on my testing prior to the upgrade. The main difference is in the size of the RRD files. 

I also ran `iostat 5` when testing and the IO wait stayed at about 99% consistently even when there were periods of no disk activity to speak of. 

This is on ubuntu precise 12.04, 64 bit, no virtualisation. 

TIA
Jesse


_______________________________________________
collectd mailing list
collectd at verplant.org
http://mailman.verplant.org/listinfo/collectd
To report this email as spam, please forward to asa at websense.com
 
*************************************************************************************
This email message (including any file attachments transmitted with it) is for the sole use of the intended recipient(s) and may contain confidential and legally privileged information. Any unauthorised review, use, alteration, disclosure or distribution of this email (including any attachments) by an unintended recipient is prohibited. If you have received this email in error, please notify the sender by return email and destroy all copies of the original message. Any confidential or legal professional privilege is not waived or lost by any mistaken delivery of the email. SPARQ Solutions accepts no responsibility for the content of any email which is sent by an employee which is of a personal nature.

Sender Details:
  SPARQ Solutions
  PO Box 15760 City East, Brisbane QLD Australia 4002
  +61 7 4931 2222

SPARQ Solutions policy is to not send unsolicited electronic messages. Suspected breaches of this policy can be reported by replying to this message including the original message and the word "UNSUBSCRIBE" in the subject. 

*************************************************************************************