[collectd] Preventing collectd high memory usage

[Yves Mettier]

Hello again,

For your information, I created a Pull Request in order to improve the 
doc.

https://github.com/collectd/collectd/pull/1471

Regards,
Yves

Le 2015-12-15 10:37, Yves Mettier a écrit :
> Hello,
> 
> You can read this : https://github.com/collectd/collectd/pull/691
> 
> In short :
> 
> queue_length
> nb values incoming. This is what you want to limit with
> WriteQueueLimitHigh and WriteQueueLimitLow.
> 
> derive-dropped
> this is what you drop when queue_length is greater than the limits you 
> set.
> If derive-dropped is nul, either everything is going well or you set
> WriteQueueLimitHigh and WriteQueueLimitLow too high.
> If derive-dropped is never nul, your problem is either that
> WriteQueueLimitHigh and WriteQueueLimitLow are too low, or you have a
> bottleneck somewhere.
> 
> cache_size
> same as "echo LISTVAL | nc -u /var/.../collectd.sock | wc -l"
> 
> Regards,
> Yves
> 
> Le 2015-12-14 20:09, Vaughn Wernersbach a écrit :
>> Hi André Luiz da Silva!  Thanks for the suggestion.  I’ll keep
>> switching to UDP in mind, though I’m inclined to stick with TCP if at
>> all possible (largely to minimize the required changes to my
>> environment).
>> 
>> All - Anyone have info on using CollectInternalStats?  Still need to
>> know what value to set WriteQueueLimitHigh to and having collectd tell
>> me definitely seems like the best bet.
>> 
>> 
>> 
>> Vaughn Wernersbach
>> BIDS Trading, L.P.
>> 
>> 111 Broadway, Suite 1603 | New York, NY 10006
>> T: 212.618.2071 |  F: 212.618.2031  |  C: 732.397.8344  | E:
>> vaughn.wernersbach at bidstrading.com<mailto:vaughn.wernersbach at bidstrading.com>
>> 
>> From: André Luiz da Silva [mailto:andresilvaluiz1 at gmail.com]
>> Sent: Monday, December 14, 2015 11:18 AM
>> To: Vaughn Wernersbach <Vaughn.Wernersbach at bidstrading.com>
>> Cc: collectd at verplant.org
>> Subject: Re: [collectd] Preventing collectd high memory usage
>> 
>> 
>> Hi, Wenersbach! Did you try to send your metrics over UDP instead of
>> TCP? It's just a suggestion if you have a reliable network.
>> 
>> 
>> 
>> Regards.
>> 
>> 
>> 
>> Em 14/12/2015 1:54 PM, "Vaughn Wernersbach"
>> <Vaughn.Wernersbach at bidstrading.com<mailto:Vaughn.Wernersbach at bidstrading.com>>
>> escreveu:
>> Hi,
>> 
>> In order to limit high memory usage (one of my servers hit 94% for
>> collectd) I’m interested in enabling the
>> WriteQueueLimitHigh<https://collectd.org/documentation/manpages/collectd.conf.5.shtml#writequeuelimithigh_highnum>
>> option, but I don’t know what value to use.  The docs suggest enabling
>> CollectInternalStats<https://collectd.org/documentation/manpages/collectd.conf.5.shtml#collectinternalstats_false_true>,
>> and that the “write_queue” plugin instance reports the info from
>> CollectInternalStats that I’m looking for.  However, I don’t see any
>> “write_queue” plugin nor do I see anything relevant being written by
>> my existing write plugins (syslog and graphite) after enabling
>> CollectInternalStats.
>> 
>> 
>> •         How do I obtain data written collected when setting
>> CollectInternalStats to true?
>> 
>> •         Alternately, are there any guidelines for setting the values
>> of WriteQueueLimitHigh?
>> 
>> 
>> 
>> Vaughn Wernersbach
>> BIDS Trading, L.P.
>> 
>> 111 Broadway, Suite 1603 | New York, NY 10006
>> T: 212.618.2071 |  F: 212.618.2031  |  C: 732.397.8344  | E:
>> vaughn.wernersbach at bidstrading.com<mailto:vaughn.wernersbach at bidstrading.com>
>> 
>> 
>> _______________________________________________
>> collectd mailing list
>> collectd at verplant.org<mailto:collectd at verplant.org>
>> http://mailman.verplant.org/listinfo/collectd
>> 
>> _______________________________________________
>> collectd mailing list
>> collectd at verplant.org
>> http://mailman.verplant.org/listinfo/collectd