[collectd] Preventing collectd high memory usage

[Yves Mettier]

Hello,

You can read this : https://github.com/collectd/collectd/pull/691

In short :

queue_length
nb values incoming. This is what you want to limit with 
WriteQueueLimitHigh and WriteQueueLimitLow.

derive-dropped
this is what you drop when queue_length is greater than the limits you 
set.
If derive-dropped is nul, either everything is going well or you set 
WriteQueueLimitHigh and WriteQueueLimitLow too high.
If derive-dropped is never nul, your problem is either that 
WriteQueueLimitHigh and WriteQueueLimitLow are too low, or you have a 
bottleneck somewhere.

cache_size
same as "echo LISTVAL | nc -u /var/.../collectd.sock | wc -l"

Regards,
Yves

Le 2015-12-14 20:09, Vaughn Wernersbach a écrit :
> Hi André Luiz da Silva!  Thanks for the suggestion.  I’ll keep
> switching to UDP in mind, though I’m inclined to stick with TCP if at
> all possible (largely to minimize the required changes to my
> environment).
> 
> All - Anyone have info on using CollectInternalStats?  Still need to
> know what value to set WriteQueueLimitHigh to and having collectd tell
> me definitely seems like the best bet.
> 
> 
> 
> Vaughn Wernersbach
> BIDS Trading, L.P.
> 
> 111 Broadway, Suite 1603 | New York, NY 10006
> T: 212.618.2071 |  F: 212.618.2031  |  C: 732.397.8344  | E:
> vaughn.wernersbach at bidstrading.com<mailto:vaughn.wernersbach at bidstrading.com>
> 
> From: André Luiz da Silva [mailto:andresilvaluiz1 at gmail.com]
> Sent: Monday, December 14, 2015 11:18 AM
> To: Vaughn Wernersbach <Vaughn.Wernersbach at bidstrading.com>
> Cc: collectd at verplant.org
> Subject: Re: [collectd] Preventing collectd high memory usage
> 
> 
> Hi, Wenersbach! Did you try to send your metrics over UDP instead of
> TCP? It's just a suggestion if you have a reliable network.
> 
> 
> 
> Regards.
> 
> 
> 
> Em 14/12/2015 1:54 PM, "Vaughn Wernersbach"
> <Vaughn.Wernersbach at bidstrading.com<mailto:Vaughn.Wernersbach at bidstrading.com>>
> escreveu:
> Hi,
> 
> In order to limit high memory usage (one of my servers hit 94% for
> collectd) I’m interested in enabling the
> WriteQueueLimitHigh<https://collectd.org/documentation/manpages/collectd.conf.5.shtml#writequeuelimithigh_highnum>
> option, but I don’t know what value to use.  The docs suggest enabling
> CollectInternalStats<https://collectd.org/documentation/manpages/collectd.conf.5.shtml#collectinternalstats_false_true>,
> and that the “write_queue” plugin instance reports the info from
> CollectInternalStats that I’m looking for.  However, I don’t see any
> “write_queue” plugin nor do I see anything relevant being written by
> my existing write plugins (syslog and graphite) after enabling
> CollectInternalStats.
> 
> 
> •         How do I obtain data written collected when setting
> CollectInternalStats to true?
> 
> •         Alternately, are there any guidelines for setting the values
> of WriteQueueLimitHigh?
> 
> 
> 
> Vaughn Wernersbach
> BIDS Trading, L.P.
> 
> 111 Broadway, Suite 1603 | New York, NY 10006
> T: 212.618.2071 |  F: 212.618.2031  |  C: 732.397.8344  | E:
> vaughn.wernersbach at bidstrading.com<mailto:vaughn.wernersbach at bidstrading.com>
> 
> 
> _______________________________________________
> collectd mailing list
> collectd at verplant.org<mailto:collectd at verplant.org>
> http://mailman.verplant.org/listinfo/collectd
> 
> _______________________________________________
> collectd mailing list
> collectd at verplant.org
> http://mailman.verplant.org/listinfo/collectd