[collectd] Batching writes from the same plugin?

[Dragos Manolescu]

Hi --

(Please contact me by direct email for clarifications and/or follow up, I am not subscribed to the mailing list.)

I wrote a collectd Java plugin that publishes the written value lists to an Apache Kafka topic. If possible, I'd like to batch in the same message values related to each other.

For example, instead of publishing 3 separate messages corresponding to what's coming from the memory plugin:

{"info":{"collectionTimestamp":"2013-02-26T10:58:17.829-08:00","host":"dragosm-linux","plugin":"memory","type":"memory","pi":"","ti":"used"},"readings":[{"name":"value","type":"gauge","value":5.00625408E8}]}
{"info":{"collectionTimestamp":"2013-02-26T10:58:17.830-08:00","host":"dragosm-linux","plugin":"memory","type":"memory","pi":"","ti":"buffered"},"readings":[{"name":"value","type":"gauge","value":5.4493184E7}]}
{"info":{"collectionTimestamp":"2013-02-26T10:58:17.830-08:00","host":"dragosm-linux","plugin":"memory","type":"memory","pi":"","ti":"cached"},"readings":[{"name":"value","type":"gauge","value":3.43908352E8}]}

If viable I would like to send a single message that contains used, buffered and cached together. I have two questions about doing that:

  1.  Are the writes from the same plugin/type/plugin instance atomic? So far I've never seen writes from different plugins interleaved but I don't want to assume that.
  2.  Is there a mechanism that signals when a plugin starts/stops writing (or alternatively to determine in advance how many writes the plugin is going to make) so I could batch the values together?

Thanks,

-Dragos

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.verplant.org/pipermail/collectd/attachments/20130226/fcc9ea20/attachment.html>