[collectd] zeormq architecture inquiry

[Shanker Balan]

Hello,

Allan Feid wrote,
> Hello,
> 
> I've been looking into using collectd combined with zeromq for transport.
> This seems like a fairly good way to scale out monitoring infrastructure as
> each node being monitored can send out requests over zeromq to one or more
> locations with little overhead. I was wondering if anyone has any experience
> doing this, or can provide some guidance on what zeromq architecture would
> scale best.
> 
> With my limited zeromq knowledge, I'm thinking that each node being
> monitored would publish to one or more "collector" nodes, which could then
> reroute the information down to a single point where graphing and alerting
> thresholds could be handled. The "collector" nodes would likely be in each
> datacenter and running collectd to write to RRD files. These instances of
> collectd, could also be publishing or pushing data down to a
> single aggregation system. I guess my real question here, is what zeromq
> pattern would make most sense for a distributed collectd monitoring system.
> Any insight or suggestions are greatly appreciated.

My team initially started out with collectd but we ran into a requirement
where we needed to have the app itself emit events in real time. It was not
clear if the collectd libraries could be used within Java, Perl/Ruby etc
applications.

After a bit of grepping around, we settled on MonDemand with LWES
(UDP+Multicast) as the transport. ZeroMQ was an option, but we liked what we
saw in MonDemand.

If the need arises in the future, we plan to replace LWES with a MQ system.

lwes.sf.net
mondemand.sf.net

Hth.

-- 
http://shankerbalan.net/