[collectd] Single graph HA and central data polling

[dormando]

Hey,

I'm trying to do similar with ganglia, but am also testing out collectd
since it claims higher efficiency/etc. It's up and working, but I have two
situations I'm trying to iron out and would enjoy some ML feedback.

One is a DRBD'ed HA cluster. I want graphs for the individual machines.
CPU usage, interface, disk io, etc. Then I'd prefer a centralized graph
for the shared services. MySQL, bind, apache, etc. This is to be able to
view a consistent graph through failovers. I see the two options as being
some hack to generate a fake server whose stats are attached to those
particular graphs, or convincing rrdtool to cleanly overlay the data from
both sides into a single graph. Latter seems easier, but is anyone doing
either?

The other issue is something that doesn't really work with the
munin/ganglia/collectd style. Lets say I have a stats parsing tool for
memcached that is used for quickly calculating and viewing the various
stats counters and calculated stats (hit rate, fill rate, etc). My program
can very quickly emit stats for all of the servers in our cluster, but how
do I cleanly get that data into a collectd cluster?

It's a poller so ... I have no idea. ganglia has gmetric spoofing but it's
pretty janky. Other approaches are to just eat it and generate all of the
graphs locally to the machine executing the polling, or again generate a
fake "memcachedcluster" server which just contains these graphs.

Anyone already doing anything like this, or have ideas/pointers on the
cleanest ways to do this with collectd? I've only been using the project
for two days but would like to increase the chances of returning useful
code to the project or community.

Thanks,
-Dormando