[collectd] Shouldnt collectd also provide an interface to the data?

[Mariusz Gronczewski]

2009/9/4 Lindsay Holmwood <lindsay at holmwood.id.au>:
> 2009/9/4 Mariusz Gronczewski <xani666 at gmail.com>:
>>
>> I also agree collectd is bad place for implementing any kind of
>> frontend it would just bloat codebase. General problem with any tool
>> using RRD is that its great piece of software but there aren't any
>> good, easy to use "graph creators". There is collection which is nice
>> for start, drraw, which is good but its a bit ugly and afaik dont
>> don't have some advanced RRDtool functions implemented and thats it.
>> If u want something like "get graph with stacked all loadavg's from
>> all servers" or "get total network bandwidth from all servers" u have
>> to write a script which generates it, or update graph in drraw
>> everytime u add/delete host.
>
> The other problem with RRDs is that they generally require you to have
> access to the files themselves to do anything useful with them. When
> you consider a reasonably loaded collectd server pulling in lots of
> stats, adding graphing on top of that is just going to slow you down.
>
> Comparison (like stacked load averages) and correlation (like cpu
> usage vs network io vs ...) are things i'd like to add to Visage. The
> infrastructure to support it is there, just need to write the UI. ;-)
Yeah but then for keeping it in some kind of db you will need much
more space and disk bandwidth because its more expensive to save
values in DB and there is a LOT of data
examples: on average one of my servers have 284 rrd files. if interval
is 10s each server generates 28.4 values per second. When my collectd
server died i fired other saving data to csv so i can export it back
when serwer will be back up, 3 days = 28 milion records in CSV, and i
have about 17 hosts.

10 servers and u already have 284 writes/s which is a lot for DB, but
in case of rrd files u write all data from last 10 minutes every 10
minutes and thats a lot less load than writing it to any kind of
normal DB

Also u have to add some kind of removing archive data, and lot of
other things to basically make "RRD-lookalike DB"
Maybe answer to "you have to have access to local rrd data" would be
some kind of daemon which does "rrdtool dump filename" over net, then
graphing tool connected to it. But then you could just share it via
NFS.

And if ure interested only in actual data u can run collectd on
"graph" node and get data using unixsock plugin


Regards
Mariusz