[collectd] Nightly cron jobs...

[Cyril Feraudet]

> Salut Cyril,
>
> On Wed, Jan 11, 2012 at 08:32:30PM +0100, Cyril Feraudet wrote:
>
>> We use collectd at work (French Bank) to collectd traditional
>> performances of actually 4275 servers. We use also Collectd to 
>> retrieve
>> several information about batch, power consuption, etc by using 
>> custom
>> "types" and "notifications". The easier way is to use unixsock 
>> plugin
>> to send custom statistics and/or notification. About collect 
>> interval
>> you can pick them each second if necessary ...
>
> I'm interested in sharing experience with large setups.
> I guess you're using the network plugin to send data to some collectd
> servers. How many of these do you have? If you're using RRD files, 
> how
> many
> per server? Do you use custom RRAs? rrdcached?
>
> In our case, we monitor 2015 clients, sending data to 4 servers with 
> 100k
> RRD files each. We're currently evaluating rrdcached performance with
> respect
> to plain rrdtool. The load average of the 4 servers is between 1 and 
> 2,
> depending on the number of clients (between 300 and 600). They 
> average
> between 500 and 1000 I/Os per second. Total network throughput
> (collectd-network) is just above 1Mbyte / second.
>

Hi,

We use rrdfile and database (for the last value with plugin 
write_mysql).
About RRA, we keep data at 1 minute interval for 1 year, then each 5 
minutes
for 3 years.

Some numbers :
- 5,7 Mo by rrd
- 470000 rrds
- 3To of rrd
- Less than 700 iops
- 4 instances of collectd on the same server
- Rrdcached wrote ~ 7000 datasets/s
- 4MB of bandwidth

Home made write_mysql plugin store the last value for each metrics, 
then a batch
perform some calcul to have cpu / groups of servers for example, load, 
power
consumption, etc .

Here our home made GUI. I'll be able to share it in GPL soon.

http://feraudet.com/collectd/

Cyril