[collectd] How many hosts MAX?

[Florian Forster]

Hi Sergiusz,

On Thu, Apr 12, 2007 at 11:02:14PM +0100, Sergiusz Pawlowicz wrote:
> how many hosts do you monitor by collectd via client-server
> architecture?

since I don't have collectd running in a `big' setup I'd be very
interested in that information myself.

At home I monitor two to five machines, depending on how many are
currently running. 

> What is your working in production environment maximum?

As far as I can tell the bottleneck is not the daemon or the network,
but writing the RRD-files.[*] I usually update around 200-300 RRD-files
(collectd 3: ~20-30 updated/second).

With collectd 4 and caching enabled (timeout set to 300 seconds) the
IO-wait is basically gone. It's really hard to tell but I think that
writing 30 values at once will allow for more than `only' 30 times the
files. If a system can handle 200 updates/sec (which, I think, is
realistic) and each RRD-file is updated every 300 seconds you get 600000
files. That's just an estimate/guess, though.

> Do you use collectd4 or collectd3?

Both, of course ;)

Regards,
-octo


[*] This is weird since the update always accesses similar regions in
    the file, so the OS _should_ be able to cache that efficiently. But
    even updating a relatively small number of RRD-files causes
    relatively much IO-wait. My guess is that the locking is to blame,
    since it may force a flush/sync. There has been some discussion on
    this on the RRDTool mailinglist unter the subject ``How to get the
    most performance when using lots of RRD files''. The common solution
    of this is to combine several values into one update which is
    implemented in collectd 4. This, indeed, reduces IO-wait by several
    magnitudes.
-- 
Florian octo Forster
Hacker in training
GnuPG: 0x91523C3D
http://verplant.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mailman.verplant.org/pipermail/collectd/attachments/20070413/2949e54e/attachment.pgp