[collectd] Scalability

[Petri Jarre]

Hi,

I have been briefly testing collectd's scalability to larger number of counters, in the 10K-100K range. Even without the rrdtool plugin this seems to be too much on a single Pentium machine. I have been using the exec plugin with a simple C program that spews out fake numbers for 10000 "cpu's".

The use case is to collect statistics from an application that has about 1000 connections with 30-40 measurable things per connection. Furthermore, there can be multiple servers with load balancing, which multiplies the amount of collected data points by the number of servers. Of course, not everything needs to be measured all the time or with the same granularity, although everything should be accessible all the time. 

I looked at collectd because it had a very promising mindset, but it looks like it does not match the task I have at hand after all. Am I missing some clever way of using collectd for this, or is this just too far from collectd's original purpose?


PJ



       
---------------------------------
Get easy, one-click access to your favorites.  Make Yahoo! your homepage.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.verplant.org/pipermail/collectd/attachments/20071115/0c165a4c/attachment.htm