[collectd] Max number of UDP sockets per collectd-process (NOT the file descriptor limit)

[DavidHalko]

I guess poll is not an option???

 :-(

I would REALLY like poll, but I am not a C programmer... :-(

Dave
http://netmgt.blogspot.com/

Sent from my iPhone

On Apr 13, 2012, at 7:00 AM, Stian Øvrevåge <sovrevage at gmail.com> wrote:

> Found this handy anecdote on http://collectd.org/wiki/index.php/Plugin:SNMP
> 
> "Maximum number of hosts
> While collectd and the SNMP plugin don't have any limitation on the
> number of hosts you can configure, the library used by the SNMP
> plugin, libnetsnmp, uses the select(2) system call. This system call
> uses a fixed-size bitfield to hold file descriptors. On many systems
> this limits the number of hosts you can query with the SNMP plugin to
> 1024 (for example when using the GNU libc).
> 
> To solve this issue, the netsnmp library must be changed. A solution
> would be to switch to the poll(2) system call which doesn't have a
> static limit on the largest file descriptor it can handle."
> 
> So my current work-around and using several collectd processes seems
> to be a permanent one :-)
> 
> Brgds
> Stian Øvrevåge
> 
> On Thu, Apr 12, 2012 at 11:21, Stian Øvrevåge <sovrevage at gmail.com> wrote:
>> Hi list,
>> 
>> Banging my head against the wall for weeks now trying to get a working
>> medium scale collectd-installation working...
>> 
>> I thought I had fixed the max number of sockets/connecting when tuning
>> /etc/security/limits.conf. It now reads:
>> 
>>    ulimit -n
>>    32768
>> 
>> I have the instances of collectd now. One of them is set to poll 2300
>> hosts. Of which an unknown number is offline at any time. I'm watching
>> strace as well as netstat and everything seems fine and "netstat -anop
>> udp|wc -l" counts the number of udp sockets created until the number
>> hits about 1092. Here it stalls and syslog logs thousand lines of
>> 
>>    "Apr 12 11:07:41 collectd-new collectd[1488]: snmp plugin: host
>> x.y.z: snmp_sess_synch_response failed:"
>> 
>> within a few seconds. Number of UDP sockets from then on are stable.
>> 
>> If I also start the other two instances the number of sockets grows to
>> 1292. Which leads me to believe that there is a per-process(or
>> thread?) limit somewhere.
>> 
>> Information on the internet on the issue is scarce other than the file
>> descriptor limit which I believe is unrelated.
>> 
>> Regards,
>> Stian Øvrevåge
> 
> _______________________________________________
> collectd mailing list
> collectd at verplant.org
> http://mailman.verplant.org/listinfo/collectd