[collectd] Max number of UDP sockets per collectd-process (NOT the file descriptor limit)

[Teet Talviste]

You can take a look at this, maybe it helps you

https://github.com/frogmaster/collectd/commit/67c4863e0aaadaa103ee07e49a17a1510e8d4eaf

> Found this handy anecdote on http://collectd.org/wiki/index.php/Plugin:SNMP
> 
> "Maximum number of hosts
> While collectd and the SNMP plugin don't have any limitation on the
> number of hosts you can configure, the library used by the SNMP
> plugin, libnetsnmp, uses the select(2) system call. This system call
> uses a fixed-size bitfield to hold file descriptors. On many systems
> this limits the number of hosts you can query with the SNMP plugin to
> 1024 (for example when using the GNU libc).
> 
> To solve this issue, the netsnmp library must be changed. A solution
> would be to switch to the poll(2) system call which doesn't have a
> static limit on the largest file descriptor it can handle."
> 
> So my current work-around and using several collectd processes seems
> to be a permanent one :-)
> 
> Brgds
> Stian Øvrevåge
> 
> On Thu, Apr 12, 2012 at 11:21, Stian Øvrevåge <sovrevage at gmail.com> wrote:
> > Hi list,
> >
> > Banging my head against the wall for weeks now trying to get a working
> > medium scale collectd-installation working...
> >
> > I thought I had fixed the max number of sockets/connecting when tuning
> > /etc/security/limits.conf. It now reads:
> >
> >    ulimit -n
> >    32768
> >
> > I have the instances of collectd now. One of them is set to poll 2300
> > hosts. Of which an unknown number is offline at any time. I'm watching
> > strace as well as netstat and everything seems fine and "netstat -anop
> > udp|wc -l" counts the number of udp sockets created until the number
> > hits about 1092. Here it stalls and syslog logs thousand lines of
> >
> >    "Apr 12 11:07:41 collectd-new collectd[1488]: snmp plugin: host
> > x.y.z: snmp_sess_synch_response failed:"
> >
> > within a few seconds. Number of UDP sockets from then on are stable.
> >
> > If I also start the other two instances the number of sockets grows to
> > 1292. Which leads me to believe that there is a per-process(or
> > thread?) limit somewhere.
> >
> > Information on the internet on the issue is scarce other than the file
> > descriptor limit which I believe is unrelated.
> >
> > Regards,
> > Stian Øvrevåge
> 
> _______________________________________________
> collectd mailing list
> collectd at verplant.org
> http://mailman.verplant.org/listinfo/collectd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.verplant.org/pipermail/collectd/attachments/20120414/ba5f78f8/attachment-0001.html>