[collectd] restart collectd daemon stops receiving data from, clients

[Amos Shapira]

2010/1/13 Florian Forster <octo at verplant.org>:
> Hi Amos,
>
> On Sat, Jan 09, 2010 at 08:46:17AM +1100, Amos Shapira wrote:
>> No. We use a virtual IP which floats between the real servers.
>
> are you positive the traffic arrives at the right location then? Some
> network equipment (tries to do /) does weird "state full" stuff to UDP
> traffic..

I'm positive - this happens also when the daemon is restarted on the
same server.

>
> Can you check with `tcpdump -p $OPTS udp port 3333' if the traffic is
> received at the right box.

The traffic keeps arriving to the collector but it doesn't get
collected or graphed.

>
> Please also check if the daemon has actually opened the correct port.
> Under GNU/Linux that's done with `netstat -lnp | grep collectd' or,
> better yet, `lsof -p $PID_OF_COLLECTD'.

It's listening on the right port and IP address.

>
> Does your fallback write on a shared storage? If so, are their clocks
> synchronized?

I'll try to clarify - we have two collectord servers which receive the
data - one is none-HA and another is a pair in HA configuration.
The problems appear in both setups.
Their clock are synchronized using NTP (as are all our servers). The
HA setup uses ext3 file system on top of DRBD to share the storage in
active/passive mode.
But this happens also on a collector which is a single, none-HA server.

>
>> Sorry I mentioned the fail-over. It just added noise to my question.
>
> Actually, the more details the better ;)
>
> When no graphs are updated, can you see any messages in the logfile on
> the server?

No messages at all, tried even in debug mode.

>
> Sorry I'm questioning your network setup so much, but version 4.6 is
> 11 months old and so far you're the first to describe this "state full"
> behavior of collectd's network plugin.

No worries. I appreciate any effort you do to help me solve this.

It's too late today to do such an upgrade but I'll upgrade to 4.8
tomorrow morning (we compile our own rpm's to avoid pulling in huge
amount of plugin dependencies).

Thanks!

--Amos