[collectd] randomly getting dl_open_worker assertion

Florian Forster octo at collectd.org
Thu Feb 21 11:03:54 CET 2013

Hi Robert,

On Wed, Feb 20, 2013 at 04:21:16PM +0000, Riches Jr, Robert M wrote:
> [2013-02-20 08:01:31] exec plugin: exec_read_one: error = Inconsistency detected by ld.so: dl-open.c: 221: dl_open_worker: Assertion `_dl_debug_initialize (0, args->nsid)->r_state == RT_CONSISTENT' failed!

this appears to be an assertion within glibc's implementation of
dlopen(3). [0] It looks like this bug from 2007 could be related: [1]

> There doesn't seem to be any rhyme or reason as to whether I get the
> expected result or the assertion failure.  I've googled for answers
> until my keyboard is wearing out, but nothing has come up that shows
> promise of a solution.

From what you describe, it feels like a concurrency issue. collectd is
using dlopen() to load the plugins, including the exec plugin. This
happens at start-up only; later the mechanism is no longer used, but the
dlopen'ed shared object are never closed, so they are still open when
fork() is called.

> Regarding the behavior when I run the real script that doesn't send
> anything to stderr, […]

I don't think this is related to I/O. It sounds more like a problem
between dlopen() and fork().

How many processors does the machine have on which this problem occurs?
Which libc are you using? Approximately, how often does this happen?

Best regards,

[0] <http://code.woboq.org/userspace/glibc/elf/dl-open.c.html#259>
[1] <http://www.sourceware.org/bugzilla/show_bug.cgi?id=4578>
collectd – The system statistics collection daemon
Website: http://collectd.org
Google+: http://collectd.org/+
GitHub:  https://github.com/collectd
Twitter: http://twitter.com/collectd
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://mailman.verplant.org/pipermail/collectd/attachments/20130221/e0e91c01/attachment.pgp>

More information about the collectd mailing list