[collectd] Threads, pthreads, pth, ...

[Florian Forster]

Hi again :)

On Thu, Dec 15, 2005 at 03:55:56PM +0100, Niki Waibel wrote:
> okay, so main motivation is not to parallelize the main loop.
> it is about making some things in some modules parallel.
> (maybe the main loop will be parallel later... ;-)

Not quite. The goal is to make the plugins run in parallel, and not things _in_
the plugins. I.e. one thread per plugin, not multiple threads for one
plugin. Lets say, `cpu' and `disk' run in parallel, but only one cpu is
updated at a time.

> example:
> main thread
>         \------ping sent to host1
>         \------ping sent to host2
>         \------ping sent to host3
> all 3 processes are waiting.

Maybe I did get something wrong about `pth', but isn't one consequence
of userland-threads that you have only one process? I.e. the kernel
thinks of the programm as being non-threaded? That means that if one
thread is waiting the process is waiting. And that would mean that all
threads are "waiting"..

You got me pretty confused, so I got out my copy of `Silberschatz'
<http://codex.cs.yale.edu/avi/os-book/os6/index.html> and read the
chapter about threads again. And there he clearly states:

``The one-to-many model maps many user-level threads to one kernel
  thread. Thread management is done in user space, so it is efficient,
  but the entire process will block if a thread makes a blocking system
  call. Also, because only one thread can access the kernel at a time,
  multiple threads are unable to tun in parallel on multiprocessors.''
  (Silberschatz, 6th ed., section 5.2.1 `Many-to-one Model', page 132)

Though - no doubt - this is a n academic book and may describe the
_concepts_ rather than the read world, I tend to believe him..

Also, I don't want to have multiple threads within plugins, because the
main programm should know as little as possible about the plugins. 

> [...]
> but imagine that there are 100 hosts, 20 answer in 5sec, 80 in 1sec:
> you'd now need at least 20*5sec + 80*1sec = 180sec (without threads).
> using pth you'd need (ideally) a bit more then 5 sec. in reality
> maybe 6-7 sec (processing of the answers).
> 
> using preemptive threads things could be finished within maybe
> about 5.1sec.

I think this would only work if we patched `multi host' support into
libping, so that is uses multiple sockets and `switch' or `poll' (see
below for I'm not writing my mails linearly ;).

I wanted to do it in a different fassion, actually:
- Upon startup, create one thread for each plugin
- Each thread calls `it's' `init ()' function and then wait's on some
  condition (pthread_cond_wait(3)).
- Every 10 seconds the main loop wakes up all threads
  (pthread_cond_broadcast(3)) and goes back to sleep.

This means, one loop is done after:
  main:      2ms
  ping:    200ms
  cpu:      10ms
  hddtemp: 100ms

So, if a host goes down, the network guy there was dumb enough to block
all ICMP traffic and the ping plugin has to wait for a timeout, _it_
(the ping module) will skip one turn while all other modules are updated
in the meantime.

Another step might be to ping all hosts in parallel, but
a) this should be done using `select(2)' or `poll(2)' and
b) doesn't help if more than one plugin is doing network stuff..

> again, not convincing, just informing that pth is there as well.

Well, but I _do_ want to be convinced if I'm doing something wrong or
could do something better or easier.. ;)

Regards,
-octo
-- 
Florian octo Forster
Hacker in training
GnuPG: 0x91523C3D
http://verplant.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mailman.verplant.org/pipermail/collectd/attachments/20051216/88b9402c/attachment.pgp