[collectd] Threads, pthreads, pth, ...

[Niki Waibel]

On 16-Dec-2005 Florian Forster wrote:
> Hi again :)
> 
> On Thu, Dec 15, 2005 at 03:55:56PM +0100, Niki Waibel wrote:
>> okay, so main motivation is not to parallelize the main loop.
>> it is about making some things in some modules parallel.
>> (maybe the main loop will be parallel later... ;-)
> 
> Not quite. The goal is to make the plugins run in parallel, and not things _in_
> the plugins. I.e. one thread per plugin, not multiple threads for one
> plugin. Lets say, `cpu' and `disk' run in parallel, but only one cpu is
> updated at a time.

then you *might* need pthreads for that purpose, although i'd try
pth first. reason: also in cpu.c and the other plugins there are
many read and write calls in which thread switching will be done.

>> example:
>> main thread
>>         \------ping sent to host1
>>         \------ping sent to host2
>>         \------ping sent to host3
>> all 3 processes are waiting.
> 
> Maybe I did get something wrong about `pth', but isn't one consequence
> of userland-threads that you have only one process?

all thread (pthreads, pth, etc) have 1 process.

> I.e. the kernel
> thinks of the programm as being non-threaded?

for non-preemptive threads -> no help from the kernel -> always 1 cpu involved.
preemptive threads -> using kernel syscall -> more cpu's can be involved.

but always 1 process (1 pid) but in the latter case maybe more tasks.

> That means that if one
> thread is waiting the process is waiting. And that would mean that all
> threads are "waiting"..

no. that is the thing *all* threading implemmentations handle.
(pth does that by overwriting the read/write/etc calls via its header
file).

> You got me pretty confused,

sorry...

> so I got out my copy of `Silberschatz'
> <http://codex.cs.yale.edu/avi/os-book/os6/index.html> and read the
> chapter about threads again. And there he clearly states:
> 
> ``The one-to-many model maps many user-level threads to one kernel
>   thread.

wowoww, that is sthg different. all that is about preemptive threads
using the kernel syscall!

> Thread management is done in user space, so it is efficient,
>   but the entire process will block if a thread makes a blocking system
>   call. Also, because only one thread can access the kernel at a time,
>   multiple threads are unable to tun in parallel on multiprocessors.''
>   (Silberschatz, 6th ed., section 5.2.1 `Many-to-one Model', page 132)

i'd recommend you to read
        man pth
it explains almost everything.

> Though - no doubt - this is a n academic book and may describe the
> _concepts_ rather than the read world, I tend to believe him..

the concepts / akademic books talk a lot about the kernel implementation.
if you have a N->N a N->1 or a N->M model.
pth does not need any of them. compleatly in userspace by overwriting
(via define) some functions.

> Also, I don't want to have multiple threads within plugins, because the
> main programm should know as little as possible about the plugins. 

put i think that in the ping module you'll just need at least the pth threads.

>> [...]
>> but imagine that there are 100 hosts, 20 answer in 5sec, 80 in 1sec:
>> you'd now need at least 20*5sec + 80*1sec = 180sec (without threads).
>> using pth you'd need (ideally) a bit more then 5 sec. in reality
>> maybe 6-7 sec (processing of the answers).
>> 
>> using preemptive threads things could be finished within maybe
>> about 5.1sec.
> 
> I think this would only work if we patched `multi host' support into
> libping, so that is uses multiple sockets and `switch' or `poll' (see
> below for I'm not writing my mails linearly ;).
> 
> I wanted to do it in a different fassion, actually:
> - Upon startup, create one thread for each plugin
> - Each thread calls `it's' `init ()' function and then wait's on some
>   condition (pthread_cond_wait(3)).
> - Every 10 seconds the main loop wakes up all threads
>   (pthread_cond_broadcast(3)) and goes back to sleep.
> 
> This means, one loop is done after:
>   main:      2ms
>   ping:    200ms
>   cpu:      10ms
>   hddtemp: 100ms
> 
> So, if a host goes down, the network guy there was dumb enough to block
> all ICMP traffic and the ping plugin has to wait for a timeout, _it_
> (the ping module) will skip one turn while all other modules are updated
> in the meantime.
> 
> Another step might be to ping all hosts in parallel, but
> a) this should be done using `select(2)' or `poll(2)' and
> b) doesn't help if more than one plugin is doing network stuff..
> 
>> again, not convincing, just informing that pth is there as well.
> 
> Well, but I _do_ want to be convinced if I'm doing something wrong or
> could do something better or easier.. ;)

ok. i'd definitely would give pth a go. if it is not sufficient
(fast enough) i'd use pthreads and implement the necessary locking.

;-) 
niki