[collectd] Collectd sysctl 'benchmark'

[Toni Ylenius]

Hi list,

I had a short conversation with Kimo about sysctlnametomib() versus 
sysctlbyname() [1] and I wanted to check if there is really a difference. At 
the same time I 'benchmarked' some other plugins. I think that a tool like 
this might aid in collectd plugin development.

I added a -b <N> flag to the collectd which runs "benchmark" by iterating the 
read loop N times and calculates total CPU time. To collect results I wrote a 
simple Makefile script. 

I ran benchmarks on a FreeBSD 8.2 system. I ran 5 instances of collectd 
simultaneously and every instance ran 100000 iterations. The collectd was 
configured to use only the plugin in question. I set the time interval as 
small as possible so that  there weren't too many 'Not sleeping because the 
next interval is 0.000 seconds in the past!' reports. 

Results (seconds of CPU time):
         Plugin:    Mean:    Variance:
         cpu.in   10.03281    0.00035
   cs-sysctl.in    3.49531    0.00312 (uses sysctlnametomib() and sysctl())
          cs.in    3.74687    0.00043 (uses sysctlbyname())
        disk.in    8.10156    0.00020
   interface.in   13.22031    0.00062
        load.in    3.31562    0.00556
      memory.in    8.67500    0.00041
        swap.in    1.82500    0.00761
    tcpconns.in   16.11094    0.00162
      uptime.in    3.16562    0.00160
       users.in   13.93281    0.00080
     zfs-arc.in   16.66094    0.00294 (modified to use sysctlbyname())
     zfs-arc.in   13.26250    0.00158 (mod. sysctlnametomib() and sysctl())

We can see that plugins use different amount of CPU. This is not real 
benchmark because values are not comparable to those produced on other 
systems. However we can easily calculate that if we use 30 seconds time 
interval 100000 iterations would mean 34 hours of real time, and we can see 
that some plugins are quite efficient.

I would like to point out two things about sysctl calls.

First, lets compare ContextSwitch plugin and my sysctl plugin [2]. It seems 
that when a plugin uses only one sysctl entry there is no significant 
difference between sysctlnametomib() and sysctlbyname(). Maybe collectd 
overhead is larger that the small performance gain. 

Second, we can see that zfs-arc plugin is faster when sysctlnametomib() is 
used. The plugin reads 16 different sysctl values on every read action and now 
the difference is noticeable.

There are two attachments:
- collectd-bench.patch (patch against version 5.0.0)
- time-suite.tgz
I didn't post zfs-arc plugin patches because they are poor quality. I will 
return to this later.

Thanks for your time.
Toni Ylenius

[1] http://mailman.verplant.org/pipermail/collectd/2011-June/004542.html
[2] https://collectd.org/gerrit/#change,10
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.verplant.org/pipermail/collectd/attachments/20110929/fb97723a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: collectd-bench.patch
Type: text/x-patch
Size: 4468 bytes
Desc: not available
URL: <http://mailman.verplant.org/pipermail/collectd/attachments/20110929/fb97723a/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: time-suite.tgz
Type: application/x-compressed-tar
Size: 1318 bytes
Desc: not available
URL: <http://mailman.verplant.org/pipermail/collectd/attachments/20110929/fb97723a/attachment-0001.bin>