[collectd] Generalized ignorelist functionality

[Luboš Staněk]

Hi Florian,
I tested the 'ignorelist' and df plugin. I sent the patches to you directly.

One good message: I have stayed with the git master! The changes are so
useful that it made me to throw away the 3.10.3!


Luboš Staněk napsal(a):
> Hi Florian,
> 
>> No. Yes. Well, just _don't_ use string-matching and always use regexes.
>> If you provide a simple string like `foobar' the regex stuff does the
>> same without much overhead, but you have the power right there if you
>> need it. Thoughts and other opinions are welcome :)
>>
> 
> It is, of course, possible.
> Using regexes even for simple string compare simplifies things.
> 

I cannot get rid of the suspicion that some 'strings' cannot work as
'regexes' right without prefixing particular characters.

I am unusable today, my mind is currently floating elsewhere (step by
the wrong leg from the bed). :(


>> Apart from these additional notices: I've merged your code into the
>> master branch and to have used the interface at least once myself I
>> implemented some selection stuff for the `df' plugin. You can now
>> select/ignore partitions depending on (fs-)type, mountpoint and device.
>> If you want to use the regex-stuff for matching mountpoints, you will
>> need to use stuff like this:
>>   Device /^/(media|proc|sys)/

I fixed it in the patch.
The 'Device' should be *_cu_mount_t->spec_device.
I have thrown off the '/dev/shm' graph! :)


>> Since the code right now simple removes a leading and a trailing slash,
>> you don't need to escape internal slashes. That's confusing. Also, the
>> way to use the string-matching code is like this:
>>   Device /media/cdrom
>> This is _not_ parsed as a regular expression, because it's _not_
>> followed by a slash. That's confusing, too.
>>
> 

Yes, it is confusing.
See my point above.


I changed the df plugin to provide more 'debug' information because I
could not get it working initially.
Lately I discovered that the IgnoreSelected is not processed right and
that the 'Device' points to another structure member.
Additionally I added one debug message to see the 'statbuf.f_blocks'.

You can see that both the string and regex ignorelist entries work well.
The log documents also the volume label used as a mount point with the
USB harddrive (Do you remember troubles with the HAL?).


Best regards,
Lubos


-------------
Here is the collectd debug log extract after fixing troubles:

configfile.c:492:cf_callback_plugin_dispatch(): shortvar = Device, var =
plugin.Device, arguments = (null), value = /sd[b,c]/, ...
configfile.c:117:cf_dispatch(): type = df, key = Device, value = /sd[b,c]/
utils_ignorelist.c:218:ignorelist_create(): Ignorelist created
0x0x9c4b658, default is collect
utils_ignorelist.c:218:ignorelist_create(): Ignorelist created
0x0x9c4b668, default is collect
utils_ignorelist.c:218:ignorelist_create(): Ignorelist created
0x0x9c4b678, default is collect
utils_ignorelist.c:314:ignorelist_add(): I'm about to add regex entry:
sd[b,c]
utils_ignorelist.c:130:ignorelist_append_regex(): regex compiled:
sd[b,c] - 0
configfile.c:150:cf_dispatch(): return (0)
configfile.c:492:cf_callback_plugin_dispatch(): shortvar = FSType, var =
plugin.FSType, arguments = (null), value = tmpfs, ...
configfile.c:117:cf_dispatch(): type = df, key = FSType, value = tmpfs
utils_ignorelist.c:321:ignorelist_add(): to add entry: tmpfs
configfile.c:150:cf_dispatch(): return (0)
configfile.c:492:cf_callback_plugin_dispatch(): shortvar =
IgnoreSelected, var = plugin.IgnoreSelected, arguments = (null), value =
true, ...
configfile.c:117:cf_dispatch(): type = df, key = IgnoreSelected, value =
true
configfile.c:150:cf_dispatch(): return (0)

utils_mount.c:570:cu_mount_getmntent(): (void); COLLECTD_MNTTAB = /etc/mtab
utils_mount.c:593:cu_mount_getmntent(): new = {dir = /, spec_device =
/dev/mapper/VolGroup00-LogVol00, type = ext3, options = rw,user_xattr,
device = rw,user_xattr}
utils_mount.c:593:cu_mount_getmntent(): new = {dir = /proc, spec_device
= /dev/proc, type = proc, options = rw, device = rw}
utils_mount.c:593:cu_mount_getmntent(): new = {dir = /sys, spec_device =
/dev/sys, type = sysfs, options = rw, device = rw}
utils_mount.c:593:cu_mount_getmntent(): new = {dir = /dev/pts,
spec_device = /dev/devpts, type = devpts, options = rw,gid=5,mode=620,
device = rw,gid=5,mode=620}
utils_mount.c:593:cu_mount_getmntent(): new = {dir = /boot, spec_device
= /dev/sda1, type = ext3, options = rw, device = rw}
utils_mount.c:593:cu_mount_getmntent(): new = {dir = /dev/shm,
spec_device = /dev/shm, type = tmpfs, options = rw, device = rw}
utils_mount.c:593:cu_mount_getmntent(): new = {dir = /home/samba/media,
spec_device = /dev/hda3, type = vfat, options =
rw,noexec,nosuid,nodev,uid=999,gid=1006,umask=011,dmask=022, device =
rw,noexec,nosuid,nodev,uid=999,gid=1006,umask=011,dmask=022}
utils_mount.c:593:cu_mount_getmntent(): new = {dir = /var/lib/cddbd,
spec_device = /dev/hda4, type = xfs, options = rw,noatime, device =
rw,noatime}
utils_mount.c:593:cu_mount_getmntent(): new = {dir =
/proc/sys/fs/binfmt_misc, spec_device = none, type = binfmt_misc,
options = rw, device = rw}
utils_mount.c:593:cu_mount_getmntent(): new = {dir = /media/TRANS,
spec_device = /dev/sdb1, type = vfat, options =
rw,noexec,nosuid,nodev,shortname=winnt,uid=999, device =
rw,noexec,nosuid,nodev,shortname=winnt,uid=999}
utils_mount.c:610:cu_mount_getmntent(): return (0x0x9c54680)

df.c:191:df_read(): statbuf.f_blocks: 58610850| spec_device:
/dev/mapper/VolGroup00-LogVol00| dir: /| type: ext3
df.c:215:df_read(): name: root| spec_device:
/dev/mapper/VolGroup00-LogVol00| dir: /| type: ext3
df.c:191:df_read(): statbuf.f_blocks: 0| spec_device: /dev/proc| dir:
/proc| type: proc
df.c:191:df_read(): statbuf.f_blocks: 0| spec_device: /dev/sys| dir:
/sys| type: sysfs
df.c:191:df_read(): statbuf.f_blocks: 0| spec_device: /dev/devpts| dir:
/dev/pts| type: devpts
df.c:191:df_read(): statbuf.f_blocks: 101086| spec_device: /dev/sda1|
dir: /boot| type: ext3
df.c:215:df_read(): name: boot| spec_device: /dev/sda1| dir: /boot|
type: ext3
df.c:191:df_read(): statbuf.f_blocks: 259412| spec_device: /dev/shm|
dir: /dev/shm| type: tmpfs
df.c:215:df_read(): name: dev-shm| spec_device: /dev/shm| dir: /dev/shm|
type: tmpfs
df.c:229:df_read(): ignorelist type: tmpfs
df.c:191:df_read(): statbuf.f_blocks: 3076952| spec_device: /dev/hda3|
dir: /home/samba/media| type: vfat
df.c:215:df_read(): name: home-samba-media| spec_device: /dev/hda3| dir:
/home/samba/media| type: vfat
df.c:191:df_read(): statbuf.f_blocks: 14325560| spec_device: /dev/hda4|
dir: /var/lib/cddbd| type: xfs
df.c:215:df_read(): name: var-lib-cddbd| spec_device: /dev/hda4| dir:
/var/lib/cddbd| type: xfs
df.c:191:df_read(): statbuf.f_blocks: 0| spec_device: none| dir:
/proc/sys/fs/binfmt_misc| type: binfmt_misc
df.c:191:df_read(): statbuf.f_blocks: 2440662| spec_device: /dev/sdb1|
dir: /media/TRANS| type: vfat
df.c:215:df_read(): name: media-TRANS| spec_device: /dev/sdb1| dir:
/media/TRANS| type: vfat
df.c:219:df_read(): ignorelist spec_device: /dev/sdb1
utils_mount.c:671:cu_mount_freelist(): (list = 0x0x9c54680)