[collectd] New plugin: "Aggregation" creates aggregated metrics.

[Yves Mettier]

Hello,

This is a great news !

As the author of a similar plugin, basic_aggregator (see my Pull 
Request #136), I'm interested in yours.

Here are some of my needs and how I would use your aggregator. If this 
is correct, feel free to copy/paste to the documentation.
But I also have questions. Please read below.

Aggregation of cpu-total/cpu-idle for host1-host4
-------------------------------------------------
(in reality, there may be hundreds of hosts)

<Plugin "aggregation">
   <Aggregation>
     Host host1
     Host host2
     Host host3
     Host host4
     Plugin "cpu"
     PluginInstance "cpu-total"
     Type "cpu"
     TypeInstance "idle"

     GroupBy "Host"

     CalculateSum true
     CalculateAverage true
   </Aggregation>
</Plugin>

The aggregator types will be :

global/aggregation-cpu-cpu-total-sum/cpu-idle
global/aggregation-cpu-cpu-total-average/cpu-idle

Is that correct ?

Aggregation of each of interface-*/if_octets for host1-host4
------------------------------------------------------------

   <Aggregation>
     Host host1
     Host host2
     Host host3
     Host host4
     Plugin "interface"
     Type "if_octets"

     GroupBy "Host"
     GroupBy "PluginInstance"

     CalculateSum true
     CalculateAverage true
   </Aggregation>

The aggregator types will be :

global/aggregation-interface-sum/if_octets
global/aggregation-interface-average/if_octets

Is that correct ?

And what if I want to aggregate them by host only ? Do I have to do a 
script that get the list of all pluginInstance (collectdctl listval | 
grep interface | awk...) and generate the configuration every N units of 
time (like every hour or every day) ?


Aggregation of cpu-total/cpu-idle for host1-host4 and for host5-host8
---------------------------------------------------------------------

I would make 2 blocks :

<Plugin "aggregation">
   <Aggregation>
     Host host1
     Host host2
     Host host3
     Host host4
     Plugin "cpu"
     PluginInstance "cpu-total"
     Type "cpu"
     TypeInstance "idle"

     GroupBy "Host"

     CalculateSum true
     CalculateAverage true
   </Aggregation>
   <Aggregation>
     Host host5
     Host host6
     Host host7
     Host host8
     Plugin "cpu"
     PluginInstance "cpu-total"
     Type "cpu"
     TypeInstance "idle"

     GroupBy "Host"

     CalculateSum true
     CalculateAverage true
   </Aggregation>
</Plugin>

But what will be the aggregator types ? Both would result in this :

global/aggregation-cpu-cpu-total-sum/cpu-idle
global/aggregation-cpu-cpu-total-average/cpu-idle


Update the aggregator configuration
-----------------------------------
Now, host1 is replaced with host11. I will of course update my 
configuration file. How will collectd take this change into account ? Do 
I have to stop/start collectd ?

See the discussion I raised in with basic_aggregator, first in the 
comments of Pull Request #136 and then in the mailing list.

Will plugin_register_reconfig() (or similar) be implemented soon ?




About basic_aggregator : all these issues are solved. But I don't think 
it's a goot idea to maintain 2 similar plugins. I don't mind if your 
"wins" and mine remains as a proof of concept.

Regards,
Yves


Le 2012-11-11 09:23, Florian Forster a écrit :
> Hi everybody,
>
> the new *Aggregation plugin* has just been merged to the _master_
> branch. With this new plugin it is possible to aggregate multiple
> matching values into one using aggregation functions such as sum and
> average. This allows to you create an aggregated view of the sum of
> "cpu" metrics on each host, for example.
>
> Feedback and bug reports are, of course, very welcome!
>
> Here are some reference links:
>
>   * An overview of the plugin in the wiki:
>     <https://collectd.org/wiki/index.php/Plugin:Aggregation>
>   * Wiki page with example configs (quite empty now, add yours!)
>     <https://collectd.org/wiki/index.php/Plugin:Aggregation/Config>
>   * Manpage entry:
>     <http://octo.cx/mancollectdaggregation>
>
> Best regards,
> —octo

-- 
- Homepage       - http://ymettier.free.fr                             
-
- GPG key        - http://ymettier.free.fr/gpg.txt                     
-
- C en action    - http://ymettier.free.fr/livres/C_en_action_ed2.html 
-
- Guide Survie C - http://www.pearson.fr/livre/?GCOI=27440100673730    
-