[collectd] configuration: is there a way to rename/reorganize metric?
Yves Mettier
ymettier at free.fr
Sun Jan 10 21:48:14 CET 2016
Hello,
You should play with the "regex" match and the "replace" or "set"
target.
Regards,
Yves
Le 2016-01-08 20:39, Dudlo a écrit :
> I have not received any response and I cannot see how I could force
> curl_json plugin to organize the metric/values differently.
>
> Is there a way in collectd to do a "global replace" where I change
> metric/value names?
>
> Given today's structure of data sent to Graphite:
>
> Metrics
> \---collectd
> \---serverA
> \---curl_json-serviceMetrics
> gauge-serverA-service1-requests
> gauge-serverA-service1-successes
> ... other 8 for service1
> gauge-serverA-service2-requests
> ... other 9 for service2
> ... and on for service3 - service10
>
> I'd like to end up with:
>
> Metrics
> \---collectd
> \---serverA
> \---curl_json-serviceMetrics
> \---service1
> gauge-serverA-service1-requests
> gauge-serverA-service1-successes
> ... other 8 for service1
> \---service2
> gauge-serverA-service2-requests
> ... other 9 for service2
> ... and on for service3 - service10
>
> In other words, I'd like to "inject" the service1 to service10 into the
> structure/hierarchy.
>
> I feel that collectd's filter mechanism could be used for this but I
> cannot see a clear way of doing it.
>
> Any suggestions?
>
> Thank you,
> dudlo
>
> ---------------------------- Original Message
> ----------------------------
> Subject: curl_json configuration question (hierarchy)
> From: "Dudlo" <collectd at mozog.com>
> Date: Fri, December 4, 2015 12:34
> To: collectd at verplant.org
> --------------------------------------------------------------------------
>
> I am using the most recent curl_json plugin for collectd. It works but
> I'd
> like to configure it to generate more 'hierarchical' data.
>
> I have a server that returns some performance date in this
> (pseudo-code)
> JSON:
> {
> "serverA": {
> "service1": {
> "requests":"17",
> "successes":"15",
> "failures":"2",
> ... and 7 other values ...
> }
> "service2": {
> "requests":"28",
> ... and 9 other values ...
> }
> ...
> "service20" {
> ...
> }
> }
> }
>
> My collectd.conf looks like:
>
> LoadPlugin curl_json
>
> <Plugin curl_json>
> <URL "http://myserver/serviceMetrics">
> Instance "serviceMetrics"
> <Key "*/*/requests">
> Type "gauge"
> </Key>
> <Key "*/*/successes">
> Type "gauge"
> </Key>
> ... and some 8 other specs
> </URL>
> </Plugin>
>
> When pushed to Graphite, this generates a "flat" list of 200 values (20
> services, 10 values per service). The graphite/whisper data structure
> looks like:
>
> Metrics
> \---collectd
> \---serverA
> \---curl_json-serviceMetrics
> gauge-serverA-service1-requests
> gauge-serverA-service1-successes
> ... other 8 for service1
> gauge-serverA-service2-requests
> ... other 9 for service2
> ... and on for service3 - service10
>
> This is nice for a first pass and would work but it is hard to manage
> to
> choose from 200+ items on the flat list when graphing.
>
> I would really like a deeper hierarchy sliced by the service:
>
> Metrics
> \---collectd
> \---serverA
> \---curl_json-serviceMetrics
> +---service1
> | gauge-serverA-service1-requests
> | gauge-serverA-service1-successes
> | ... other 8 for service1
> +---service2
> | gauge-serverA-service2-requests
> | ... other 9 for service2
> ... and on for service3 - service10
>
> I do not know how to achieve it. I see:
>
> *** Option 1 - doable but unpractical. Change the collectd.conf to:
>
> <Plugin curl_json>
> <URL "http://myserver/serviceMetrics">
> Instance "serviceMetrics-service1"
> <Key "*/service1/requests">
> Type "gauge"
> </Key>
> <Key "*/service1/successes">
> Type "gauge"
> </Key>
> ... and some 8 other specs
> </URL>
> <URL "http://myserver/serviceMetrics">
> Instance "serviceMetrics-service2"
> <Key "*/service2/requests">
> Type "gauge"
> </Key>
> <Key "*/service2/successes">
> Type "gauge"
> </Key>
> ... and some 8 other specs
> </URL>
> ... and 8 more sections for service3 - service10
> </Plugin>
>
> There are two issues:
> 1. The services are dynamic. There may be 5 or 25 of them and there are
> no
> name patterns in there (unlike the pseud-code in my example here). So I
> would have to update collectd.conf with every service
> addition/deletion/name change.
> 2. It is expensive - I'd have to redundantly call the serviceMetrics
> web
> service 20 times (for each service), every time reading only one
> service
> section.
>
> *** Option 2 - impossible with curl_json plugin.
>
> Ideally, I'd like to "force" the hierarchy on the value using the
> Instance
> clause inside the <Key>, something like:
>
> <Plugin curl_json>
> <URL "http://myserver/serviceMetrics">
> Instance "serviceMetrics"
> <Key "*/*/requests">
> Type "gauge"
> Instance "*/requests"
> </Key>
> <Key "*/*/successes">
> Type "gauge"
> Instance "*/successes"
> </Key>
> ... and some 8 other specs
> </URL>
> </Plugin>
>
> The 'Instance "*/requests" ' clause does not work. It simply sets the
> name to the literal string (replacing special characters with
> underscores). In a way, I would like to use the concept of "capturing
> group" as in regex replacement. Sadly, curl_json is not ready for it.
>
> I even tried changing the web service definition to use multi-value
> type,
> as in load type (in types.db, load type is defined as
> "shortterm:GAUGE:0:5000, midterm:GAUGE:0:5000, longterm:GAUGE:0:5000").
> Again, curl_json is not ready for such data structure - it expects a
> single numeric value.
>
> Is there a way I could coerce curl_json into generating values in a
> hierarchy?
>
> Thank you,
> dudlo
>
>
>
> _______________________________________________
> collectd mailing list
> collectd at verplant.org
> http://mailman.verplant.org/listinfo/collectd
More information about the collectd
mailing list