[collectd] [Feature Request/Version 6.0]: Adding Labels to Collectd Metrics #3094
Manoj Srivastava
manoj.srivastava.1962 at gmail.com
Tue Jan 7 22:54:44 CET 2020
Progress report: I have managed to get the underlying data-structures
mostly changed, I am in the process of modifying the write plugins to deal
with having since source ,etric data with labels instead of the
multi-source value_list_t.
> git diffstat master
Makefile.am | 33 ++-
src/aggregation.c | 6 +-
src/amqp.c | 4 +-
src/barometer.c | 156 +++++++-----
src/check_uptime.c | 27 +--
src/csv.c | 2 +-
src/daemon/filter_chain.c | 30 +--
src/daemon/filter_chain.h | 12 +-
src/daemon/metrics_list_test.c | 500
+++++++++++++++++++++++++++++++++++++++
src/daemon/plugin.c | 802
+++++++++++++++++++++++++++++++++++++++++++++++++++-----------
src/daemon/plugin.h | 260
+++++++++++++++++++-
src/daemon/utils_cache.c | 674
+++++++++++++++++++++++++++++-----------------------
src/daemon/utils_cache.h | 79 ++++---
src/daemon/utils_cache_mock.c | 8 +-
src/match_value.c | 2 +-
src/mqtt.c | 2 +-
src/postgresql.c | 3 +-
src/target_notification.c | 2 +-
src/threshold.c | 12 +-
src/utils/cmds/getval.c | 21 +-
src/utils/cmds/putval.c | 2 +-
src/utils/common/common.c | 128 +++++++---
src/utils/common/common.h | 9 +-
src/utils/format_graphite/format_graphite.c | 2 +-
src/utils/format_json/format_json.c | 2 +-
src/utils/format_stackdriver/format_stackdriver.c | 8 +-
src/write_riemann_threshold.c | 4 +-
27 files changed, 2123 insertions(+), 667 deletions(-)
Manoj
On Wed, Dec 4, 2019 at 12:28 PM Manoj Srivastava <
manoj.srivastava.1962 at gmail.com> wrote:
> Added another commit along the way:
>
> [metric-label-support]: Modify write queue to use metric_t
>
> This commit adds an updated write queue data structure, and updates
> the enqueue and dequeue calling paths. Note that while this compiles
> and the tests pass, thisis not a functioning collectd by itself, the
> caching and networking code is a work in progress.
>
> https://github.com/srivasta/collectd/tree/metric-label-support
>
> Manoj
>
> On Mon, Dec 2, 2019 at 3:58 PM Manoj Srivastava <
> manoj.srivastava.1962 at gmail.com> wrote:
>
>> Added a commit with utility functions for cloning and freeing the new
>> metric, identity and metadata list head data structures that will be used
>> by the helper functions that enqueue and dequeue from the the future new
>> write queue. I have that mostly code complete, and am writing unit tests
>> for the new queue (well, mostly, since I have not yet written the caching
>> functionality, and the network code will need updates too).
>>
>> Manoj
>>
>> On Wed, Nov 27, 2019 at 2:21 AM Matthias Runge <mrunge at matthias-runge.de>
>> wrote:
>>
>>> Hi Manoj,
>>>
>>> thank you for the heads-up here. This is a great achievement.
>>>
>>> A tiny typo, your repo is this one
>>> https://github.com/srivasta/collectd/tree/metric-label-support
>>>
>>> Matthias
>>>
>>> On 25/11/2019 23:07, Manoj Srivastava wrote:
>>> > Hi,
>>> >
>>> > I have implemented a mechanism to convert from the current
>>> > values_list_t metric representation to the new metric_list_t
>>> > representation. This method can be called from write_queue_enque method
>>> > to convert metric data from unmigrated read plugins to provide for a
>>> > backward compatible transition path for read plugins. The next step
>>> will
>>> > be to update the storage mechanism for the write queue, and then to
>>> > update the caching and networking code.
>>> >
>>> > Comments appreciated.
>>> >
>>> > Manoj
>>> >
>>> >
>>> > Commit: https://github.com/srivasta/collectd/tree/metric-label-s
>>> > Design
>>> > doc:
>>> https://docs.google.com/document/d/173gGP3tUD3yfN2NNHxCv0BsKsacfDlSyoaQIn7MqLtQ/edit?usp=sharing
>>> >
>>> >
>>> > On Wed, Oct 9, 2019 at 10:08 AM Manoj Srivastava
>>> > <manoj.srivastava.1962 at gmail.com
>>> > <mailto:manoj.srivastava.1962 at gmail.com>> wrote:
>>> >
>>> > Hi,
>>> >
>>> > I have an initial design proposal on how to convert from a
>>> > value_list_t format to the new, single data source metrics_t format
>>> > for metrics
>>> > at
>>> https://docs.google.com/document/d/1B4drbdkPoXvMFrBx57cnKtDgeehQ1zkptq9Ozx7MnG8/edit?usp=sharing
>>> >
>>> > Comments appreciated.
>>> >
>>> > Manoj
>>> >
>>> > On Mon, Sep 30, 2019 at 12:03 PM Manoj Srivastava
>>> > <manoj.srivastava.1962 at gmail.com
>>> > <mailto:manoj.srivastava.1962 at gmail.com>> wrote:
>>> >
>>> > Hi,
>>> >
>>> > I have an initial stalking horse commit based on the 5.9
>>> > branch that implements the data structures in the design docs,
>>> > and has unit tests to the avl tree used in the structures. I
>>> > think we need some discussion on how to convert the current
>>> > value_list_t 5-tupples --> name + labels structure in the new
>>> > format. Also, how should we deal with metadata lists in the new
>>> > format?
>>> >
>>> > I'll try and put together the design thoughts in a document
>>> > and share that around, and perhaps a code snippet or two t go
>>> > along with it.
>>> >
>>> > Manoj
>>> >
>>> > New basic data structures and start of branch:
>>> > https://github.com/srivasta/collectd/tree/metric-label-support
>>> >
>>> > Adding Labels to Collectd Metrics
>>> >
>>> https://docs.google.com/document/d/173gGP3tUD3yfN2NNHxCv0BsKsacfDlSyoaQIn7MqLtQ/edit?usp=sharing
>>> > Compare data structures for metric labels (collectd)
>>> >
>>> https://docs.google.com/document/d/1RZ7YAO6lWcDDvevzxFGVDgv4UQpdgWoT7OCP9Vpb2CU/edit?usp=sharing
>>> > --
>>> > The Basis of the Universe Isn’t Matter or Energy—It’s Data
>>> > Manoj Srivastava <srivasta at golden-gryphon.com
>>> > <mailto:srivasta at golden-gryphon.com>>
>>> >
>>> >
>>> >
>>> > --
>>> > The Basis of the Universe Isn’t Matter or Energy—It’s Data
>>> > Manoj Srivastava <srivasta at golden-gryphon.com
>>> > <mailto:srivasta at golden-gryphon.com>>
>>> >
>>> >
>>> >
>>> > --
>>> > The Basis of the Universe Isn’t Matter or Energy—It’s Data
>>> > Manoj Srivastava <srivasta at golden-gryphon.com
>>> > <mailto:srivasta at golden-gryphon.com>>
>>> >
>>> > _______________________________________________
>>> > collectd mailing list
>>> > collectd at verplant.org
>>> > https://mailman.verplant.org/listinfo/collectd
>>> >
>>>
>>>
>>
>> --
>> The Basis of the Universe Isn’t Matter or Energy—It’s Data
>> Manoj Srivastava <srivasta at golden-gryphon.com>
>>
>
>
> --
> The Basis of the Universe Isn’t Matter or Energy—It’s Data
> Manoj Srivastava <srivasta at golden-gryphon.com>
>
--
The Basis of the Universe Isn’t Matter or Energy—It’s Data
Manoj Srivastava <srivasta at golden-gryphon.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.verplant.org/pipermail/collectd/attachments/20200107/b2f5034e/attachment-0001.html>
More information about the collectd
mailing list