<div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Tue, Jan 19, 2016 at 5:18 PM Pawel Akonom <<a href="mailto:pawel.akonom@vnomic.com" target="_blank">pawel.akonom@vnomic.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
I am trying to visualize some system metrics on grafana. Metrics are<br>
sent from CollectD to Logstash, then from Logstash to Elasticsearch and<br>
from Elasticsearch to Grafana. The problem is rx tx if_octets and<br>
if_packets metric is growing all the time. I would like to see<br>
differential metric on the graph - how many octets or packets was send<br>
and received in last 10 seconds.<br>
<br>
I am using collectd 5.5 on CentOS 6.7 and I have compiled it from source<br>
code. In /usr/share/collectd/types.db if_packets and if_octets are set<br>
to DERIVE. As I understand from wiki page<br>
<a href="https://collectd.org/wiki/index.php/Data_source" rel="noreferrer" target="_blank">https://collectd.org/wiki/index.php/Data_source</a> this data should be sent<br>
as a change from the last time:<br>
<br>
[root@hdptest-mster2 collectd-5.5.0]# grep if_packets<br>
/usr/share/collectd/types.db<br>
if_packets rx:DERIVE:0:U, tx:DERIVE:0:U<br>
[root@hdptest-mster2 collectd-5.5.0]# grep if_octets<br>
/usr/share/collectd/types.db<br>
if_octets rx:DERIVE:0:U, tx:DERIVE:0:U<br>
<br>
Here is my configuration file:<br>
<br>
[root@hdptest-mster2 collectd-5.5.0]# egrep -v ^#.* /etc/collectd.conf |<br>
sed '/^$/d'<br>
Hostname "ACI_184_vora2_Master_MT2"<br>
TypesDB "/usr/share/collectd/types.db"<br>
LoadPlugin cpu<br>
LoadPlugin df<br>
LoadPlugin disk<br>
LoadPlugin exec<br>
LoadPlugin interface<br>
LoadPlugin load<br>
LoadPlugin memory<br>
LoadPlugin network<br>
<Plugin cpu><br>
ReportByCpu true<br>
ReportByState true<br>
ValuesPercentage true<br>
</Plugin><br>
<Plugin df><br>
Device "/dev/sda1"<br>
Device "/dev/sdb1"<br>
FSType "ext4"<br>
ValuesPercentage true<br>
</Plugin><br>
<Plugin disk><br>
Disk "/^sd[a-b]?$/"<br>
IgnoreSelected false<br>
</Plugin><br>
<Plugin interface><br>
Interface "eth0"<br>
IgnoreSelected false<br>
</Plugin><br>
<Plugin memory><br>
ValuesAbsolute true<br>
ValuesPercentage false<br>
</Plugin><br>
<Plugin network><br>
<Server "192.168.112.228" "25826"><br>
</Server><br>
<Server "192.168.112.227" "25826"><br>
</Server><br>
</Plugin><br>
<br>
This is a sample of raw data sent to logstash from this host:<br>
<br>
[root@hdptest-elk logstash]# tail -f logstash-rawdata.log | grep MT2 |<br>
grep if_octets<br>
{"host":"ACI_184_vora2_Master_MT2","@timestamp":"2016-01-19T21:37:41.485Z","plugin":"interface","collectd_type":"if_octets","plugin_instance":"eth0","rx":4202999326,"tx":1272176172,"@version":"1","type":"collectd"}<br>
{"host":"ACI_184_vora2_Master_MT2","@timestamp":"2016-01-19T21:37:51.485Z","plugin":"interface","plugin_instance":"eth0","collectd_type":"if_octets","rx":4203048744,"tx":1272200296,"@version":"1","type":"collectd"}<br>
{"host":"ACI_184_vora2_Master_MT2","@timestamp":"2016-01-19T21:38:01.485Z","plugin":"interface","collectd_type":"if_octets","plugin_instance":"eth0","rx":4203131562,"tx":1272215389,"@version":"1","type":"collectd"}<br>
{"host":"ACI_184_vora2_Master_MT2","@timestamp":"2016-01-19T21:38:11.485Z","plugin":"interface","collectd_type":"if_octets","plugin_instance":"eth0","rx":4203152162,"tx":1272241466,"@version":"1","type":"collectd"}<br>
<br>
It looks like data are sent as a GAUGE value instead of DERIVE. What I<br>
am doing wrong? How can I configure CollectD to send differential<br>
statistics for the interfaces?<br>
<br>
Best Regards,<br>
Pawel<br>
<br>
<br></blockquote></div><div dir="ltr"><div class="gmail_quote"><div>What is your logstash configuration? I'm assuming that logstash is configured to listen on udp/25826 and you are using the network plugin as a write plugin to this port?<br><br></div><div>I don't have the answer to your question, but I have a similar setup and I see the same issue, where grafana renders the datapoints as a counter, rather than deltas between successive datapoints.<br><br></div><div>From what I understand about collectd, it will cache values in the global cache (<a href="https://collectd.org/wiki/index.php/Global_cache">https://collectd.org/wiki/index.php/Global_cache</a>) where DERIVE values are converted to a rate.<br><br></div><div>The interface plugins collects values from /proc/net/dev and sends them to the network plugin on the client as is. It is here, I believe, that the network plugin sends the values to the logstash listener, if that is how you have it setup, again with the values intact and not as a rate. This is why grafana renders them as such.<br><br></div><div>In a typical setup, many clients will use the network plugin to send plugin datapoints to a server instance running the network plugin in Listening mode. From there, an output plugin is configured to send to somewhere else, like write_graphite, for example. If you are familiar with graphite, the datapoints first go to carbon. If you run ngrep on the carbon port, you would see that the values for if_octets and if_packets are the rate values (I assume from the global cache) and not the raw DERIVE values that continue to grow.<br><br></div><div>It's almost like the network plugin, when configured to Listen, should have an option like StoreRates=True, like some of the other plugins, but I don't think the network plugin was intended for this.<br><br></div><div>Giovanni<br></div><div><br></div><br></div></div></div>