[collectd] Collection station for interface performance data, storing the data in a MySQL db

[Florian Forster]

Hi Bart,

On Tue, Sep 22, 2009 at 11:41:29AM +0200, Bart van den Heuvel wrote:
> Can Collectd fetch data from lots of interfaces and then store it into
> a MySQL db?

no, currently collectd cannot store data *in* MySQL or any other
relational database system (RDBMS).

The problem is the databases: If you have only 100 new values per
second, this translates to 8,640,000 new values every day. This becomes
a problem for databases pretty quickly, so you need to be clever there
and partition the data.

There's a sample PostgreSQL script [0] by Bob Cotton, which uses the
“Star schema” [1] to partition the data. The idea is basically to have
one table to hold host/plugin/type and separate tables for the actual
values (those are the ones growing fast). For the value tables you
create one table for each day (or every 6 hours or whatever is
reasonable for your setup).

What keeps me from including Bob's SQL script together with a C based
plugin to write data into the PostgreSQL database is that the SQL script
currently needs a maintenance job to run once a day. I'd much prefer if
the SQL script created required tables as it needs them. Unfortunately,
my SQL-foo is not good enough to adapt that.

The matter has been on your wishlist [2] for a while now. If you (or
anyone else on the list) is interested to work into that direction,
please let me know :)

Regards,
—octo

[0] <http://github.com/bcotton/collectd_dbstore>
[1] <http://en.wikipedia.org/wiki/Star_schema>
[2] <http://collectd.org/wiki/index.php/Roadmap#Store_to_RDBMS>
-- 
Florian octo Forster
Hacker in training
GnuPG: 0x91523C3D
http://verplant.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mailman.verplant.org/pipermail/collectd/attachments/20090925/1204b0af/attachment.pgp