[collectd] lua plugin

[Adrian Perez]

Hi!

I used Lua myself in some projects, and it is a nice language to be
used for extensions -- in fact one of its goals is to be easy to embed
and extend, so I think it is a natural to consider embedding it into
collectd :)

Lately I have not been using Lua a lot, but I keep reading the lua-l
mailing list and doing some tiny quick-hack scripts for different
needs.


On Tue, 23 Nov 2010 21:33:23 +0100
Schmurfy <schmurfy at gmail.com> wrote:

> On 23 November 2010 21:05, Florian Forster <octo at collectd.org> wrote:
> 
> > Hi Julien,
> >
> > On Mon, Nov 22, 2010 at 11:03:14PM +0100, Schmurfy wrote:
> > > this discussion started with octo as private messages on github
> > > but he wisely asked me to post it on the mailing list so anyone
> > > can take part in it so here I am !
> >
> > thanks for taking this here :) And sorry for not looking closer at
> > your branch yet. I certainly hope to make some time later this week.
> >
> > No problems :)
> 
> 
> > > […] the problem comes from the fact that the official lua
> > > distribution is a static library which cannot be embedded in the
> > > shared library built for collectd (I tried but failed miserably)
> >
> > The problem is that in order to be able to link static libraries
> > into shared objects (dynamic libraries), the static library has be
> > be compiled with some special linker settings (-fPIC). This is an
> > annoying problem, because many people, including many maintainers
> > of packages of libraries, don't know this and have a hard time
> > reproducing this bug, because it doesn't show up on a x86 system.
> >
> > About the two options:
> >
> > > The first one is to link to a shared library release of lua […]
> > > The second one is to include the lua sources themselves into the
> > > collectd plugin, […]
> >
> > [...]
> >
> > I'd go for a variant of the first option: Simply do a normal check
> > for the library using autoconf and link with libtool. This will
> > automatically get you the shared library if it is available and fall
> > back to the static library if necessary. Being the default behavior
> > this is what most people, including package maintainers, expect and
> > know how to deal with.

In general modern distributions already have Lua packaged as shared
libraries, so going for the check-and-use will be easier, and avoids
having additional third party code in the repo as Octo says. I am
quite sure that the rest of the Lua fellow developers would recommend
this.

Though there is a situation when actually including your own Lua
sources is not only widely accepted, but recommended: if you patch
Lua in some way (there are a number of patches to the Lua source
code that make it incompatible with the stock version [0], then it
is okay to include a copy of your patched Lua sources.

Usually it is recommended to use the stock implementation, unless you
have very specific needs which cannot be covered using the existing
third-party libraries.

> I will certainly need your help on this ;)

For sure people from the list could lend a hand :)

> > [...]
> >
> > In the worst case, the API provided to scripts depends on the
> > scripting language's version. If so, that's how it is. Period.
> >
> > [...]

For the particular case of Lua, language changes in general occur only
in major releases (e.g. from 4.x to 5.x), which also break the API.
New releases take several years to appear, so I think it will not be a
problem at all using the dynamic libraries prepackaged in distributions.

> > [...]
> >
> > If we can handle these complex and mainly "stand alone" languages
> > and they manage to maintain mechanisms to work with changes made to
> > them, I'm confident a language *intended to be integrated* will
> > manage to keep up, too.
> >
> As far as i am am concerned it was more a thought than a deeply
> anchored misconception ^^
> As long as it works I am fine with the painless method but my only
> experience on embedded language until now is with ruby 1.8 and while
> it can be used embedded it is clearly not designed at all for that
> and I ended up embedding the sources but that was not really a
> decision I wanted to make. I just wish to find a way that makes the
> installation painless for users.

As I said before, Lua is intended to be embedded in other applications,
and in my experience it is quite pleasant to work with. Even using just
the provided API (i.e. without using binding generators) feels simple
enough [1].
 
> > > I was also thinking about something else but linked to lua: have
> > > anyone thought about using a higher level language to configure
> > > filters ?
> >
> > That does sound like an interesting though. It'll be a lot of work
> > to implement though. Be sure to toss around some ideas on the list
> > before implementing anything.
>
> I won't even start really thinking about it before I have a working
> lua plugin and after that I need to really understand what happens
> behind these filters, so it is more of a long term idea.

I agree that having a simple functional Lua module and adding
additional features later looks like a good plan.

Doing filters in a simple yet full-featured language like Lua would
definitely be a killer feature. Some apps are already using Lua for
such task, for example the Wireshark network analyzer [2],
imapfilter [3], or syslogintr [4]. You may want to check how those
embed Lua to get some ideas.

Well, I was about to write “those are my two cents”, but in the end it
was a long-ish mail :D

Please feel free to ask doubts, and also you can ask in the lua-l
mailing list [5] -- people there is very kind and always try to help
newcomers.

Cheers,

---
[0] http://lua-users.org/wiki/LuaPowerPatches
[1] http://www.debian-administration.org/articles/264
[2] http://wiki.wireshark.org/Lua
[3] http://imapfilter.hellug.gr/
[4] http://www.conman.org/software/syslogintr/
[5] http://www.lua.org/lua-l.html
-- 
Adrian Perez <aperez at igalia.com>
Igalia - Free Software Engineering
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://mailman.verplant.org/pipermail/collectd/attachments/20101124/4f14cc13/attachment-0001.pgp>