[collectd] Generic JMX plugin and JVM restart

[Bryan Varner]

The problem I’ve had in the past, is that it stops collecting metrics for MBeans published by an application when it’s redeployed (JVM doesn’t restart, but the application does).

I don’t think it’s an issue with GenericJMX, so to speak, since it re-queries for each bean every read cycle. It seems to be something with the way jboss (or maybe the mbeanserver) is handling things.

I had this issue when I was using GenericJMX with GlassFish 3.x and JBoss community 7 / EAP 6, both running on the Oracle JVM.

Regards,

-Bryan

On Sep 29, 2014, at 12:49 AM, Toni Moreno <toni.moreno at gmail.com> wrote:

> Hi to all.
> 
> I've been working with GenericJMX jvm plugin (current master version in git --collectd 5.4.1--) and I've tested with hotSpot jdk , jrockit from oracle and j9vm from IBM. 
> 
> And collectd -  GenericJMX - Always reconnects when JVM process restart.. no need to restart collectd.
> 
> Best Regards
> 
> 2014-09-25 18:40 GMT+02:00 Rahul Agarwal <RAgarwal at livegamer.com>:
> Thanks Bryan. So I tried it out. Initially it worked and I was collecting data! It is nice that there are minimal changes from GenericJMX in the configs.
> 
> But after a re-deploy the reconnection failed. Now I have even tried restarted collectd but Fast jmx still seems to be unable to connect. I am able to connect to jmx via jconsole so the server is certainly good.
> 
> Not sure if Fast jmx is remembering the bad state somewhere.
> 
>  
> 
>  
> 
> From: Bryan Varner [mailto:bryan at varnernet.com] 
> Sent: Tuesday, September 23, 2014 5:43 PM
> To: Rahul Agarwal
> Cc: collectd at verplant.org
> Subject: Re: [collectd] Generic JMX plugin and JVM restart
> 
>  
> 
>  
> 
> Nice, I can try it out. Does it re-connect on JVM restart?
> 
>  
> 
> Short answer: Yes.
> 
>  
> 
>  
> 
> Long answer:
> 
>  
> 
> JMX over RMI (at least the Oracle/Sun/OpenJDK implementation) has a heartbeat timer (default I think is like 60 seconds or 5 minutes, I don’t recall which) that pings the remote server.
> 
> If that times out (or the socket is explicitly closed) it marks the connection failed. The FastJMX plugin will detect that (it takes a while — several minutes is common — by the time the heartbeat fires and any timeout threshold is hit) and then initiates a reconnect attempt.
> 
>  
> 
> Reconnects occur with a multiplied back off, so you don’t just keep hammering a ‘dead’ server. I believe the upper limit on the back off is 5 minutes, but I could be mistaken. Eventually I’ll get around to making that configurable.
> 
>  
> 
> Best of luck! Let me know if you have any issues!
> 
>  
> 
> Regards,
> 
> -Bryan
> 
>  
> 
> 
> _______________________________________________
> collectd mailing list
> collectd at verplant.org
> http://mailman.verplant.org/listinfo/collectd
> 
> 
> 
> 
> -- 
> Att
> 
> Toni Moreno
> 
> 699706656
> 
> 
> Si no quieres perderte en el olvido tan pronto como estés muerto y corrompido,
> 
> escribe cosas dignas de leerse, o haz cosas dignas de escribirse.
> 
>  
> Benjamin Franklin 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.verplant.org/pipermail/collectd/attachments/20140929/cdffbca8/attachment.html>