snmpget on multiple OID fails on Eaton UPS
We have around 22 Eaton UPS devices currently monitored on Observium and I have been investigating today why about 25% of them refuse to return a syslocation.
It seems that the devices just don't reliably return snmpget when multiple OID are queried at once, at least for these objects.
Specifically, when taking the following command from the poller debug and running it manually:
/usr/bin/snmpget -t '4' -r '2' -v3 -l 'authPriv' -n '' -x 'DES' -X 'removed' -a 'MD5' -A 'removed' -u 'observium' -Pu -OQUs -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'device':'161' sysUpTime.0 sysLocation.0 sysContact.0 sysName.0
The request times out even when increasing the timeout.
When reducing the number of OID queried to 3 or less the device seems to respond:
/usr/bin/snmpget -t '4' -r '2' -v3 -l 'authPriv' -n '' -x 'DES' -X 'removed' -a 'MD5' -A 'removed' -u 'observium' -Pu -OQUs -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'device':'161' sysUpTime.0 sysLocation.0 sysContact.0 sysUpTime.0 = 0:0:31:01.19 sysLocation.0 = location sysContact.0 = user@example.com
/usr/bin/snmpget -t '4' -r '2' -v3 -l 'authPriv' -n '' -x 'DES' -X 'removed' -a 'MD5' -A 'removed' -u 'observium' -Pu -OQUs -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'device':'161' sysLocation.0 sysContact.0 sysName.0 sysLocation.0 = location sysContact.0 = user@example.com sysName.0 = device
Is there a way to configure the behaviour to reduce the number of simultaneous OID requested to make this work reliably?
Thanks, Tim C
Lol, excellent behaviour.
There isn't any way of configuring this at the moment, since being able to snmpget more than one thing is something you should reasonably be able to expect any SNMP stack to handle.
Is this behaviour also apparent when using v2c?
Adam.
Sent from BlueMail
On 12 Sep 2018, 17:17, at 17:17, Mailing Lists lists@coop3r.com wrote:
We have around 22 Eaton UPS devices currently monitored on Observium and I have been investigating today why about 25% of them refuse to return a syslocation.
It seems that the devices just don't reliably return snmpget when multiple OID are queried at once, at least for these objects.
Specifically, when taking the following command from the poller debug and running it manually:
/usr/bin/snmpget -t '4' -r '2' -v3 -l 'authPriv' -n '' -x 'DES' -X 'removed' -a 'MD5' -A 'removed' -u 'observium' -Pu -OQUs -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'device':'161' sysUpTime.0 sysLocation.0 sysContact.0 sysName.0
The request times out even when increasing the timeout.
When reducing the number of OID queried to 3 or less the device seems to respond:
/usr/bin/snmpget -t '4' -r '2' -v3 -l 'authPriv' -n '' -x 'DES' -X 'removed' -a 'MD5' -A 'removed' -u 'observium' -Pu -OQUs -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'device':'161' sysUpTime.0 sysLocation.0 sysContact.0 sysUpTime.0 = 0:0:31:01.19 sysLocation.0 = location sysContact.0 = user@example.com
/usr/bin/snmpget -t '4' -r '2' -v3 -l 'authPriv' -n '' -x 'DES' -X 'removed' -a 'MD5' -A 'removed' -u 'observium' -Pu -OQUs -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'device':'161' sysLocation.0 sysContact.0 sysName.0 sysLocation.0 = location sysContact.0 = user@example.com sysName.0 = device
Is there a way to configure the behaviour to reduce the number of simultaneous OID requested to make this work reliably?
Thanks, Tim C
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
We have ability set limit in os definition. But I think limit to 3 Oids in one request will reduce polling time..
Tim, you sure that this device not have fixed (new) firmware?
Adam Armstrong wrote on 12/09/2018 19:34:
Lol, excellent behaviour.
There isn't any way of configuring this at the moment, since being able to snmpget more than one thing is something you should reasonably be able to expect any SNMP stack to handle.
Is this behaviour also apparent when using v2c?
Adam.
Sent from BlueMail http://www.bluemail.me/r?b=13569 On 12 Sep 2018, at 17:17, Mailing Lists <lists@coop3r.com mailto:lists@coop3r.com> wrote:
We have around 22 Eaton UPS devices currently monitored on Observium and I have been investigating today why about 25% of them refuse to return a syslocation. It seems that the devices just don't reliably return snmpget when multiple OID are queried at once, at least for these objects. Specifically, when taking the following command from the poller debug and running it manually: /usr/bin/snmpget -t '4' -r '2' -v3 -l 'authPriv' -n '' -x 'DES' -X 'removed' -a 'MD5' -A 'removed' -u 'observium' -Pu -OQUs -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'device':'161' sysUpTime.0 sysLocation.0 sysContact.0 sysName.0 The request times out even when increasing the timeout. When reducing the number of OID queried to 3 or less the device seems to respond: /usr/bin/snmpget -t '4' -r '2' -v3 -l 'authPriv' -n '' -x 'DES' -X 'removed' -a 'MD5' -A 'removed' -u 'observium' -Pu -OQUs -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'device':'161' sysUpTime.0 sysLocation.0 sysContact.0 sysUpTime.0 = 0:0:31:01.19 sysLocation.0 = location sysContact.0 = user@example.com <mailto:user@example.com> /usr/bin/snmpget -t '4' -r '2' -v3 -l 'authPriv' -n '' -x 'DES' -X 'removed' -a 'MD5' -A 'removed' -u 'observium' -Pu -OQUs -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'device':'161' sysLocation.0 sysContact.0 sysName.0 sysLocation.0 = location sysContact.0 = user@example.com <mailto:user@example.com> sysName.0 = device Is there a way to configure the behaviour to reduce the number of simultaneous OID requested to make this work reliably? Thanks, Tim C ------------------------------------------------------------------------ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hello,
As you have a lot of Eaton UPS, do you also have a restart of the SNMP service every 6 hours ?
Best Regards
Thibault
De : observium observium-bounces@observium.org De la part de Mike Stupalov Envoyé : jeudi 13 septembre 2018 10:40 À : Observium observium@observium.org Objet : Re: [Observium] snmpget on multiple OID fails on Eaton UPS
We have ability set limit in os definition. But I think limit to 3 Oids in one request will reduce polling time..
Tim, you sure that this device not have fixed (new) firmware?
Adam Armstrong wrote on 12/09/2018 19:34:
Lol, excellent behaviour.
There isn't any way of configuring this at the moment, since being able to snmpget more than one thing is something you should reasonably be able to expect any SNMP stack to handle.
Is this behaviour also apparent when using v2c?
Adam.
Sent from BlueMail http://www.bluemail.me/r?b=13569
On 12 Sep 2018, at 17:17, Mailing Lists <lists@coop3r.com mailto:lists@coop3r.com > wrote:
We have around 22 Eaton UPS devices currently monitored on Observium and I have been investigating today why about 25% of them refuse to return a syslocation.
It seems that the devices just don't reliably return snmpget when multiple OID are queried at once, at least for these objects.
Specifically, when taking the following command from the poller debug and running it manually:
/usr/bin/snmpget -t '4' -r '2' -v3 -l 'authPriv' -n '' -x 'DES' -X 'removed' -a 'MD5' -A 'removed' -u 'observium' -Pu -OQUs -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'device':'161' sysUpTime.0 sysLocation.0 sysContact.0 sysName.0
The request times out even when increasing the timeout.
When reducing the number of OID queried to 3 or less the device seems to respond:
/usr/bin/snmpget -t '4' -r '2' -v3 -l 'authPriv' -n '' -x 'DES' -X 'removed' -a 'MD5' -A 'removed' -u 'observium' -Pu -OQUs -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'device':'161' sysUpTime.0 sysLocation.0 sysContact.0
sysUpTime.0 = 0:0:31:01.19
sysLocation.0 = location
sysContact.0 = user@example.com mailto:user@example.com
/usr/bin/snmpget -t '4' -r '2' -v3 -l 'authPriv' -n '' -x 'DES' -X 'removed' -a 'MD5' -A 'removed' -u 'observium' -Pu -OQUs -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'device':'161' sysLocation.0 sysContact.0 sysName.0
sysLocation.0 = location
sysContact.0 = user@example.com mailto:user@example.com
sysName.0 = device
Is there a way to configure the behaviour to reduce the number of simultaneous OID requested to make this work reliably?
Thanks,
Tim C
_____
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
On 13 September 2018 at 11:25, Thibault Richard thibs@thibs.com wrote:
Hello,
As you have a lot of Eaton UPS, do you also have a restart of the SNMP service every 6 hours ?
Best Regards
Thibault
Hi Thibault,
We don't see that on ours that I have noticed. Not yet anyway....
I'm not sure, how this device os detected.
Please attach debug for:
./poller.php -d -m os -h <device>
Mailing Lists wrote on 13/09/2018 14:29:
On 13 September 2018 at 11:25, Thibault Richard <thibs@thibs.com mailto:thibs@thibs.com> wrote:
Hello, As you have a lot of Eaton UPS, do you also have a restart of the SNMP service every 6 hours ? Best Regards ThibaultHi Thibault,
We don't see that on ours that I have noticed. Not yet anyway....
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hello,
Thanks for your reply. I’ve attached the debug
Best Regards
Thibault
De : observium observium-bounces@observium.org De la part de Mike Stupalov Envoyé : jeudi 13 septembre 2018 13:56 À : Observium observium@observium.org Objet : Re: [Observium] snmpget on multiple OID fails on Eaton UPS
I'm not sure, how this device os detected.
Please attach debug for:
./poller.php -d -m os -h <device>
Mailing Lists wrote on 13/09/2018 14:29:
On 13 September 2018 at 11:25, Thibault Richard <thibs@thibs.com mailto:thibs@thibs.com > wrote:
Hello,
As you have a lot of Eaton UPS, do you also have a restart of the SNMP service every 6 hours ?
Best Regards
Thibault
Hi Thibault,
We don't see that on ours that I have noticed. Not yet anyway....
_______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Oups sorry … a typo in my last command.
Attached the correct debug log (which include my SNMP community ;-( )
De : observium observium-bounces@observium.org De la part de Thibault Richard Envoyé : jeudi 13 septembre 2018 14:12 À : 'Observium' observium@observium.org Objet : Re: [Observium] snmpget on multiple OID fails on Eaton UPS
Hello,
Thanks for your reply. I’ve attached the debug
Best Regards
Thibault
De : observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > De la part de Mike Stupalov Envoyé : jeudi 13 septembre 2018 13:56 À : Observium <observium@observium.org mailto:observium@observium.org > Objet : Re: [Observium] snmpget on multiple OID fails on Eaton UPS
I'm not sure, how this device os detected.
Please attach debug for:
./poller.php -d -m os -h <device>
Mailing Lists wrote on 13/09/2018 14:29:
On 13 September 2018 at 11:25, Thibault Richard <thibs@thibs.com mailto:thibs@thibs.com > wrote:
Hello,
As you have a lot of Eaton UPS, do you also have a restart of the SNMP service every 6 hours ?
Best Regards
Thibault
Hi Thibault,
We don't see that on ours that I have noticed. Not yet anyway....
_______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Ok, max Oids limited to 3 in r9437 for this os. Please try with latest revision (pro, trunk).
If possible, attach full poller debug (after update): ./poller.php -d -h <device>
Thibault Richard wrote on 13/09/2018 15:20:
Oups sorry … a typo in my last command.
Attached the correct debug log (which include my SNMP community ;-( )
*De :*observium observium-bounces@observium.org *De la part de* Thibault Richard *Envoyé :* jeudi 13 septembre 2018 14:12 *À :* 'Observium' observium@observium.org *Objet :* Re: [Observium] snmpget on multiple OID fails on Eaton UPS
Hello,
Thanks for your reply. I’ve attached the debug
Best Regards
Thibault
*De :*observium <observium-bounces@observium.org mailto:observium-bounces@observium.org> *De la part de* Mike Stupalov *Envoyé :* jeudi 13 septembre 2018 13:56 *À :* Observium <observium@observium.org mailto:observium@observium.org> *Objet :* Re: [Observium] snmpget on multiple OID fails on Eaton UPS
I'm not sure, how this device os detected.
Please attach debug for:
./poller.php -d -m os -h <device>
Mailing Lists wrote on 13/09/2018 14:29:
On 13 September 2018 at 11:25, Thibault Richard <thibs@thibs.com <mailto:thibs@thibs.com>> wrote: Hello, As you have a lot of Eaton UPS, do you also have a restart of the SNMP service every 6 hours ? Best Regards Thibault Hi Thibault, We don't see that on ours that I have noticed. Not yet anyway.... _______________________________________________ observium mailing list observium@observium.org <mailto:observium@observium.org> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium-- Mike Stupalov Observium Limited, http://observium.org
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
On 13 September 2018 at 13:45, Mike Stupalov mike@observium.org wrote:
Ok, max Oids limited to 3 in r9437 for this os. Please try with latest revision (pro, trunk).
If possible, attach full poller debug (after update): ./poller.php -d -h <device>
Hi Mike,
I've sent the poller debug from r9437 on a failing device to you unicast off list.
Thanks, Tim C
Please try again with updated revision 9438.
Mailing Lists wrote on 13.09.18 16:44:
On 13 September 2018 at 13:45, Mike Stupalov <mike@observium.org mailto:mike@observium.org> wrote:
Ok, max Oids limited to 3 in r9437 for this os. Please try with latest revision (pro, trunk). If possible, attach full poller debug (after update): ./poller.php -d -h <device>Hi Mike,
I've sent the poller debug from r9437 on a failing device to you unicast off list.
Thanks, Tim C _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
On Fri, 14 Sep 2018 at 12:53, Mike Stupalov mike@observium.org wrote:
Please try again with updated revision 9438.
Hi Mike,
That did the trick, seems to have limited the query to 3 which now responds. Would you like another debug output to have a look at?
Thanks, Tim C
if issue fixed - not required :)
Mailing Lists wrote on 14.09.18 16:03:
On Fri, 14 Sep 2018 at 12:53, Mike Stupalov <mike@observium.org mailto:mike@observium.org> wrote:
Please try again with updated revision 9438.Hi Mike,
That did the trick, seems to have limited the query to 3 which now responds. Would you like another debug output to have a look at?
Thanks, Tim C _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
On Fri, 14 Sep 2018 at 14:52, Mike Stupalov mike@observium.org wrote:
if issue fixed - not required :)
Great, thanks!
Hello,
I ‘ve attached the new debug file
Best Regards
Thibault
De : Mike Stupalov mike@stupalov.ru De la part de Mike Stupalov Envoyé : jeudi 13 septembre 2018 14:45 À : Observium observium@observium.org; Thibault Richard thibs@thibs.com Objet : Re: [Observium] snmpget on multiple OID fails on Eaton UPS
Ok, max Oids limited to 3 in r9437 for this os. Please try with latest revision (pro, trunk).
If possible, attach full poller debug (after update): ./poller.php -d -h <device>
Thibault Richard wrote on 13/09/2018 15:20:
Oups sorry … a typo in my last command.
Attached the correct debug log (which include my SNMP community ;-( )
De : observium mailto:observium-bounces@observium.org observium-bounces@observium.org De la part de Thibault Richard Envoyé : jeudi 13 septembre 2018 14:12 À : 'Observium' mailto:observium@observium.org observium@observium.org Objet : Re: [Observium] snmpget on multiple OID fails on Eaton UPS
Hello,
Thanks for your reply. I’ve attached the debug
Best Regards
Thibault
De : observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > De la part de Mike Stupalov Envoyé : jeudi 13 septembre 2018 13:56 À : Observium <observium@observium.org mailto:observium@observium.org > Objet : Re: [Observium] snmpget on multiple OID fails on Eaton UPS
I'm not sure, how this device os detected.
Please attach debug for:
./poller.php -d -m os -h <device>
Mailing Lists wrote on 13/09/2018 14:29:
On 13 September 2018 at 11:25, Thibault Richard <thibs@thibs.com mailto:thibs@thibs.com > wrote:
Hello,
As you have a lot of Eaton UPS, do you also have a restart of the SNMP service every 6 hours ?
Best Regards
Thibault
Hi Thibault,
We don't see that on ours that I have noticed. Not yet anyway....
_______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
On 13 September 2018 at 09:40, Mike Stupalov mike@observium.org wrote:
We have ability set limit in os definition. But I think limit to 3 Oids in one request will reduce polling time..
Tim, you sure that this device not have fixed (new) firmware?
Hi Mike,
We are one release behind the latest firmware, in the latest firmware SNMP is even more broken than in this one, leaking memory into the ethernet port name returned in SNMP.
The multiple OID problem _seems_ to go away using SNMPv1 (because of course the device doesn't support v2c) so I am going to try that out on a few more and see if any other weirdness shows up.
I might actually try the latest firmware with v1 as well to see if that fixes the weird leaking memory thing, because I can't remember if I did that or not now....
Thanks, Tim C
participants (4)
-
Adam Armstrong -
Mailing Lists -
Mike Stupalov -
Thibault Richard