- observium - lists.observium.org

Re: [Observium] TP-Link switches freeze with new Observium CE
by Mike Stupalov 10 Sep '18

10 Sep '18

Hrm, on all my tested TP-Link switches this feature improved polling speed. That why separate_walk feature for this os enabled. But such problems are certainly unacceptable, I will disable that for os. Anyway, if you have not so many devices, goto device edit page -> Modules -> disable 'separate_walk' option. > Attila Nagy <mailto:tylla_at_memetic.org@tylla.hu> > 7 September 2018 at 19:31 > Hi List! > > Firstly our appreciations go to the team for the new CE version. > Thanks a lot! > > Now the somewhat bitter part: we updated yesterday evening to the > latest CE version and today morning we were confronted with the > problem that a few Tp-Link switches have frozen. To be more precise, > only the management part was frozen, so the switch was degraded to a > dumb non-manageable switch. After reboot the switches return to full > function only to freeze again on next SNMP poll. > > The switches in question are all T1600G-28TS or T1600G-28PS > (poor-man's manageable gigabit switches :-)) > There is no newer firmware update available for them. > They worked perfectly with the previous CE version up until the update. > > We have other Tp-Link switches which are not affected by the same > problem: TL-SG2424, TL-SG2216, TL-SG2008. > > > We tried to narrow down the problem, and it seems that discovery works > as it should, the freeze enacts during the polling, more exactly when > the ports module runs. Looking at the debug log the following command > produces the freeze: > /usr/bin/snmpget -v2c -c 'public' -Pu -OQUs -m IF-MIB -M > /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp > 'udp':'switch-test':'161' ifInMulticastPkts.1 ifOutMulticastPkts.1 > ifInBroadcastPkts.1 ifOutBroadcastPkts.1 ifHCInOctets.1 > ifHCOutOctets.1 ifHCInUcastPkts.1 ifHCOutUcastPkts.1 > ifHCInMulticastPkts.1 ifHCOutMulticastPkts.1 ifHCInBroadcastPkts.1 > ifHCOutBroadcastPkts.1 ifInOctets.1 ifOutOctets.1 ifInUcastPkts.1 > ifOutUcastPkts.1 ifInNUcastPkts.1 ifOutNUcastPkts.1 ifInDiscards.1 > ifOutDiscards.1 ifInErrors.1 ifOutErrors.1 ifInUnknownProtos.1 ifMtu.1 > ifSpeed.1 ifPhysAddress.1 ifAdminStatus.1 ifLastChange.1 > ifPromiscuousMode.1 ifConnectorPresent.1 > > From here onward there is no SNMP response from the switch, the SYS > LED stops blinking and is frozen in whichever state it was, and there > is no network connectivity to the switch (does not respond to ping and > the management interface is inaccessible). The dumb switch hardware > was still working for some time, but this was inconsistent, some > switches have frozen to a complete halt some time after, while others > were still working as dumb switches. As these are under constant load > we didn't had the time to more thoroughly investigate on them, we > restored their functionality and disabled them in Observium. > We have one extra hardware that we can put online for testing if needed. > > > My questions would be: > - Should I include a full debug log or the above info is enough? > - Has anyone met the same problem/has a solution to it? > - Can we know what was changed on the above command line, so maybe as > a workaround we can disable that single feature? > - I suspect this is a firmware issue, but I am not sure (that's why > I'm asking here first, here I have way bigger chances to receive > meaningful answers compared to Tp-Link's support), maybe somebody can > help me narrow down the firmware bug if it is really one. > > Thank you guys in advance. > > Best regards, > Tylla > > > > _______________________________________________ > observium mailing list > observium(a)observium.org > http://postman.memetic.org/cgi-bin/mailman/listinfo/observium -- Mike Stupalov https://stupalov.com -- Mike Stupalov Observium Limited, http://observium.org

4 4

Observium site blank after logging in
by Stallings, Breisch 10 Sep '18

10 Sep '18

I've acquired an Observium implementation, which worked perfectly fine a few months ago. Nothing that I know of has changed over the past few months. Now we can get to the login screen as usual. As soon as you enter your credentials and login, the very next screen is completely blank and nothing appears. We've rebooted the box several times. Today I upgraded to the latest release - 9431. I'm not very proficient working through Linux troubleshooting, which is was the base OS is. Has anyone else seen this issue and possibly how to troubleshoot/resolve it? Thank You. ****************************************************************************************** This message may contain confidential or proprietary information intended only for the use of the addressee(s) named above or may contain information that is legally privileged. If you are not the intended addressee, or the person responsible for delivering it to the intended addressee, you are hereby notified that reading, disseminating, distributing or copying this message is strictly prohibited. If you have received this message by mistake, please immediately notify us by replying to the message and delete the original message and any copies immediately thereafter. Thank you. ****************************************************************************************** CLLD

2 1

bng_active_sessions not graphed anymore
by Klimek, Denis 10 Sep '18

10 Sep '18

Hi, can someone help me to debug the graphing of "bng_active_sessions"? Which module is responsible etc pp? I noticed that Observium stopped graphing it for every Cisco ASR device for already 2 weeks. Based on our logfiles there was no update at that time. Mit freundlichem Gruß Stadtwerke Norderstedt Denis Klimek Professional Network Engineer IP-Systemtechnik Tel: 040 / 521 04 - 1049 Mobil: 0151 / 652 219 06 dklimek(a)stadtwerke-norderstedt.de<mailto:dklimek@stadtwerke-norderstedt.de> www.stadtwerke-norderstedt.de<http://www.stadtwerke-norderstedt.de/>

2 1

Add UPS-MIB to Mikrotik discovery
by Kaiser, Erich 09 Sep '18

09 Sep '18

Any chance you can add the UPS-MIB to the mikrotik discovery? A lot of guys are using the mikrotik to monitor UPS via USB/serial. Sincerely, Erich Kaiser The Fusion Network erich(a)gotfusion.net Office: 815-570-3101

2 3

Re: [Observium] Devices not being discovered
by Michael 08 Sep '18

08 Sep '18

The answer is in the CLI output when you tried adding the device there. ==== # ./add_device.php jr01 home v2c 161 udp Observium CE 18.9.9420 Add Device(s) Try to add jr01: Trying v2c community *** [0] ... No reply on given community *** using v2c. Could not reach jr01 with given SNMP parameters using v2c. Devices failed: 1. ==== Your device is not answering your SNMP queries. I'd start by checking the following: - wrong community string - wrong dns/hosts entry - snmpd on the target not listening/bound correctly for remote v2c queries - local firewall on target device Cheers, Michael > On 8 Sep 2018, at 1:17 pm, Michael <michael(a)smarsz.com> wrote: > > The answer is in the CLI output when you tried adding the device there. > > > ==== > # ./add_device.php jr01 home v2c 161 udp > Observium CE 18.9.9420 > Add Device(s) > > Try to add jr01: > Trying v2c community *** [0] ... > No reply on given community *** using v2c. > Could not reach jr01 with given SNMP parameters using v2c. > Devices failed: 1. > ==== > > > Your device is not answering your SNMP queries. I'd start by checking the following: > > - wrong community string > - wrong dns/hosts entry > - snmpd on the target not listening/bound correctly for remote v2c queries > - local firewall on target device > > > Cheers, > > Michael > > > > > > > > >> On 8 Sep 2018, at 10:42 am, Adam Armstrong <adama(a)memetic.org> wrote: >> >> No, I mean that poller run was pointless because how are you going to poll something which isn't even in observium? >> >> Adam. >> >> Sent from BlueMail >> On 8 Sep 2018, at 01:01, Roland Hill <roland.lists(a)hillnet.co.nz> wrote: >> On Fri, 07 Sep 2018 or thereabouts, Adam Armstrong came forth with: >> >> You do not have any devices in the database called "jr01". >> >> I deleted jr01 yesterday thinking that re-adding it might magically fix >> whatever the issue is. >> >> jr01 exists, can be founf on the network and has snmp running. >> >> ==== >> # fping jr01 >> jr01 is alive >> >> # systemctl status snmpd >> ● snmpd.service - Simple Network Management Protocol (SNMP) Daemon. >> Loaded: loaded (/lib/systemd/system/snmpd.service; enabled; vendor preset: enabled) >> Active: active (running) since Fri 2018-09-07 17:59:00 NZST; 17h ago >> Process: 1118 ExecStartPre=/bin/mkdir -p /var/run/agentx (code=exited, status=0/SUCCESS) >> Main PID: 1121 (snmpd) >> Tasks: 1 (limit: 4915) >> CGroup: /system.slice/snmpd.service >> └─1121 /usr/sbin/snmpd -Lsd -Lf /dev/null -u Debian-snmp -g Debian-snmp -I -smux mteTrigger mteTriggerConf -f >> >> >> # ps aux | grep snmpd >> Debian-+ 1121 0.0 0.1 65024 11364 ? Ss Sep07 0:44 /usr/sbin/snmpd -Lsd -Lf /dev/null -u Debian-snmp -g Debian-snmp -I -smux mteTrigger mteTriggerConf -f >> >> ==== >> >> Adding the device jr01 from the command line fails: >> >> ==== >> # ./add_device.php jr01 home v2c 161 udp >> Observium CE 18.9.9420 >> Add Device(s) >> >> Try to add jr01: >> Trying v2c community *** [0] ... >> No reply on given community *** using v2c. >> Could not reach jr01 with given SNMP parameters using v2c. >> Devices failed: 1. >> ==== >> >> I'm out of idea's but will go back to first principles and walk the >> install guide again just in case I have overlooked anything. jr01 has >> previously been discovered okay, but maybe I changed something and don't >> remember. >> >> I appreciate your help Adam. >> _______________________________________________ >> observium mailing list >> observium(a)observium.org >> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium >

3 3

Devices not being discovered
by Roland Hill 08 Sep '18

08 Sep '18

Hi all, I've been running the CE version of Observium for a few years now, always upgrading as the new releases come out. Earlier this year my server hosting Observium updated to Ubuntu 18.04 and php7x. As previously raised on list, this was incompatible with the previous CE version. No problem, I waited it out. Now my issue. Since upgrading to the latest CE version, 5 out of my 7 devices (home network) are no longer being discovered. I can confirm DNS is working. I've debugged as much as I can, including reinstalling snmpd on the Ubuntu boxes. The devices which are staying down are my Ubuntu boxes and Unifi AP and Secure gateway. Any direction on debugging would be greatly appreciated. -- Regards, Roland

2 11

TP-Link switches freeze with new Observium CE
by Attila Nagy 07 Sep '18

07 Sep '18

Hi List! Firstly our appreciations go to the team for the new CE version. Thanks a lot! Now the somewhat bitter part: we updated yesterday evening to the latest CE version and today morning we were confronted with the problem that a few Tp-Link switches have frozen. To be more precise, only the management part was frozen, so the switch was degraded to a dumb non-manageable switch. After reboot the switches return to full function only to freeze again on next SNMP poll. The switches in question are all T1600G-28TS or T1600G-28PS (poor-man's manageable gigabit switches :-)) There is no newer firmware update available for them. They worked perfectly with the previous CE version up until the update. We have other Tp-Link switches which are not affected by the same problem: TL-SG2424, TL-SG2216, TL-SG2008. We tried to narrow down the problem, and it seems that discovery works as it should, the freeze enacts during the polling, more exactly when the ports module runs. Looking at the debug log the following command produces the freeze: /usr/bin/snmpget -v2c -c 'public' -Pu -OQUs -m IF-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'switch-test':'161' ifInMulticastPkts.1 ifOutMulticastPkts.1 ifInBroadcastPkts.1 ifOutBroadcastPkts.1 ifHCInOctets.1 ifHCOutOctets.1 ifHCInUcastPkts.1 ifHCOutUcastPkts.1 ifHCInMulticastPkts.1 ifHCOutMulticastPkts.1 ifHCInBroadcastPkts.1 ifHCOutBroadcastPkts.1 ifInOctets.1 ifOutOctets.1 ifInUcastPkts.1 ifOutUcastPkts.1 ifInNUcastPkts.1 ifOutNUcastPkts.1 ifInDiscards.1 ifOutDiscards.1 ifInErrors.1 ifOutErrors.1 ifInUnknownProtos.1 ifMtu.1 ifSpeed.1 ifPhysAddress.1 ifAdminStatus.1 ifLastChange.1 ifPromiscuousMode.1 ifConnectorPresent.1 From here onward there is no SNMP response from the switch, the SYS LED stops blinking and is frozen in whichever state it was, and there is no network connectivity to the switch (does not respond to ping and the management interface is inaccessible). The dumb switch hardware was still working for some time, but this was inconsistent, some switches have frozen to a complete halt some time after, while others were still working as dumb switches. As these are under constant load we didn't had the time to more thoroughly investigate on them, we restored their functionality and disabled them in Observium. We have one extra hardware that we can put online for testing if needed. My questions would be: - Should I include a full debug log or the above info is enough? - Has anyone met the same problem/has a solution to it? - Can we know what was changed on the above command line, so maybe as a workaround we can disable that single feature? - I suspect this is a firmware issue, but I am not sure (that's why I'm asking here first, here I have way bigger chances to receive meaningful answers compared to Tp-Link's support), maybe somebody can help me narrow down the firmware bug if it is really one. Thank you guys in advance. Best regards, Tylla

2 1

Power Outage
by Ricardo Franco Cantero 06 Sep '18

06 Sep '18

Hi, What would be the best way to measure for a power outage? I see that there is a sensor for voltage. Would a Min Warn value trigger it on time to send a alert? Met vriendelijke groet, Ricardo Franco Cantero

3 2

Alert my customers
by Ricardo Franco Cantero 05 Sep '18

05 Sep '18

Hi, We monitor our clients through Observium. I wanted to know how I can add individual alert contacts so when a connection of a customer goes down, it sends an alert to us but also directly to the client who's affected. Met vriendelijke groet, Ricardo Franco Cantero

2 1

Association Ruleset
by Ricardo Franco Cantero 05 Sep '18

05 Sep '18

Hi, When I add an alert checker, now I need to add an Association Ruleset. It didn't used to ask for one and I can't seem to find the option to select all, like with the * How do I achieve this? Met vriendelijke groet, Ricardo Franco Cantero

2 1