- observium - lists.observium.org

Performance Issue - High CPU with 'mysqld'?
by Gordon Cheng (gocheng) 26 Oct '18

26 Oct '18

Hi all, I have a new Observium setup (verion 18.9.9428 (9th September 2018)) with the following VM spec: Cores 4 Memory 64GB HD 500GB CentOS 7 PHP 7.0.31 MySQL 5.5.60-MariaDB and with the following network devices to be monitored: Devices - 634 (total), 597 (up), 37 (down) Ports - 58,032 (total), 27,664 (up), 19,967 (down) Sensors - 47,639 (total), 43,243 (ok), 4,396 (down) Statuses - 5,880 (total), 5,854 (ok), 10 (alert) --- Issue is that we are frequently seeing the following error when connecting on GUI/CLI, with slow response time on GUI/CLI: DB Error 1040: Too many connections And we are seeing gaps (broken graphs) like the following: [cid:image001.png@01D46AC5.2F7A6900] We have tried to bump up the number of 'max_connections' from original 151 to 500 and then 200, but it didn't help: sjc-observium-1:/etc# grep max_conn my.cnf max_connections=200 sjc-observium-1:/etc# And it's observed that the CPU load is heavy with 'mysqld': top - 09:36:17 up 35 days, 18:54, 2 users, load average: 74.18, 89.49, 82.67 Tasks: 665 total, 79 running, 586 sleeping, 0 stopped, 0 zombie %Cpu(s): 62.9 us, 36.9 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.2 si, 0.0 st KiB Mem : 65810376 total, 2511836 free, 5989292 used, 57309248 buff/cache KiB Swap: 4193020 total, 4141016 free, 52004 used. 59172312 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5152 mysql 20 0 2749132 664216 9164 S 20.7 1.0 4058:17 mysqld 31879 root 20 0 419676 27564 8264 R 4.2 0.0 0:00.13 php 31769 root 20 0 417212 25532 8336 S 3.6 0.0 0:00.12 php 31933 root 20 0 416868 25116 8148 S 3.2 0.0 0:00.10 php 31935 root 20 0 416736 24700 7832 S 3.2 0.0 0:00.10 php <snip> My 'config.php' is pretty much with default values. --- I'm new to Observium. It'd be greatly appreciated if someone can shed some light how it can be resolved. Thanks. - Gordon

2 2

DB and Polling issues after upgrading to r9505
by Christopher Smith 25 Oct '18

25 Oct '18

We had many devices not set to the correct location due to https://jira.observium.org/browse/OBS-2767 Updated to r9505 and after doing so we reset the geo location via web interface (device properties -> Geolocation -> Reset GEO location) On any device that this was done to it seems like they do not exist to the poller any longer, none have been polled since the above action was taken. I based this info from the Per Device stats in the Pollerlog section (all of the devices that had their geo location reset were last polled 5+ hours ago, everything else that was not touched are being polled as expected every 5 minutes) I see these errors in the db.log file, they appear to have shown up when the Reset GEO location operation was done on each device: [2018/10/24 14:48:43 -0400] index.php(4115): Failed dbQuery (#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE `location_id` = ?' at line 1), Query: UPDATE `devices_locations` set WHERE `location_id` = ? [2018/10/24 14:49:44 -0400] index.php(9439): Failed dbQuery (#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE `location_id` = ?' at line 1), Query: UPDATE `devices_locations` set WHERE `location_id` = ? [2018/10/24 14:50:23 -0400] index.php(9439): Failed dbQuery (#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE `location_id` = ?' at line 1), Query: UPDATE `devices_locations` set WHERE `location_id` = ? [2018/10/24 14:50:57 -0400] index.php(44932): Failed dbQuery (#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE `location_id` = ?' at line 1), Query: UPDATE `devices_locations` set WHERE `location_id` = ? [2018/10/24 14:51:28 -0400] index.php(25685): Failed dbQuery (#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE `location_id` = ?' at line 1), Query: UPDATE `devices_locations` set WHERE `location_id` = ? [2018/10/24 14:52:29 -0400] index.php(25888): Failed dbQuery (#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE `location_id` = ?' at line 1), Query: UPDATE `devices_locations` set WHERE `location_id` = ? Observium system info: Observium 18.10.9505 (24th October 2018) OS Linux 3.10.0-862.3.3.el7.x86_64 [amd64] (CentOS 7) Apache 2.4.6 (CentOS) OpenSSL/1.0.2k-fips PHP/7.0.31 PHP 7.0.31 (OPcache: ENABLED) Python 2.7.5 MySQL 5.5.56-MariaDB (extension: mysqli 5.5.56-MariaDB) SNMP NET-SNMP 5.7.2 RRDtool 1.6.0 Fping 3.10 (IPv4 only)

2 2

Update - Port descripton changed ( possible bug or ... )
by Sigurður Hannesson 23 Oct '18

23 Oct '18

Hi, just updated Version Information Observium 18.10.9472 (16th October 2018) OS Linux 4.9.0-3-amd64 [amd64] (Debian 9.1) Apache 2.4.25 (Debian) PHP 7.0.19-1 (OPcache: ENABLED) Python 2.7.13 MySQL 10.1.23-MariaDB-9+deb9u1 (extension: mysqli 5.0.12-dev) SNMP NET-SNMP 5.7.3 RRDtool 1.6.0 Fping 3.15 (IPv4 and IPv6) . Noticed that most of my port groups had 0 ports in them. Looking at the port description they had changed deviceA 1/1/3<http://172.24.41.34/device/device=31/tab=port/port=5230/> \AKRA-3 No1\ instead of AKRA-3 No1 this of course means that all my port groups are not matching the criteria. br., SH

1 0

Redundancy Mode (C2900)
by Robert Williams 22 Oct '18

22 Oct '18

Hi, We've had a few customer devices generate alerts overnight for 'Operational Redundancy Mode' = nonRedundant Example below: It appears to be a newly detected element during a discovery run - but in all (of our) scenarios it is now in alarm as the customers do not have them operating in redundancy mode. As far as I can tell it's not configured at all on them. Obviously I can filter them out from the alert ("Customer - Status Alert") but it would be nice if I could hide them from the main pages (or stop it from being polled at all). Alternatively, does anyone know what element in our device config may be triggering this to show up, so that I can modify the configs to remove it? Thanks! Robert Williams Custodian Data Centres https://www.CustodianDC.com

3 3

Alert Checker for Printer Toner
by Walter Onda 19 Oct '18

19 Oct '18

Hello I spent a long time to make a toner alert when it arrive lower than 2 %, it doesn't matter which color. Is this condition correct? Because some of this printer has just 1% but it shown as "ok". Has maybe somebody an suggestion? Regards and have a nice weekend Walter

2 3

Discovering storage again - every time discovery is run
by Niklas Larsson 18 Oct '18

18 Oct '18

Hi, after r9479 (ish) we get this in the log every time discovery is running: Attached log from: ./discovery.php -d -h gluster02.example.com -m storage storage module is creating the logging. /niklas

3 10

Distinct alerting when several custom OID on the same device
by Thibault Richard 17 Oct '18

17 Oct '18

Hello, I've finally succeed to create alerting on custom OID Unfortunately, I'm now facing another issue I've one device with 2 custom OIDs * The device called "depanne" have 2 custom OID named "mailnotsent" and "checkdbconnection" I 've created one alert with this basic condition Currently if one of the 2 OID got a different value than 0, I receive the alert message "checkdbconnections fails" which correspond to the message I've defined It's OK when "checkdbconnections" fails but not when "mailnotsent' fails In other words, if the OID value for "mailnotsent" is not 0, I'm receiving a non-related message. I would like to create one alert related to one OID As checker condition for a custom OID I can only use a metric and the only available metric is the OID value. I know I cannot filter on the condition So I'm trying to filter thanks to the "association" In the previous version (which is still the documented one) it was possible to specify a device and an entity But as the current version no longer propose us a free field for this, I've tried to filter with several solution like this one It unfortunately doesn't work. How could I specify an alert is only linked to one OID ? Best Regards Thibault

2 7

MIB for FS.COM switches
by Pieter Meyer 16 Oct '18

16 Oct '18

Hi guys How can we add better support for switches from FS.COM Regards Pieter Meyer Converged Group | Licensed ISP 0258/IECNS/JAN/09 75 Witkoppen Rd, North Riding, Gauteng, 2162 Office: +27 10 591 4600 | Mobile: +27 83 233 6614 <https://goo.gl/maps/WHe3soT1fHm> map | <http://www.convergedgroup.net/> website | <http://www.convergedgroup.net/blog/> company blog | <http://www.linkedin.com/company/converged-telecoms> linkedin CAUTION: This email and any attachments may contain information that is confidential and subject to legal privilege. If you are not the intended recipient, you must not read, use, disseminate, distribute or copy this email or any attachments. If you have received this email in error, please notify us immediately and erase this email and any attachments. Thank you. DISCLAIMER: To the maximum extent permitted by law, Converged Group is not liable (including in respect of negligence) for viruses or other defects or for changes made to this email or to any attachments. Before opening or using attachments, check them for viruses and other defects. The information contained in this document is confidential to the addressee and is not the view nor the official policy of Converged Group unless otherwise stated.

2 1

Stormshield firewalls log
by Luca Sasdelli 16 Oct '18

16 Oct '18

Hello all, we've several Stormshield firewall (FreeBSD-based) to monitor with Observium; most of them send their syslog properly, but with three of them, the syslogs are correctly received on observium server (seen with tcpdump), but not displayed within the respective entries. The hostnames does match the ones used with the device names. The Syslog setup allows unknown host. The syslog messages filters doesn't match any possible string in the incoming logs on mentioned devices. By enabling syslog debug, the observium log writes the correspondent files successfully. What else should I check for this issue? Rev. 18.10.9470 Thanks! Luca

3 4

Citrix Netscaler 12.x
by Adam Armstrong 15 Oct '18

15 Oct '18

Hi Guys, Citrix seem to have broken SNMP in Netscaler 12.x. Best not to try those versions for now if you value your data collection. Adam. ⁣Sent from BlueMail

3 3