Arch -> extension (since 16.04 upgrade)
Hi,
Just completed a release upgrade on Observium itself (14.04 to 16.04) and now it is reporting its own architecture as 'extension'.
Wouldn't be an issue really, but every single time it polls it updates the event log like this:
I've attached a debug output of the poller as this may help to track it down.
Interestingly, a lot of our other servers run 16.04 now but it only reports this for 'itself' - not the others.
Anyone have any ideas? Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com http://www.CustodianDC.com
Run poller with -m system -d for this host and see what it says. Its probably parsing an error into that field!
Adam.
Sent from BlueMail
On 7 Jul 2016, 17:05, at 17:05, Robert Williams Robert@CustodianDC.com wrote:
Hi,
Just completed a release upgrade on Observium itself (14.04 to 16.04) and now it is reporting its own architecture as 'extension'.
Wouldn't be an issue really, but every single time it polls it updates the event log like this:
I've attached a debug output of the poller as this may help to track it down.
Interestingly, a lot of our other servers run 16.04 now but it only reports this for 'itself' - not the others.
Anyone have any ideas? Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com http://www.CustodianDC.com
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hi – thanks, here you go :)
Only place I can find the word extension is here?
[0;36m o [1;37mMySQL [0m 5.7.12-0ubuntu1.1 (extension: mysqli 5.0.12-dev)
Interestingly, when I run the debug of the one module by itself, the result for “Arch” is null.
Yet when I run all the modules together (copy attached to my first email) it returns “extension” instead?!
So I can produce a different behaviour depending on whether I debug just the system module, or all the modules. A normal polling run obviously produces the “extension” result, I assume because it runs all the modules.
My guess would be that there is a variable which is being left from one of the modules which is leaking into the system module?
Here is the section of the full debug log, which returns “extension”:
including: includes/polling/os.inc.php [0;36m##### [1;37mModule Start: [1;31mos[0;36m #####[0m [0m
SQL[[0;33mSELECT * FROM `observium_attribs`[0m][0m SQL RUNTIME[[0;32m0.00038004s[0m][0m
SQL[[0;33mSELECT * FROM `entPhysical` WHERE `device_id` = '76' AND `entPhysicalContainedIn` = '0'[0m][0m SQL RUNTIME[[0;32m0.00045204s[0m][0m [0;36m o [1;37mOS Poller [0m Group
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m MIB-Dell-10892 -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/dell 'udp':'localhost':'161' chassisModelName.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.05s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'MIB-Dell-10892' AND `oid` = 'chassisModelName.1';[0m][0m SQL RUNTIME[[0;32m0.00053406s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1352'[0m][0m SQL RUNTIME[[0;32m0.00449586s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m CPQSINFO-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/hp 'udp':'localhost':'161' cpqSiProductName.0[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0432s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'CPQSINFO-MIB' AND `oid` = 'cpqSiProductName.0';[0m][0m SQL RUNTIME[[0;32m0.00036812s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1355'[0m][0m SQL RUNTIME[[0;32m0.00263810s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0379s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111';[0m][0m SQL RUNTIME[[0;32m0.00040197s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1356'[0m][0m SQL RUNTIME[[0;32m0.00282502s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.101.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0371s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.101.1';[0m][0m SQL RUNTIME[[0;32m0.00040507s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1358'[0m][0m SQL RUNTIME[[0;32m0.00268698s[0m][0m [0;36m o [1;37mHardware [0m Generic x86 [64bit] [0;36m o [1;37mVersion [0m 4.4.0-28-generic [0;36m o [1;37mFeatures [0m [0;34m<empty>[0m [0;36m o [1;37mSerial [0m [0;34m<empty>[0m [0;36m o [1;37mAsset [0m [0;34m<empty>[0m
SQL[[0;33mINSERT INTO `eventlog` (`device_id`,`entity_id`,`entity_type`,`timestamp`,`severity`,`message`) VALUES ('76','76','device',NOW(),'6','Arch -> extension')[0m][0m SQL RUNTIME[[0;32m0.00246716s[0m][0m [0;36m o [1;37mModule time [0m 0.1949s
Weird…?
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com http://www.CustodianDC.com From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: 07 July 2016 17:52 To: Observium Network Observation System observium@observium.org Cc: Observium Network Observation System observium@observium.org Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
Run poller with -m system -d for this host and see what it says. Its probably parsing an error into that field!
Adam.
Sent from BlueMailhttp://www.bluemail.me/r On 7 Jul 2016, at 17:05, Robert Williams <Robert@CustodianDC.commailto:Robert@CustodianDC.com> wrote: Hi,
Just completed a release upgrade on Observium itself (14.04 to 16.04) and now it is reporting its own architecture as ‘extension’.
Wouldn’t be an issue really, but every single time it polls it updates the event log like this:
[cid:1B5D0A9E55561A4E985F21695D4E5946@cdc.local]
I’ve attached a debug output of the poller as this may help to track it down.
Interestingly, a lot of our other servers run 16.04 now but it only reports this for ‘itself’ – not the others.
Anyone have any ideas? Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.commailto:Robert@CustodianDC.com http://www.CustodianDC.com
________________________________
observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
What is the Unix agent returning on this host?
(Did you install distro and/or os scripts?)
Adam.
Sent from BlueMail
On 8 Jul 2016, 09:51, at 09:51, Robert Williams Robert@CustodianDC.com wrote:
Hi – thanks, here you go :)
Only place I can find the word extension is here?
[0;36m o [1;37mMySQL [0m 5.7.12-0ubuntu1.1 (extension: mysqli 5.0.12-dev)
Interestingly, when I run the debug of the one module by itself, the result for “Arch” is null.
Yet when I run all the modules together (copy attached to my first email) it returns “extension” instead?!
So I can produce a different behaviour depending on whether I debug just the system module, or all the modules. A normal polling run obviously produces the “extension” result, I assume because it runs all the modules.
My guess would be that there is a variable which is being left from one of the modules which is leaking into the system module?
Here is the section of the full debug log, which returns “extension”:
including: includes/polling/os.inc.php [0;36m##### [1;37mModule Start: [1;31mos[0;36m #####[0m [0m
SQL[[0;33mSELECT * FROM `observium_attribs`[0m][0m SQL RUNTIME[[0;32m0.00038004s[0m][0m
SQL[[0;33mSELECT * FROM `entPhysical` WHERE `device_id` = '76' AND `entPhysicalContainedIn` = '0'[0m][0m SQL RUNTIME[[0;32m0.00045204s[0m][0m [0;36m o [1;37mOS Poller [0m Group
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m MIB-Dell-10892 -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/dell 'udp':'localhost':'161' chassisModelName.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.05s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'MIB-Dell-10892' AND `oid` = 'chassisModelName.1';[0m][0m SQL RUNTIME[[0;32m0.00053406s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1352'[0m][0m SQL RUNTIME[[0;32m0.00449586s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m CPQSINFO-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/hp 'udp':'localhost':'161' cpqSiProductName.0[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0432s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'CPQSINFO-MIB' AND `oid` = 'cpqSiProductName.0';[0m][0m SQL RUNTIME[[0;32m0.00036812s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1355'[0m][0m SQL RUNTIME[[0;32m0.00263810s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0379s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111';[0m][0m SQL RUNTIME[[0;32m0.00040197s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1356'[0m][0m SQL RUNTIME[[0;32m0.00282502s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.101.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0371s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.101.1';[0m][0m SQL RUNTIME[[0;32m0.00040507s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1358'[0m][0m SQL RUNTIME[[0;32m0.00268698s[0m][0m [0;36m o [1;37mHardware [0m Generic x86 [64bit] [0;36m o [1;37mVersion [0m 4.4.0-28-generic [0;36m o [1;37mFeatures [0m [0;34m<empty>[0m [0;36m o [1;37mSerial [0m [0;34m<empty>[0m [0;36m o [1;37mAsset [0m [0;34m<empty>[0m
SQL[[0;33mINSERT INTO `eventlog` (`device_id`,`entity_id`,`entity_type`,`timestamp`,`severity`,`message`) VALUES ('76','76','device',NOW(),'6','Arch -> extension')[0m][0m SQL RUNTIME[[0;32m0.00246716s[0m][0m [0;36m o [1;37mModule time [0m 0.1949s
Weird…?
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com http://www.CustodianDC.com From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: 07 July 2016 17:52 To: Observium Network Observation System observium@observium.org Cc: Observium Network Observation System observium@observium.org Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
Run poller with -m system -d for this host and see what it says. Its probably parsing an error into that field!
Adam.
Sent from BlueMailhttp://www.bluemail.me/r On 7 Jul 2016, at 17:05, Robert Williams <Robert@CustodianDC.commailto:Robert@CustodianDC.com> wrote: Hi,
Just completed a release upgrade on Observium itself (14.04 to 16.04) and now it is reporting its own architecture as ‘extension’.
Wouldn’t be an issue really, but every single time it polls it updates the event log like this:
[cid:1B5D0A9E55561A4E985F21695D4E5946@cdc.local]
I’ve attached a debug output of the poller as this may help to track it down.
Interestingly, a lot of our other servers run 16.04 now but it only reports this for ‘itself’ – not the others.
Anyone have any ideas? Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.commailto:Robert@CustodianDC.com http://www.CustodianDC.com
observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Sounds like both distro and mysql agent scripts are installed, and the error output from mysql ("The MySQL extension is not loaded") is spilling over into the <<distro>> output ...
Tom
On 08/07/2016 11:53, Adam Armstrong wrote:
What is the Unix agent returning on this host?
(Did you install distro and/or os scripts?)
Adam.
Sent from BlueMail http://www.bluemail.me/r
On 8 Jul 2016, at 09:51, Robert Williams <Robert@CustodianDC.com mailto:Robert@CustodianDC.com> wrote:
Hi – thanks, here you go :) Only place I can find the word extension is here? [0;36m o [1;37mMySQL [0m 5.7.12-0ubuntu1.1 (extension: mysqli 5.0.12-dev) Interestingly, when I run the debug of the one module _by itself_, the result for “Arch” is *null*. Yet when I run all the modules together (copy attached to my first email) it returns “*extension*” instead?! So I can produce a different behaviour depending on whether I debug just the system module, or all the modules. A normal polling run obviously produces the “extension” result, I assume because it runs all the modules. My guess would be that there is a variable which is being left from one of the modules which is leaking into the system module? Here is the section of the full debug log, which returns “extension”: including: includes/polling/os.inc.php [0;36m##### [1;37mModule Start: [1;31mos[0;36m #####[0m [0m SQL[[0;33mSELECT * FROM `observium_attribs`[0m][0m SQL RUNTIME[[0;32m0.00038004s[0m][0m SQL[[0;33mSELECT * FROM `entPhysical` WHERE `device_id` = '76' AND `entPhysicalContainedIn` = '0'[0m][0m SQL RUNTIME[[0;32m0.00045204s[0m][0m [0;36m o [1;37mOS Poller [0m Group CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m MIB-Dell-10892 -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/dell 'udp':'localhost':'161' chassisModelName.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.05s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'MIB-Dell-10892' AND `oid` = 'chassisModelName.1';[0m][0m SQL RUNTIME[[0;32m0.00053406s[0m][0m SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1352'[0m][0m SQL RUNTIME[[0;32m0.00449586s[0m][0m CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m CPQSINFO-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/hp 'udp':'localhost':'161' cpqSiProductName.0[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0432s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'CPQSINFO-MIB' AND `oid` = 'cpqSiProductName.0';[0m][0m SQL RUNTIME[[0;32m0.00036812s[0m][0m SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1355'[0m][0m SQL RUNTIME[[0;32m0.00263810s[0m][0m CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0379s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111';[0m][0m SQL RUNTIME[[0;32m0.00040197s[0m][0m SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1356'[0m][0m SQL RUNTIME[[0;32m0.00282502s[0m][0m CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.101.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0371s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.101.1';[0m][0m SQL RUNTIME[[0;32m0.00040507s[0m][0m SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1358'[0m][0m SQL RUNTIME[[0;32m0.00268698s[0m][0m [0;36m o [1;37mHardware [0m Generic x86 [64bit] [0;36m o [1;37mVersion [0m 4.4.0-28-generic [0;36m o [1;37mFeatures [0m [0;34m<empty>[0m [0;36m o [1;37mSerial [0m [0;34m<empty>[0m [0;36m o [1;37mAsset [0m [0;34m<empty>[0m SQL[[0;33mINSERT INTO `eventlog` (`device_id`,`entity_id`,`entity_type`,`timestamp`,`severity`,`message`) VALUES ('76','76','device',NOW(),'6','Arch -> extension')[0m][0m SQL RUNTIME[[0;32m0.00246716s[0m][0m [0;36m o [1;37mModule time [0m 0.1949s Weird…? Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com http://www.CustodianDC.com *From:*observium [mailto:observium-bounces@observium.org] *On Behalf Of *Adam Armstrong *Sent:* 07 July 2016 17:52 *To:* Observium Network Observation System <observium@observium.org> *Cc:* Observium Network Observation System <observium@observium.org> *Subject:* Re: [Observium] Arch -> extension (since 16.04 upgrade) Run poller with -m system -d for this host and see what it says. Its probably parsing an error into that field! Adam. Sent from BlueMail <http://www.bluemail.me/r> On 7 Jul 2016, at 17:05, Robert Williams <Robert@CustodianDC.com <mailto:Robert@CustodianDC.com>> wrote: Hi, Just completed a release upgrade on Observium itself (14.04 to 16.04) and now it is reporting its own architecture as ‘extension’. Wouldn’t be an issue really, but every single time it polls it updates the event log like this: I’ve attached a debug output of the poller as this may help to track it down. Interestingly, a lot of our other servers run 16.04 now but it only reports this for ‘itself’ – not the others. Anyone have any ideas? Cheers! Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com <mailto:Robert@CustodianDC.com> http://www.CustodianDC.com ------------------------------------------------------------------------ observium mailing list observium@observium.org <mailto:observium@observium.org> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium ------------------------------------------------------------------------ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hi,
Thanks - I’ve attached the unix-agent debug now.
As for the scripts, it has the following installed locally (from many years ago!):
:/usr/lib/observium_agent/local# ls -lh total 68K -rwxr-xr-x 1 root root 2.9K Oct 10 2013 apache -rwxr-xr-x 1 root root 714 Oct 10 2013 dpkg -rwxr-xr-x 1 root root 51K Oct 10 2013 mysql -rwxr-xr-x 1 root root 747 Nov 19 2013 mysql.cnf -rwxr-xr-x 1 root root 3.2K Mar 22 2014 ntpd
From an entire poller run in debug, the only place I can find the word ‘extension’ is indeed in the mysql output, so a leak from there would make sense as suggested.
I’ve also checked the prerequisites for the MySQL agent and they are all there still after the upgrade – BUT – I can see from the graphs that it is now broken:
[cid:image002.jpg@01D1D91E.E163B7B0]
In the agent debug, line 1899 has “The MySQL extension is not loaded"” – but I can’t see why/why/how this is broken since the 16.04 upgrade.
I’m going to assume that if I fix the MySQL agent, then the arch issue will go away.
So, any ideas on the MySQL problem?
Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com http://www.CustodianDC.com From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: 08 July 2016 10:53 To: Observium Network Observation System observium@observium.org Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
What is the Unix agent returning on this host?
(Did you install distro and/or os scripts?)
Adam.
Sent from BlueMailhttp://www.bluemail.me/r On 8 Jul 2016, at 09:51, Robert Williams <Robert@CustodianDC.commailto:Robert@CustodianDC.com> wrote: Hi – thanks, here you go :)
Only place I can find the word extension is here?
[0;36m o [1;37mMySQL [0m 5.7.12-0ubuntu1.1 (extension: mysqli 5.0.12-dev)
Interestingly, when I run the debug of the one module by itself, the result for “Arch” is null.
Yet when I run all the modules together (copy attached to my first email) it returns “extension” instead?!
So I can produce a different behaviour depending on whether I debug just the system module, or all the modules. A normal polling run obviously produces the “extension” result, I assume because it runs all the modules.
My guess would be that there is a variable which is being left from one of the modules which is leaking into the system module?
Here is the section of the full debug log, which returns “extension”:
including: includes/polling/os.inc.php [0;36m##### [1;37mModule Start: [1;31mos[0;36m #####[0m [0m
SQL[[0;33mSELECT * FROM `observium_attribs`[0m][0m SQL RUNTIME[[0;32m0.00038004s[0m][0m
SQL[[0;33mSELECT * FROM `entPhysical` WHERE `device_id` = '76' AND `entPhysicalContainedIn` = '0'[0m][0m SQL RUNTIME[[0;32m0.00045204s[0m][0m [0;36m o [1;37mOS Poller [0m Group
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m MIB-Dell-10892 -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/dell 'udp':'localhost':'161' chassisModelName.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.05s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'MIB-Dell-10892' AND `oid` = 'chassisModelName.1';[0m][0m SQL RUNTIME[[0;32m0.00053406s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1352'[0m][0m SQL RUNTIME[[0;32m0.00449586s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m CPQSINFO-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/hp 'udp':'localhost':'161' cpqSiProductName.0[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0432s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'CPQSINFO-MIB' AND `oid` = 'cpqSiProductName.0';[0m][0m SQL RUNTIME[[0;32m0.00036812s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1355'[0m][0m SQL RUNTIME[[0;32m0.00263810s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0379s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111';[0m][0m SQL RUNTIME[[0;32m0.00040197s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1356'[0m][0m SQL RUNTIME[[0;32m0.00282502s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.101.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0371s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.101.1';[0m][0m SQL RUNTIME[[0;32m0.00040507s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1358'[0m][0m SQL RUNTIME[[0;32m0.00268698s[0m][0m [0;36m o [1;37mHardware [0m Generic x86 [64bit] [0;36m o [1;37mVersion [0m 4.4.0-28-generic [0;36m o [1;37mFeatures [0m [0;34m<empty>[0m [0;36m o [1;37mSerial [0m [0;34m<empty>[0m [0;36m o [1;37mAsset [0m [0;34m<empty>[0m
SQL[[0;33mINSERT INTO `eventlog` (`device_id`,`entity_id`,`entity_type`,`timestamp`,`severity`,`message`) VALUES ('76','76','device',NOW(),'6','Arch -> extension')[0m][0m SQL RUNTIME[[0;32m0.00246716s[0m][0m [0;36m o [1;37mModule time [0m 0.1949s
Weird…?
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.commailto:Robert@CustodianDC.com http://www.CustodianDC.com From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: 07 July 2016 17:52 To: Observium Network Observation System <observium@observium.orgmailto:observium@observium.org> Cc: Observium Network Observation System <observium@observium.orgmailto:observium@observium.org> Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
Run poller with -m system -d for this host and see what it says. Its probably parsing an error into that field!
Adam.
Sent from BlueMailhttp://www.bluemail.me/r On 7 Jul 2016, at 17:05, Robert Williams <Robert@CustodianDC.commailto:Robert@CustodianDC.com> wrote: Hi,
Just completed a release upgrade on Observium itself (14.04 to 16.04) and now it is reporting its own architecture as ‘extension’.
Wouldn’t be an issue really, but every single time it polls it updates the event log like this:
[cid:1B5D0A9E55561A4E985F21695D4E5946@cdc.local]
I’ve attached a debug output of the poller as this may help to track it down.
Interestingly, a lot of our other servers run 16.04 now but it only reports this for ‘itself’ – not the others.
Anyone have any ideas? Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.commailto:Robert@CustodianDC.com http://www.CustodianDC.com
________________________________
observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
________________________________
observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
What do you get if you run the script manually?
/usr/lib/observium_agent/local/mysql
On 8 Jul 2016, at 13:44, Robert Williams Robert@CustodianDC.com wrote:
Hi,
Thanks - I’ve attached the unix-agent debug now.
As for the scripts, it has the following installed locally (from many years ago!):
:/usr/lib/observium_agent/local# ls -lh total 68K -rwxr-xr-x 1 root root 2.9K Oct 10 2013 apache -rwxr-xr-x 1 root root 714 Oct 10 2013 dpkg -rwxr-xr-x 1 root root 51K Oct 10 2013 mysql -rwxr-xr-x 1 root root 747 Nov 19 2013 mysql.cnf -rwxr-xr-x 1 root root 3.2K Mar 22 2014 ntpd
From an entire poller run in debug, the only place I can find the word ‘extension’ is indeed in the mysql output, so a leak from there would make sense as suggested.
I’ve also checked the prerequisites for the MySQL agent and they are all there still after the upgrade – BUT – I can see from the graphs that it is now broken:
<image002.jpg>
In the agent debug, line 1899 has “The MySQL extension is not loaded"” – but I can’t see why/why/how this is broken since the 16.04 upgrade.
I’m going to assume that if I fix the MySQL agent, then the arch issue will go away.
So, any ideas on the MySQL problem?
Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com mailto:Robert@custodiandc.com http://www.CustodianDC.com http://www.custodiandc.com/ From: observium [mailto:observium-bounces@observium.org mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: 08 July 2016 10:53 To: Observium Network Observation System <observium@observium.org mailto:observium@observium.org> Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
What is the Unix agent returning on this host?
(Did you install distro and/or os scripts?)
Adam.
Sent from BlueMail http://www.bluemail.me/r On 8 Jul 2016, at 09:51, Robert Williams <Robert@CustodianDC.com mailto:Robert@CustodianDC.com> wrote: Hi – thanks, here you go :)
Only place I can find the word extension is here?
[0;36m o [1;37mMySQL [0m 5.7.12-0ubuntu1.1 (extension: mysqli 5.0.12-dev)
Interestingly, when I run the debug of the one module by itself, the result for “Arch” is null.
Yet when I run all the modules together (copy attached to my first email) it returns “extension” instead?!
So I can produce a different behaviour depending on whether I debug just the system module, or all the modules. A normal polling run obviously produces the “extension” result, I assume because it runs all the modules.
My guess would be that there is a variable which is being left from one of the modules which is leaking into the system module?
Here is the section of the full debug log, which returns “extension”:
including: includes/polling/os.inc.php [0;36m##### [1;37mModule Start: [1;31mos[0;36m #####[0m [0m
SQL[[0;33mSELECT * FROM `observium_attribs`[0m][0m SQL RUNTIME[[0;32m0.00038004s[0m][0m
SQL[[0;33mSELECT * FROM `entPhysical` WHERE `device_id` = '76' AND `entPhysicalContainedIn` = '0'[0m][0m SQL RUNTIME[[0;32m0.00045204s[0m][0m [0;36m o [1;37mOS Poller [0m Group
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m MIB-Dell-10892 -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/dell 'udp':'localhost':'161' chassisModelName.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.05s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'MIB-Dell-10892' AND `oid` = 'chassisModelName.1';[0m][0m SQL RUNTIME[[0;32m0.00053406s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1352'[0m][0m SQL RUNTIME[[0;32m0.00449586s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m CPQSINFO-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/hp 'udp':'localhost':'161' cpqSiProductName.0[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0432s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'CPQSINFO-MIB' AND `oid` = 'cpqSiProductName.0';[0m][0m SQL RUNTIME[[0;32m0.00036812s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1355'[0m][0m SQL RUNTIME[[0;32m0.00263810s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0379s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111';[0m][0m SQL RUNTIME[[0;32m0.00040197s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1356'[0m][0m SQL RUNTIME[[0;32m0.00282502s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.101.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0371s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.101.1';[0m][0m SQL RUNTIME[[0;32m0.00040507s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1358'[0m][0m SQL RUNTIME[[0;32m0.00268698s[0m][0m [0;36m o [1;37mHardware [0m Generic x86 [64bit] [0;36m o [1;37mVersion [0m 4.4.0-28-generic [0;36m o [1;37mFeatures [0m [0;34m<empty>[0m [0;36m o [1;37mSerial [0m [0;34m<empty>[0m [0;36m o [1;37mAsset [0m [0;34m<empty>[0m
SQL[[0;33mINSERT INTO `eventlog` (`device_id`,`entity_id`,`entity_type`,`timestamp`,`severity`,`message`) VALUES ('76','76','device',NOW(),'6','Arch -> extension')[0m][0m SQL RUNTIME[[0;32m0.00246716s[0m][0m [0;36m o [1;37mModule time [0m 0.1949s
Weird…?
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com mailto:Robert@CustodianDC.com http://www.CustodianDC.com http://www.custodiandc.com/ From: observium [mailto:observium-bounces@observium.org mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: 07 July 2016 17:52 To: Observium Network Observation System <observium@observium.org mailto:observium@observium.org> Cc: Observium Network Observation System <observium@observium.org mailto:observium@observium.org> Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
Run poller with -m system -d for this host and see what it says. Its probably parsing an error into that field!
Adam.
Sent from BlueMail http://www.bluemail.me/r On 7 Jul 2016, at 17:05, Robert Williams <Robert@CustodianDC.com mailto:Robert@CustodianDC.com> wrote: Hi,
Just completed a release upgrade on Observium itself (14.04 to 16.04) and now it is reporting its own architecture as ‘extension’.
Wouldn’t be an issue really, but every single time it polls it updates the event log like this:
I’ve attached a debug output of the poller as this may help to track it down.
Interestingly, a lot of our other servers run 16.04 now but it only reports this for ‘itself’ – not the others.
Anyone have any ideas? Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com mailto:Robert@CustodianDC.com http://www.CustodianDC.com http://www.custodiandc.com/
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
<localhost-unix-agent.txt>_______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
The script may want the mysql module rather than the mysqli module.
Observium prefers mysqli, so you may not have the mysql extension.
Adam.
Sent from BlueMail
On 8 Jul 2016, 13:45, at 13:45, Robert Williams Robert@CustodianDC.com wrote:
Hi,
Thanks - I’ve attached the unix-agent debug now.
As for the scripts, it has the following installed locally (from many years ago!):
:/usr/lib/observium_agent/local# ls -lh total 68K -rwxr-xr-x 1 root root 2.9K Oct 10 2013 apache -rwxr-xr-x 1 root root 714 Oct 10 2013 dpkg -rwxr-xr-x 1 root root 51K Oct 10 2013 mysql -rwxr-xr-x 1 root root 747 Nov 19 2013 mysql.cnf -rwxr-xr-x 1 root root 3.2K Mar 22 2014 ntpd
From an entire poller run in debug, the only place I can find the word ‘extension’ is indeed in the mysql output, so a leak from there would make sense as suggested.
I’ve also checked the prerequisites for the MySQL agent and they are all there still after the upgrade – BUT – I can see from the graphs that it is now broken:
[cid:image002.jpg@01D1D91E.E163B7B0]
In the agent debug, line 1899 has “The MySQL extension is not loaded"” – but I can’t see why/why/how this is broken since the 16.04 upgrade.
I’m going to assume that if I fix the MySQL agent, then the arch issue will go away.
So, any ideas on the MySQL problem?
Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com http://www.CustodianDC.com From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: 08 July 2016 10:53 To: Observium Network Observation System observium@observium.org Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
What is the Unix agent returning on this host?
(Did you install distro and/or os scripts?)
Adam.
Sent from BlueMailhttp://www.bluemail.me/r On 8 Jul 2016, at 09:51, Robert Williams <Robert@CustodianDC.commailto:Robert@CustodianDC.com> wrote: Hi – thanks, here you go :)
Only place I can find the word extension is here?
[0;36m o [1;37mMySQL [0m 5.7.12-0ubuntu1.1 (extension: mysqli 5.0.12-dev)
Interestingly, when I run the debug of the one module by itself, the result for “Arch” is null.
Yet when I run all the modules together (copy attached to my first email) it returns “extension” instead?!
So I can produce a different behaviour depending on whether I debug just the system module, or all the modules. A normal polling run obviously produces the “extension” result, I assume because it runs all the modules.
My guess would be that there is a variable which is being left from one of the modules which is leaking into the system module?
Here is the section of the full debug log, which returns “extension”:
including: includes/polling/os.inc.php [0;36m##### [1;37mModule Start: [1;31mos[0;36m #####[0m [0m
SQL[[0;33mSELECT * FROM `observium_attribs`[0m][0m SQL RUNTIME[[0;32m0.00038004s[0m][0m
SQL[[0;33mSELECT * FROM `entPhysical` WHERE `device_id` = '76' AND `entPhysicalContainedIn` = '0'[0m][0m SQL RUNTIME[[0;32m0.00045204s[0m][0m [0;36m o [1;37mOS Poller [0m Group
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m MIB-Dell-10892 -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/dell 'udp':'localhost':'161' chassisModelName.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.05s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'MIB-Dell-10892' AND `oid` = 'chassisModelName.1';[0m][0m SQL RUNTIME[[0;32m0.00053406s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1352'[0m][0m SQL RUNTIME[[0;32m0.00449586s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m CPQSINFO-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/hp 'udp':'localhost':'161' cpqSiProductName.0[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0432s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'CPQSINFO-MIB' AND `oid` = 'cpqSiProductName.0';[0m][0m SQL RUNTIME[[0;32m0.00036812s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1355'[0m][0m SQL RUNTIME[[0;32m0.00263810s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0379s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111';[0m][0m SQL RUNTIME[[0;32m0.00040197s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1356'[0m][0m SQL RUNTIME[[0;32m0.00282502s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.101.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0371s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.101.1';[0m][0m SQL RUNTIME[[0;32m0.00040507s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1358'[0m][0m SQL RUNTIME[[0;32m0.00268698s[0m][0m [0;36m o [1;37mHardware [0m Generic x86 [64bit] [0;36m o [1;37mVersion [0m 4.4.0-28-generic [0;36m o [1;37mFeatures [0m [0;34m<empty>[0m [0;36m o [1;37mSerial [0m [0;34m<empty>[0m [0;36m o [1;37mAsset [0m [0;34m<empty>[0m
SQL[[0;33mINSERT INTO `eventlog` (`device_id`,`entity_id`,`entity_type`,`timestamp`,`severity`,`message`) VALUES ('76','76','device',NOW(),'6','Arch -> extension')[0m][0m SQL RUNTIME[[0;32m0.00246716s[0m][0m [0;36m o [1;37mModule time [0m 0.1949s
Weird…?
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.commailto:Robert@CustodianDC.com http://www.CustodianDC.com From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: 07 July 2016 17:52 To: Observium Network Observation System <observium@observium.orgmailto:observium@observium.org> Cc: Observium Network Observation System <observium@observium.orgmailto:observium@observium.org> Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
Run poller with -m system -d for this host and see what it says. Its probably parsing an error into that field!
Adam.
Sent from BlueMailhttp://www.bluemail.me/r On 7 Jul 2016, at 17:05, Robert Williams <Robert@CustodianDC.commailto:Robert@CustodianDC.com> wrote: Hi,
Just completed a release upgrade on Observium itself (14.04 to 16.04) and now it is reporting its own architecture as ‘extension’.
Wouldn’t be an issue really, but every single time it polls it updates the event log like this:
[cid:1B5D0A9E55561A4E985F21695D4E5946@cdc.local]
I’ve attached a debug output of the poller as this may help to track it down.
Interestingly, a lot of our other servers run 16.04 now but it only reports this for ‘itself’ – not the others.
Anyone have any ideas? Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.commailto:Robert@CustodianDC.com http://www.CustodianDC.com
observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hi,
by first, your mysql agent script is too old, it not support mysqli extension. second, I added additional "fix" to mysql script (in r7956) for prevent unexpected script messages before and instead app identification.
svn up and use new mysql script!
On 08.07.16 15:44, Robert Williams wrote:
Hi,
Thanks - I’ve attached the unix-agent debug now.
As for the scripts, it has the following installed locally (from many years ago!):
:/usr/lib/observium_agent/local# ls -lh
total 68K
-rwxr-xr-x 1 root root 2.9K Oct 10 2013 apache
-rwxr-xr-x 1 root root 714 Oct 10 2013 dpkg
-rwxr-xr-x 1 root root 51K Oct 10 2013 mysql
-rwxr-xr-x 1 root root 747 Nov 19 2013 mysql.cnf
-rwxr-xr-x 1 root root 3.2K Mar 22 2014 ntpd
From an entire poller run in debug, the only place I can find the word ‘extension’ is indeed in the mysql output, so a leak from there would make sense as suggested.
I’ve also checked the prerequisites for the MySQL agent and they are all there still after the upgrade – BUT – I can see from the graphs that it is now broken:
In the agent debug, line 1899 has “The MySQL _extension_ is not loaded"” – but I can’t see why/why/how this is broken since the 16.04 upgrade.
I’m going to assume that if I fix the MySQL agent, then the arch issue will go away.
So, any ideas on the MySQL problem?
Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com http://www.CustodianDC.com
*From:*observium [mailto:observium-bounces@observium.org] *On Behalf Of *Adam Armstrong *Sent:* 08 July 2016 10:53 *To:* Observium Network Observation System observium@observium.org *Subject:* Re: [Observium] Arch -> extension (since 16.04 upgrade)
What is the Unix agent returning on this host?
(Did you install distro and/or os scripts?)
Adam.
Sent from BlueMail http://www.bluemail.me/r
On 8 Jul 2016, at 09:51, Robert Williams <Robert@CustodianDC.com mailto:Robert@CustodianDC.com> wrote:
Hi – thanks, here you go :) Only place I can find the word extension is here? [0;36m o [1;37mMySQL [0m 5.7.12-0ubuntu1.1 (extension: mysqli 5.0.12-dev) Interestingly, when I run the debug of the one module _by itself_, the result for “Arch” is *null*. Yet when I run all the modules together (copy attached to my first email) it returns “*extension*” instead?! So I can produce a different behaviour depending on whether I debug just the system module, or all the modules. A normal polling run obviously produces the “extension” result, I assume because it runs all the modules. My guess would be that there is a variable which is being left from one of the modules which is leaking into the system module? Here is the section of the full debug log, which returns “extension”: including: includes/polling/os.inc.php [0;36m##### [1;37mModule Start: [1;31mos[0;36m #####[0m [0m SQL[[0;33mSELECT * FROM `observium_attribs`[0m][0m SQL RUNTIME[[0;32m0.00038004s[0m][0m SQL[[0;33mSELECT * FROM `entPhysical` WHERE `device_id` = '76' AND `entPhysicalContainedIn` = '0'[0m][0m SQL RUNTIME[[0;32m0.00045204s[0m][0m [0;36m o [1;37mOS Poller [0m Group CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m MIB-Dell-10892 -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/dell 'udp':'localhost':'161' chassisModelName.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.05s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'MIB-Dell-10892' AND `oid` = 'chassisModelName.1';[0m][0m SQL RUNTIME[[0;32m0.00053406s[0m][0m SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1352'[0m][0m SQL RUNTIME[[0;32m0.00449586s[0m][0m CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m CPQSINFO-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/hp 'udp':'localhost':'161' cpqSiProductName.0[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0432s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'CPQSINFO-MIB' AND `oid` = 'cpqSiProductName.0';[0m][0m SQL RUNTIME[[0;32m0.00036812s[0m][0m SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1355'[0m][0m SQL RUNTIME[[0;32m0.00263810s[0m][0m CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0379s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111';[0m][0m SQL RUNTIME[[0;32m0.00040197s[0m][0m SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1356'[0m][0m SQL RUNTIME[[0;32m0.00282502s[0m][0m CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.101.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0371s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.101.1';[0m][0m SQL RUNTIME[[0;32m0.00040507s[0m][0m SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1358'[0m][0m SQL RUNTIME[[0;32m0.00268698s[0m][0m [0;36m o [1;37mHardware [0m Generic x86 [64bit] [0;36m o [1;37mVersion [0m 4.4.0-28-generic [0;36m o [1;37mFeatures [0m [0;34m<empty>[0m [0;36m o [1;37mSerial [0m [0;34m<empty>[0m [0;36m o [1;37mAsset [0m [0;34m<empty>[0m SQL[[0;33mINSERT INTO `eventlog` (`device_id`,`entity_id`,`entity_type`,`timestamp`,`severity`,`message`) VALUES ('76','76','device',NOW(),'6','Arch -> extension')[0m][0m SQL RUNTIME[[0;32m0.00246716s[0m][0m [0;36m o [1;37mModule time [0m 0.1949s Weird…? Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com <mailto:Robert@CustodianDC.com> http://www.CustodianDC.com *From:*observium [mailto:observium-bounces@observium.org] *On Behalf Of *Adam Armstrong *Sent:* 07 July 2016 17:52 *To:* Observium Network Observation System <observium@observium.org <mailto:observium@observium.org>> *Cc:* Observium Network Observation System <observium@observium.org <mailto:observium@observium.org>> *Subject:* Re: [Observium] Arch -> extension (since 16.04 upgrade) Run poller with -m system -d for this host and see what it says. Its probably parsing an error into that field! Adam. Sent from BlueMail <http://www.bluemail.me/r> On 7 Jul 2016, at 17:05, Robert Williams <Robert@CustodianDC.com <mailto:Robert@CustodianDC.com>> wrote: Hi, Just completed a release upgrade on Observium itself (14.04 to 16.04) and now it is reporting its own architecture as ‘extension’. Wouldn’t be an issue really, but every single time it polls it updates the event log like this: I’ve attached a debug output of the poller as this may help to track it down. Interestingly, a lot of our other servers run 16.04 now but it only reports this for ‘itself’ – not the others. Anyone have any ideas? Cheers! Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com <mailto:Robert@CustodianDC.com> http://www.CustodianDC.com ------------------------------------------------------------------------ observium mailing list observium@observium.org <mailto:observium@observium.org> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium ------------------------------------------------------------------------ observium mailing list observium@observium.org <mailto:observium@observium.org> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hi,
Thanks for that - ok, I’ve updated the script and SVN up’d. New error :)
Log now shows this instead:
2m 30s localhost Arch -> (1045) 2m 30s Package removed: The extension MySQL 2m 30s Package installed: Connect ((1045)) version Error
Running the (new) script locally, I now get:
# /usr/lib/observium_agent/local/mysql PHP Warning: syntax error, unexpected '!' in /etc/mysql/debian.cnf on line 1 in usr/lib/observium_agent/local/mysql.cnf on line 10 Connect Error (1045) Access denied for user ''@'localhost' (using password: NO)
The debian.cnf file it speaks of is this:
# Automatically generated for Debian scripts. DO NOT TOUCH! [client] host = localhost user = debian-sys-maint password = aw3vw34sefd1 socket = /var/run/mysqld/mysqld.sock [mysql_upgrade] host = localhost user = debian-sys-maint password = c234f34d1224 socket = /var/run/mysqld/mysqld.sock
Any ideas?
Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com http://www.CustodianDC.com From: observium [mailto:observium-bounces@observium.org] On Behalf Of Mike Stupalov Sent: 08 July 2016 14:34 To: observium@observium.org Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
Hi,
by first, your mysql agent script is too old, it not support mysqli extension. second, I added additional "fix" to mysql script (in r7956) for prevent unexpected script messages before and instead app identification.
svn up and use new mysql script!
On 08.07.16 15:44, Robert Williams wrote: Hi,
Thanks - I’ve attached the unix-agent debug now.
As for the scripts, it has the following installed locally (from many years ago!):
:/usr/lib/observium_agent/local# ls -lh total 68K -rwxr-xr-x 1 root root 2.9K Oct 10 2013 apache -rwxr-xr-x 1 root root 714 Oct 10 2013 dpkg -rwxr-xr-x 1 root root 51K Oct 10 2013 mysql -rwxr-xr-x 1 root root 747 Nov 19 2013 mysql.cnf -rwxr-xr-x 1 root root 3.2K Mar 22 2014 ntpd
From an entire poller run in debug, the only place I can find the word ‘extension’ is indeed in the mysql output, so a leak from there would make sense as suggested.
I’ve also checked the prerequisites for the MySQL agent and they are all there still after the upgrade – BUT – I can see from the graphs that it is now broken:
[cid:image001.jpg@01D1D92A.F2E6F090]
In the agent debug, line 1899 has “The MySQL extension is not loaded"” – but I can’t see why/why/how this is broken since the 16.04 upgrade.
I’m going to assume that if I fix the MySQL agent, then the arch issue will go away.
So, any ideas on the MySQL problem?
Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.commailto:Robert@CustodianDC.com http://www.CustodianDC.com From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: 08 July 2016 10:53 To: Observium Network Observation System observium@observium.orgmailto:observium@observium.org Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
What is the Unix agent returning on this host?
(Did you install distro and/or os scripts?)
Adam.
Sent from BlueMailhttp://www.bluemail.me/r On 8 Jul 2016, at 09:51, Robert Williams <Robert@CustodianDC.commailto:Robert@CustodianDC.com> wrote: Hi – thanks, here you go :)
Only place I can find the word extension is here?
[0;36m o [1;37mMySQL [0m 5.7.12-0ubuntu1.1 (extension: mysqli 5.0.12-dev)
Interestingly, when I run the debug of the one module by itself, the result for “Arch” is null.
Yet when I run all the modules together (copy attached to my first email) it returns “extension” instead?!
So I can produce a different behaviour depending on whether I debug just the system module, or all the modules. A normal polling run obviously produces the “extension” result, I assume because it runs all the modules.
My guess would be that there is a variable which is being left from one of the modules which is leaking into the system module?
Here is the section of the full debug log, which returns “extension”:
including: includes/polling/os.inc.php [0;36m##### [1;37mModule Start: [1;31mos[0;36m #####[0m [0m
SQL[[0;33mSELECT * FROM `observium_attribs`[0m][0m SQL RUNTIME[[0;32m0.00038004s[0m][0m
SQL[[0;33mSELECT * FROM `entPhysical` WHERE `device_id` = '76' AND `entPhysicalContainedIn` = '0'[0m][0m SQL RUNTIME[[0;32m0.00045204s[0m][0m [0;36m o [1;37mOS Poller [0m Group
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m MIB-Dell-10892 -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/dell 'udp':'localhost':'161' chassisModelName.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.05s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'MIB-Dell-10892' AND `oid` = 'chassisModelName.1';[0m][0m SQL RUNTIME[[0;32m0.00053406s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1352'[0m][0m SQL RUNTIME[[0;32m0.00449586s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m CPQSINFO-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/hp 'udp':'localhost':'161' cpqSiProductName.0[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0432s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'CPQSINFO-MIB' AND `oid` = 'cpqSiProductName.0';[0m][0m SQL RUNTIME[[0;32m0.00036812s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1355'[0m][0m SQL RUNTIME[[0;32m0.00263810s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0379s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111';[0m][0m SQL RUNTIME[[0;32m0.00040197s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1356'[0m][0m SQL RUNTIME[[0;32m0.00282502s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.101.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0371s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.101.1';[0m][0m SQL RUNTIME[[0;32m0.00040507s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1358'[0m][0m SQL RUNTIME[[0;32m0.00268698s[0m][0m [0;36m o [1;37mHardware [0m Generic x86 [64bit] [0;36m o [1;37mVersion [0m 4.4.0-28-generic [0;36m o [1;37mFeatures [0m [0;34m<empty>[0m [0;36m o [1;37mSerial [0m [0;34m<empty>[0m [0;36m o [1;37mAsset [0m [0;34m<empty>[0m
SQL[[0;33mINSERT INTO `eventlog` (`device_id`,`entity_id`,`entity_type`,`timestamp`,`severity`,`message`) VALUES ('76','76','device',NOW(),'6','Arch -> extension')[0m][0m SQL RUNTIME[[0;32m0.00246716s[0m][0m [0;36m o [1;37mModule time [0m 0.1949s
Weird…?
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.commailto:Robert@CustodianDC.com http://www.CustodianDC.com From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: 07 July 2016 17:52 To: Observium Network Observation System <observium@observium.orgmailto:observium@observium.org> Cc: Observium Network Observation System <observium@observium.orgmailto:observium@observium.org> Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
Run poller with -m system -d for this host and see what it says. Its probably parsing an error into that field!
Adam.
Sent from BlueMailhttp://www.bluemail.me/r On 7 Jul 2016, at 17:05, Robert Williams <Robert@CustodianDC.commailto:Robert@CustodianDC.com> wrote: Hi,
Just completed a release upgrade on Observium itself (14.04 to 16.04) and now it is reporting its own architecture as ‘extension’.
Wouldn’t be an issue really, but every single time it polls it updates the event log like this:
[cid:1B5D0A9E55561A4E985F21695D4E5946@cdc.local]
I’ve attached a debug output of the poller as this may help to track it down.
Interestingly, a lot of our other servers run 16.04 now but it only reports this for ‘itself’ – not the others.
Anyone have any ideas? Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.commailto:Robert@CustodianDC.com http://www.CustodianDC.com
________________________________
observium mailing list
observium@observium.orgmailto:observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
________________________________
observium mailing list
observium@observium.orgmailto:observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________
observium mailing list
observium@observium.orgmailto:observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
--
Mike Stupalov
You need to point it at a mysql.cnf that matches Observium’s expected, not the system mysql.cnf
Mine looks like this:
<?php
# Observium MySQL Agent Script -> Config File # (c) 2012, Tom Laermans for Observium (www.observium.org)
$mysql_user = ‘observium_mon’; $mysql_pass = ‘MySecretPassword’; ?>
R
On 8 Jul 2016, at 15:13, Robert Williams Robert@CustodianDC.com wrote:
Hi,
Thanks for that - ok, I’ve updated the script and SVN up’d. New error :)
Log now shows this instead:
2m 30s localhost Arch -> (1045) 2m 30s Package removed: The extension MySQL 2m 30s Package installed: Connect ((1045)) version Error
Running the (new) script locally, I now get:
# /usr/lib/observium_agent/local/mysql PHP Warning: syntax error, unexpected '!' in /etc/mysql/debian.cnf on line 1 in usr/lib/observium_agent/local/mysql.cnf on line 10 Connect Error (1045) Access denied for user ''@'localhost' (using password: NO)
The debian.cnf file it speaks of is this:
# Automatically generated for Debian scripts. DO NOT TOUCH! [client] host = localhost user = debian-sys-maint password = aw3vw34sefd1 socket = /var/run/mysqld/mysqld.sock [mysql_upgrade] host = localhost user = debian-sys-maint password = c234f34d1224 socket = /var/run/mysqld/mysqld.sock
Any ideas?
Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com mailto:Robert@custodiandc.com http://www.CustodianDC.com http://www.custodiandc.com/ From: observium [mailto:observium-bounces@observium.org mailto:observium-bounces@observium.org] On Behalf Of Mike Stupalov Sent: 08 July 2016 14:34 To: observium@observium.org mailto:observium@observium.org Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
Hi,
by first, your mysql agent script is too old, it not support mysqli extension. second, I added additional "fix" to mysql script (in r7956) for prevent unexpected script messages before and instead app identification.
svn up and use new mysql script!
On 08.07.16 15:44, Robert Williams wrote: Hi,
Thanks - I’ve attached the unix-agent debug now.
As for the scripts, it has the following installed locally (from many years ago!):
:/usr/lib/observium_agent/local# ls -lh total 68K -rwxr-xr-x 1 root root 2.9K Oct 10 2013 apache -rwxr-xr-x 1 root root 714 Oct 10 2013 dpkg -rwxr-xr-x 1 root root 51K Oct 10 2013 mysql -rwxr-xr-x 1 root root 747 Nov 19 2013 mysql.cnf -rwxr-xr-x 1 root root 3.2K Mar 22 2014 ntpd
From an entire poller run in debug, the only place I can find the word ‘extension’ is indeed in the mysql output, so a leak from there would make sense as suggested.
I’ve also checked the prerequisites for the MySQL agent and they are all there still after the upgrade – BUT – I can see from the graphs that it is now broken:
<image001.jpg>
In the agent debug, line 1899 has “The MySQL extension is not loaded"” – but I can’t see why/why/how this is broken since the 16.04 upgrade.
I’m going to assume that if I fix the MySQL agent, then the arch issue will go away.
So, any ideas on the MySQL problem?
Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com mailto:Robert@CustodianDC.com http://www.CustodianDC.com http://www.custodiandc.com/ From: observium [mailto:observium-bounces@observium.org mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: 08 July 2016 10:53 To: Observium Network Observation System observium@observium.org mailto:observium@observium.org Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
What is the Unix agent returning on this host?
(Did you install distro and/or os scripts?)
Adam.
Sent from BlueMail http://www.bluemail.me/r On 8 Jul 2016, at 09:51, Robert Williams <Robert@CustodianDC.com mailto:Robert@CustodianDC.com> wrote: Hi – thanks, here you go :)
Only place I can find the word extension is here?
[0;36m o [1;37mMySQL [0m 5.7.12-0ubuntu1.1 (extension: mysqli 5.0.12-dev)
Interestingly, when I run the debug of the one module by itself, the result for “Arch” is null.
Yet when I run all the modules together (copy attached to my first email) it returns “extension” instead?!
So I can produce a different behaviour depending on whether I debug just the system module, or all the modules. A normal polling run obviously produces the “extension” result, I assume because it runs all the modules.
My guess would be that there is a variable which is being left from one of the modules which is leaking into the system module?
Here is the section of the full debug log, which returns “extension”:
including: includes/polling/os.inc.php [0;36m##### [1;37mModule Start: [1;31mos[0;36m #####[0m [0m
SQL[[0;33mSELECT * FROM `observium_attribs`[0m][0m SQL RUNTIME[[0;32m0.00038004s[0m][0m
SQL[[0;33mSELECT * FROM `entPhysical` WHERE `device_id` = '76' AND `entPhysicalContainedIn` = '0'[0m][0m SQL RUNTIME[[0;32m0.00045204s[0m][0m [0;36m o [1;37mOS Poller [0m Group
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m MIB-Dell-10892 -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/dell 'udp':'localhost':'161' chassisModelName.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.05s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'MIB-Dell-10892' AND `oid` = 'chassisModelName.1';[0m][0m SQL RUNTIME[[0;32m0.00053406s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1352'[0m][0m SQL RUNTIME[[0;32m0.00449586s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m CPQSINFO-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/hp 'udp':'localhost':'161' cpqSiProductName.0[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0432s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'CPQSINFO-MIB' AND `oid` = 'cpqSiProductName.0';[0m][0m SQL RUNTIME[[0;32m0.00036812s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1355'[0m][0m SQL RUNTIME[[0;32m0.00263810s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0379s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111';[0m][0m SQL RUNTIME[[0;32m0.00040197s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1356'[0m][0m SQL RUNTIME[[0;32m0.00282502s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.101.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0371s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.101.1';[0m][0m SQL RUNTIME[[0;32m0.00040507s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1358'[0m][0m SQL RUNTIME[[0;32m0.00268698s[0m][0m [0;36m o [1;37mHardware [0m Generic x86 [64bit] [0;36m o [1;37mVersion [0m 4.4.0-28-generic [0;36m o [1;37mFeatures [0m [0;34m<empty>[0m [0;36m o [1;37mSerial [0m [0;34m<empty>[0m [0;36m o [1;37mAsset [0m [0;34m<empty>[0m
SQL[[0;33mINSERT INTO `eventlog` (`device_id`,`entity_id`,`entity_type`,`timestamp`,`severity`,`message`) VALUES ('76','76','device',NOW(),'6','Arch -> extension')[0m][0m SQL RUNTIME[[0;32m0.00246716s[0m][0m [0;36m o [1;37mModule time [0m 0.1949s
Weird…?
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com mailto:Robert@CustodianDC.com http://www.CustodianDC.com http://www.custodiandc.com/ From: observium [mailto:observium-bounces@observium.org mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: 07 July 2016 17:52 To: Observium Network Observation System <observium@observium.org mailto:observium@observium.org> Cc: Observium Network Observation System <observium@observium.org mailto:observium@observium.org> Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
Run poller with -m system -d for this host and see what it says. Its probably parsing an error into that field!
Adam.
Sent from BlueMail http://www.bluemail.me/r On 7 Jul 2016, at 17:05, Robert Williams <Robert@CustodianDC.com mailto:Robert@CustodianDC.com> wrote: Hi,
Just completed a release upgrade on Observium itself (14.04 to 16.04) and now it is reporting its own architecture as ‘extension’.
Wouldn’t be an issue really, but every single time it polls it updates the event log like this:
I’ve attached a debug output of the poller as this may help to track it down.
Interestingly, a lot of our other servers run 16.04 now but it only reports this for ‘itself’ – not the others.
Anyone have any ideas? Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com mailto:Robert@CustodianDC.com http://www.CustodianDC.com http://www.custodiandc.com/
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium http://postman.memetic.org/cgi-bin/mailman/listinfo/observium -- Mike Stupalov http://observium.org http://observium.org/
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
mysql.cnf also update from latest svn, this php7 "change in function"
On 08.07.16 17:13, Robert Williams wrote:
Hi,
Thanks for that - ok, I’ve updated the script and SVN up’d. New error :)
Log now shows this instead:
2m 30s localhost Arch -> (1045)
2m 30s Package removed: The extension MySQL
2m 30s Package installed: Connect ((1045)) version Error
Running the (new) script locally, I now get:
# /usr/lib/observium_agent/local/mysql
PHP Warning: syntax error, unexpected '!' in /etc/mysql/debian.cnf on line 1
in usr/lib/observium_agent/local/mysql.cnf on line 10
Connect Error (1045) Access denied for user ''@'localhost' (using password: NO)
The debian.cnf file it speaks of is this:
# Automatically generated for Debian scripts. DO NOT TOUCH!
[client]
host = localhost
user = debian-sys-maint
password = aw3vw34sefd1
socket = /var/run/mysqld/mysqld.sock
[mysql_upgrade]
host = localhost
user = debian-sys-maint
password = c234f34d1224
socket = /var/run/mysqld/mysqld.sock
Any ideas?
Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com http://www.CustodianDC.com
*From:*observium [mailto:observium-bounces@observium.org] *On Behalf Of *Mike Stupalov *Sent:* 08 July 2016 14:34 *To:* observium@observium.org *Subject:* Re: [Observium] Arch -> extension (since 16.04 upgrade)
Hi,
by first, your mysql agent script is too old, it not support mysqli extension. second, I added additional "fix" to mysql script (in r7956) for prevent unexpected script messages before and instead app identification.
svn up and use new mysql script!
On 08.07.16 15:44, Robert Williams wrote:
Hi, Thanks - I’ve attached the unix-agent debug now. As for the scripts, it has the following installed locally (from many years ago!): :/usr/lib/observium_agent/local# ls -lh total 68K -rwxr-xr-x 1 root root 2.9K Oct 10 2013 apache -rwxr-xr-x 1 root root 714 Oct 10 2013 dpkg -rwxr-xr-x 1 root root 51K Oct 10 2013 mysql -rwxr-xr-x 1 root root 747 Nov 19 2013 mysql.cnf -rwxr-xr-x 1 root root 3.2K Mar 22 2014 ntpd From an entire poller run in debug, the only place I can find the word ‘extension’ is indeed in the mysql output, so a leak from there would make sense as suggested. I’ve also checked the prerequisites for the MySQL agent and they are all there still after the upgrade – BUT – I can see from the graphs that it is now broken: In the agent debug, line 1899 has “The MySQL _extension_ is not loaded"” – but I can’t see why/why/how this is broken since the 16.04 upgrade. I’m going to assume that if I fix the MySQL agent, then the arch issue will go away. So, any ideas on the MySQL problem? Cheers! Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com <mailto:Robert@CustodianDC.com> http://www.CustodianDC.com *From:*observium [mailto:observium-bounces@observium.org] *On Behalf Of *Adam Armstrong *Sent:* 08 July 2016 10:53 *To:* Observium Network Observation System <observium@observium.org> <mailto:observium@observium.org> *Subject:* Re: [Observium] Arch -> extension (since 16.04 upgrade) What is the Unix agent returning on this host? (Did you install distro and/or os scripts?) Adam. Sent from BlueMail <http://www.bluemail.me/r> On 8 Jul 2016, at 09:51, Robert Williams <Robert@CustodianDC.com <mailto:Robert@CustodianDC.com>> wrote: Hi – thanks, here you go :) Only place I can find the word extension is here? [0;36m o [1;37mMySQL [0m 5.7.12-0ubuntu1.1 (extension: mysqli 5.0.12-dev) Interestingly, when I run the debug of the one module _by itself_, the result for “Arch” is *null*. Yet when I run all the modules together (copy attached to my first email) it returns “*extension*” instead?! So I can produce a different behaviour depending on whether I debug just the system module, or all the modules. A normal polling run obviously produces the “extension” result, I assume because it runs all the modules. My guess would be that there is a variable which is being left from one of the modules which is leaking into the system module? Here is the section of the full debug log, which returns “extension”: including: includes/polling/os.inc.php [0;36m##### [1;37mModule Start: [1;31mos[0;36m #####[0m [0m SQL[[0;33mSELECT * FROM `observium_attribs`[0m][0m SQL RUNTIME[[0;32m0.00038004s[0m][0m SQL[[0;33mSELECT * FROM `entPhysical` WHERE `device_id` = '76' AND `entPhysicalContainedIn` = '0'[0m][0m SQL RUNTIME[[0;32m0.00045204s[0m][0m [0;36m o [1;37mOS Poller [0m Group CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m MIB-Dell-10892 -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/dell 'udp':'localhost':'161' chassisModelName.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.05s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'MIB-Dell-10892' AND `oid` = 'chassisModelName.1';[0m][0m SQL RUNTIME[[0;32m0.00053406s[0m][0m SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1352'[0m][0m SQL RUNTIME[[0;32m0.00449586s[0m][0m CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m CPQSINFO-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/hp 'udp':'localhost':'161' cpqSiProductName.0[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0432s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'CPQSINFO-MIB' AND `oid` = 'cpqSiProductName.0';[0m][0m SQL RUNTIME[[0;32m0.00036812s[0m][0m SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1355'[0m][0m SQL RUNTIME[[0;32m0.00263810s[0m][0m CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0379s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111';[0m][0m SQL RUNTIME[[0;32m0.00040197s[0m][0m SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1356'[0m][0m SQL RUNTIME[[0;32m0.00282502s[0m][0m CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.101.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0371s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.101.1';[0m][0m SQL RUNTIME[[0;32m0.00040507s[0m][0m SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1358'[0m][0m SQL RUNTIME[[0;32m0.00268698s[0m][0m [0;36m o [1;37mHardware [0m Generic x86 [64bit] [0;36m o [1;37mVersion [0m 4.4.0-28-generic [0;36m o [1;37mFeatures [0m [0;34m<empty>[0m [0;36m o [1;37mSerial [0m [0;34m<empty>[0m [0;36m o [1;37mAsset [0m [0;34m<empty>[0m SQL[[0;33mINSERT INTO `eventlog` (`device_id`,`entity_id`,`entity_type`,`timestamp`,`severity`,`message`) VALUES ('76','76','device',NOW(),'6','Arch -> extension')[0m][0m SQL RUNTIME[[0;32m0.00246716s[0m][0m [0;36m o [1;37mModule time [0m 0.1949s Weird…? Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com <mailto:Robert@CustodianDC.com> http://www.CustodianDC.com *From:*observium [mailto:observium-bounces@observium.org] *On Behalf Of *Adam Armstrong *Sent:* 07 July 2016 17:52 *To:* Observium Network Observation System <observium@observium.org <mailto:observium@observium.org>> *Cc:* Observium Network Observation System <observium@observium.org <mailto:observium@observium.org>> *Subject:* Re: [Observium] Arch -> extension (since 16.04 upgrade) Run poller with -m system -d for this host and see what it says. Its probably parsing an error into that field! Adam. Sent from BlueMail <http://www.bluemail.me/r> On 7 Jul 2016, at 17:05, Robert Williams <Robert@CustodianDC.com <mailto:Robert@CustodianDC.com>> wrote: Hi, Just completed a release upgrade on Observium itself (14.04 to 16.04) and now it is reporting its own architecture as ‘extension’. Wouldn’t be an issue really, but every single time it polls it updates the event log like this: I’ve attached a debug output of the poller as this may help to track it down. Interestingly, a lot of our other servers run 16.04 now but it only reports this for ‘itself’ – not the others. Anyone have any ideas? Cheers! Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com <mailto:Robert@CustodianDC.com> http://www.CustodianDC.com ------------------------------------------------------------------------ observium mailing list observium@observium.org <mailto:observium@observium.org> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium ------------------------------------------------------------------------ observium mailing list observium@observium.org <mailto:observium@observium.org> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org <mailto:observium@observium.org> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium-- Mike Stupalov http://observium.org
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Aha!!
Didn’t do that file at the same time, it’s working now, much awesomeness. Foy anyone else following this, while searching I also found a feature I knew nothing about - the distro detection script, with handy instructions here: http://www.observium.org/docs/distro_script/
Dropped that in just now and that seems to replace / improve on the normal detection significantly; and I now have a groovy Ubuntu logo for the device :)
4m 10s localhost Arch -> amd64 4m 10s localhost Distro_ver -> 16.04 4m 10s localhost Distro -> Ubuntu 4m 10s localhost Features -> Ubuntu 16.04
Thanks again as always guys!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com http://www.CustodianDC.com From: observium [mailto:observium-bounces@observium.org] On Behalf Of Mike Stupalov Sent: 08 July 2016 15:18 To: Observium Network Observation System observium@observium.org Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
mysql.cnf also update from latest svn, this php7 "change in function"
On 08.07.16 17:13, Robert Williams wrote: Hi,
Thanks for that - ok, I’ve updated the script and SVN up’d. New error :)
Log now shows this instead:
2m 30s localhost Arch -> (1045) 2m 30s Package removed: The extension MySQL 2m 30s Package installed: Connect ((1045)) version Error
Running the (new) script locally, I now get:
# /usr/lib/observium_agent/local/mysql PHP Warning: syntax error, unexpected '!' in /etc/mysql/debian.cnf on line 1 in usr/lib/observium_agent/local/mysql.cnf on line 10 Connect Error (1045) Access denied for user ''@'localhost' (using password: NO)
The debian.cnf file it speaks of is this:
# Automatically generated for Debian scripts. DO NOT TOUCH! [client] host = localhost user = debian-sys-maint password = aw3vw34sefd1 socket = /var/run/mysqld/mysqld.sock [mysql_upgrade] host = localhost user = debian-sys-maint password = c234f34d1224 socket = /var/run/mysqld/mysqld.sock
Any ideas?
Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.commailto:Robert@CustodianDC.com http://www.CustodianDC.com From: observium [mailto:observium-bounces@observium.org] On Behalf Of Mike Stupalov Sent: 08 July 2016 14:34 To: observium@observium.orgmailto:observium@observium.org Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
Hi,
by first, your mysql agent script is too old, it not support mysqli extension. second, I added additional "fix" to mysql script (in r7956) for prevent unexpected script messages before and instead app identification.
svn up and use new mysql script!
On 08.07.16 15:44, Robert Williams wrote: Hi,
Thanks - I’ve attached the unix-agent debug now.
As for the scripts, it has the following installed locally (from many years ago!):
:/usr/lib/observium_agent/local# ls -lh total 68K -rwxr-xr-x 1 root root 2.9K Oct 10 2013 apache -rwxr-xr-x 1 root root 714 Oct 10 2013 dpkg -rwxr-xr-x 1 root root 51K Oct 10 2013 mysql -rwxr-xr-x 1 root root 747 Nov 19 2013 mysql.cnf -rwxr-xr-x 1 root root 3.2K Mar 22 2014 ntpd
From an entire poller run in debug, the only place I can find the word ‘extension’ is indeed in the mysql output, so a leak from there would make sense as suggested.
I’ve also checked the prerequisites for the MySQL agent and they are all there still after the upgrade – BUT – I can see from the graphs that it is now broken:
[cid:image001.jpg@01D1D92D.AA435470]
In the agent debug, line 1899 has “The MySQL extension is not loaded"” – but I can’t see why/why/how this is broken since the 16.04 upgrade.
I’m going to assume that if I fix the MySQL agent, then the arch issue will go away.
So, any ideas on the MySQL problem?
Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.commailto:Robert@CustodianDC.com http://www.CustodianDC.com From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: 08 July 2016 10:53 To: Observium Network Observation System observium@observium.orgmailto:observium@observium.org Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
What is the Unix agent returning on this host?
(Did you install distro and/or os scripts?)
Adam.
Sent from BlueMailhttp://www.bluemail.me/r On 8 Jul 2016, at 09:51, Robert Williams <Robert@CustodianDC.commailto:Robert@CustodianDC.com> wrote: Hi – thanks, here you go :)
Only place I can find the word extension is here?
[0;36m o [1;37mMySQL [0m 5.7.12-0ubuntu1.1 (extension: mysqli 5.0.12-dev)
Interestingly, when I run the debug of the one module by itself, the result for “Arch” is null.
Yet when I run all the modules together (copy attached to my first email) it returns “extension” instead?!
So I can produce a different behaviour depending on whether I debug just the system module, or all the modules. A normal polling run obviously produces the “extension” result, I assume because it runs all the modules.
My guess would be that there is a variable which is being left from one of the modules which is leaking into the system module?
Here is the section of the full debug log, which returns “extension”:
including: includes/polling/os.inc.php [0;36m##### [1;37mModule Start: [1;31mos[0;36m #####[0m [0m
SQL[[0;33mSELECT * FROM `observium_attribs`[0m][0m SQL RUNTIME[[0;32m0.00038004s[0m][0m
SQL[[0;33mSELECT * FROM `entPhysical` WHERE `device_id` = '76' AND `entPhysicalContainedIn` = '0'[0m][0m SQL RUNTIME[[0;32m0.00045204s[0m][0m [0;36m o [1;37mOS Poller [0m Group
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m MIB-Dell-10892 -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/dell 'udp':'localhost':'161' chassisModelName.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.05s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'MIB-Dell-10892' AND `oid` = 'chassisModelName.1';[0m][0m SQL RUNTIME[[0;32m0.00053406s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1352'[0m][0m SQL RUNTIME[[0;32m0.00449586s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m CPQSINFO-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/hp 'udp':'localhost':'161' cpqSiProductName.0[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0432s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'CPQSINFO-MIB' AND `oid` = 'cpqSiProductName.0';[0m][0m SQL RUNTIME[[0;32m0.00036812s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1355'[0m][0m SQL RUNTIME[[0;32m0.00263810s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0379s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.3.1.1.6.100.105.115.116.114.111';[0m][0m SQL RUNTIME[[0;32m0.00040197s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1356'[0m][0m SQL RUNTIME[[0;32m0.00282502s[0m][0m
CMD[[0;33m/usr/bin/snmpget -t '5' -r '3' -v2c -c *** -Pu -Oqv -m UCD-SNMP-MIB -M /opt/observium/mibs 'udp':'localhost':'161' .1.3.6.1.4.1.2021.7890.1.101.1[0m] [0m CMD EXITCODE[[0;32m0[0m] CMD RUNTIME[[0;32m0.0371s[0m][0m STDOUT[ No Such Object available on this agent at this OID ][0m SNMP STATUS[[0;31mFALSE[0m][0m SNMP ERROR[[0;31m#1000 - Failed response[0m][0m
SQL[[0;33mSELECT * FROM `snmp_errors` WHERE `device_id` = '76' AND `error_code` = '1000' AND `snmp_cmd` = 'snmpget' AND `snmp_options` = '-Oqv' AND `mib` = 'UCD-SNMP-MIB' AND `oid` = '.1.3.6.1.4.1.2021.7890.1.101.1';[0m][0m SQL RUNTIME[[0;32m0.00040507s[0m][0m
SQL[[0;33mUPDATE `snmp_errors` set `error_count` ='75559',`updated` ='1467906102' WHERE `error_id` = '1358'[0m][0m SQL RUNTIME[[0;32m0.00268698s[0m][0m [0;36m o [1;37mHardware [0m Generic x86 [64bit] [0;36m o [1;37mVersion [0m 4.4.0-28-generic [0;36m o [1;37mFeatures [0m [0;34m<empty>[0m [0;36m o [1;37mSerial [0m [0;34m<empty>[0m [0;36m o [1;37mAsset [0m [0;34m<empty>[0m
SQL[[0;33mINSERT INTO `eventlog` (`device_id`,`entity_id`,`entity_type`,`timestamp`,`severity`,`message`) VALUES ('76','76','device',NOW(),'6','Arch -> extension')[0m][0m SQL RUNTIME[[0;32m0.00246716s[0m][0m [0;36m o [1;37mModule time [0m 0.1949s
Weird…?
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.commailto:Robert@CustodianDC.com http://www.CustodianDC.com From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: 07 July 2016 17:52 To: Observium Network Observation System <observium@observium.orgmailto:observium@observium.org> Cc: Observium Network Observation System <observium@observium.orgmailto:observium@observium.org> Subject: Re: [Observium] Arch -> extension (since 16.04 upgrade)
Run poller with -m system -d for this host and see what it says. Its probably parsing an error into that field!
Adam.
Sent from BlueMailhttp://www.bluemail.me/r On 7 Jul 2016, at 17:05, Robert Williams <Robert@CustodianDC.commailto:Robert@CustodianDC.com> wrote: Hi,
Just completed a release upgrade on Observium itself (14.04 to 16.04) and now it is reporting its own architecture as ‘extension’.
Wouldn’t be an issue really, but every single time it polls it updates the event log like this:
[cid:1B5D0A9E55561A4E985F21695D4E5946@cdc.local]
I’ve attached a debug output of the poller as this may help to track it down.
Interestingly, a lot of our other servers run 16.04 now but it only reports this for ‘itself’ – not the others.
Anyone have any ideas? Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.commailto:Robert@CustodianDC.com http://www.CustodianDC.com
________________________________
observium mailing list
observium@observium.orgmailto:observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
________________________________
observium mailing list
observium@observium.orgmailto:observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________
observium mailing list
observium@observium.orgmailto:observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
--
Mike Stupalov
_______________________________________________
observium mailing list
observium@observium.orgmailto:observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
--
Mike Stupalov
participants (5)
-
Adam Armstrong -
Mike Stupalov -
Richard Franks -
Robert Williams -
Tom Laermans