Observium - Unable to generate email notifications from Syslog Alerts

Hi All,
Observium Professional 17.11.8960 (29th November 2017) OS: Linux 3.16.0-4-amd64 [amd64] (Debian 8.7) Apache: 2.4.10 (Debian) PHP: 5.6.30-0+deb8u1 (OPcache: ENABLED) Python: 2.7.9 MySQL: 5.5.54-0+deb8u1 (extension: mysqli 5.5.54) SNMP: NET-SNMP 5.7.2.1 RRDtool: 1.4.8 (rrdcached 1.4.8: unix:/var/run/rrdcached.sock)
We are unable to generate email notifications from Syslog Alerts.
We are able to generate email notifications from Alert Checkers for the same contacts, and we are able to generate Syslog Alerts on Observium from the Syslog Rules but the associated Contacts do not receive email notifications for those Syslog Alerts.
Essentially everything works except for the last step 'Generating Notifications' following the documentation: http://docs.observium.org/alerting_syslog/
We've run a Rebuild via the Web UI, have re-created the Syslog Rules and Contacts and have been through the global configuration but can't find anything obvious.
Has anyone come across this problem before, or have any ideas on what would be the next step?
Thanks, Shaun? ?
______________________________________________________________________ This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________

We have the exact same issue but haven’t been able to resolve it.
[cid:image001.jpg@01CE9215.D39244B0]https://www.it.ox.ac.uk/
Richard Wigzell Technical Lead, Storage Platforms IT Services | University of Oxford 13 Banbury Road, Oxford OX2 6NN
T: 01865 616282 www.it.ox.ac.ukhttps://www.it.ox.ac.uk/
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Shaun Stokes Sent: 25 January 2018 11:10 To: observium@observium.org Subject: [Observium] Observium - Unable to generate email notifications from Syslog Alerts
Hi All,
Observium Professional 17.11.8960 (29th November 2017) OS: Linux 3.16.0-4-amd64 [amd64] (Debian 8.7) Apache: 2.4.10 (Debian) PHP: 5.6.30-0+deb8u1 (OPcache: ENABLED) Python: 2.7.9 MySQL: 5.5.54-0+deb8u1 (extension: mysqli 5.5.54) SNMP: NET-SNMP 5.7.2.1 RRDtool: 1.4.8 (rrdcached 1.4.8: unix:/var/run/rrdcached.sock)
We are unable to generate email notifications from Syslog Alerts.
We are able to generate email notifications from Alert Checkers for the same contacts, and we are able to generate Syslog Alerts on Observium from the Syslog Rules but the associated Contacts do not receive email notifications for those Syslog Alerts.
Essentially everything works except for the last step 'Generating Notifications' following the documentation: http://docs.observium.org/alerting_syslog/
We've run a Rebuild via the Web UI, have re-created the Syslog Rules and Contacts and have been through the global configuration but can't find anything obvious.
Has anyone come across this problem before, or have any ideas on what would be the next step?
Thanks, Shaun
______________________________________________________________________ This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________

Have had syslog / email related issues for a while. Reported this almost a year ago: http://jira.observium.org/browse/OBS-2208
Not identical; we were still getting emails at the time, it was just generating spurious emails as well. It might not work at all now. Maybe it was disabled intentionally for this reason.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Richard Wigzell Sent: Thursday, January 25, 2018 8:08 AM To: observium@observium.org Subject: Re: [Observium] Observium - Unable to generate email notifications from Syslog Alerts
We have the exact same issue but haven’t been able to resolve it.
Richard Wigzell Technical Lead, Storage Platforms IT Services | University of Oxford
13 Banbury Road, Oxford OX2 6NN
T: 01865 616282 https://www.it.ox.ac.uk/ www.it.ox.ac.uk
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Shaun Stokes Sent: 25 January 2018 11:10 To: observium@observium.org mailto:observium@observium.org Subject: [Observium] Observium - Unable to generate email notifications from Syslog Alerts
Hi All,
Observium Professional 17.11.8960 (29th November 2017)
OS: Linux 3.16.0-4-amd64 [amd64] (Debian 8.7)
Apache: 2.4.10 (Debian)
PHP: 5.6.30-0+deb8u1 (OPcache: ENABLED)
Python: 2.7.9
MySQL: 5.5.54-0+deb8u1 (extension: mysqli 5.5.54)
SNMP: NET-SNMP 5.7.2.1
RRDtool: 1.4.8 (rrdcached 1.4.8: unix:/var/run/rrdcached.sock)
We are unable to generate email notifications from Syslog Alerts.
We are able to generate email notifications from Alert Checkers for the same contacts, and we are able to generate Syslog Alerts on Observium from the Syslog Rules but the associated Contacts do not receive email notifications for those Syslog Alerts.
Essentially everything works except for the last step 'Generating Notifications' following the documentation:
http://docs.observium.org/alerting_syslog/
We've run a Rebuild via the Web UI, have re-created the Syslog Rules and Contacts and have been through the global configuration but can't find anything obvious.
Has anyone come across this problem before, or have any ideas on what would be the next step?
Thanks,
Shaun
______________________________________________________________________ This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________

There were a lot of changes related to Syslog and Notifications on 19/20 June 2017, this is around the time this feature stopped working for us, it seems plausible that one of these could be the culprit.
r8668 | adama | 2017-06-20 21:48:12 +0200 (Tue, 20 Jun 2017) | 2 lines [FIX] Minor rearrangement of syslog notification queuer. ------------------------------------------------------------------------ r8667 | sid3windr | 2017-06-20 10:48:41 +0200 (Tue, 20 Jun 2017) | 1 line [FIX] Fix email backend trying to mail [device_id]@[observium_hostname] when sending alert notifications ------------------------------------------------------------------------ r8666 | mike | 2017-06-19 21:06:33 +0200 (Mon, 19 Jun 2017) | 2 lines [TRIVIAL] Fixed syslog notifications (according to latest changes). ------------------------------------------------------------------------ r8665 | adama | 2017-06-19 02:09:00 +0200 (Mon, 19 Jun 2017) | 2 lines [FIX] Remove "alerted" field in syslog alertlog
?
________________________________ From: observium observium-bounces@observium.org on behalf of Richard Wigzell richard.wigzell@it.ox.ac.uk Sent: 25 January 2018 15:07 To: observium@observium.org Subject: Re: [Observium] Observium - Unable to generate email notifications from Syslog Alerts
We have the exact same issue but haven't been able to resolve it.
[cid:image001.jpg@01CE9215.D39244B0]https://www.it.ox.ac.uk/
Richard Wigzell Technical Lead, Storage Platforms IT Services | University of Oxford 13 Banbury Road, Oxford OX2 6NN
T: 01865 616282 www.it.ox.ac.ukhttps://www.it.ox.ac.uk/
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Shaun Stokes Sent: 25 January 2018 11:10 To: observium@observium.org Subject: [Observium] Observium - Unable to generate email notifications from Syslog Alerts
Hi All,
Observium Professional 17.11.8960 (29th November 2017) OS: Linux 3.16.0-4-amd64 [amd64] (Debian 8.7) Apache: 2.4.10 (Debian) PHP: 5.6.30-0+deb8u1 (OPcache: ENABLED) Python: 2.7.9 MySQL: 5.5.54-0+deb8u1 (extension: mysqli 5.5.54) SNMP: NET-SNMP 5.7.2.1 RRDtool: 1.4.8 (rrdcached 1.4.8: unix:/var/run/rrdcached.sock)
We are unable to generate email notifications from Syslog Alerts.
We are able to generate email notifications from Alert Checkers for the same contacts, and we are able to generate Syslog Alerts on Observium from the Syslog Rules but the associated Contacts do not receive email notifications for those Syslog Alerts.
Essentially everything works except for the last step 'Generating Notifications' following the documentation: http://docs.observium.org/alerting_syslog/
We've run a Rebuild via the Web UI, have re-created the Syslog Rules and Contacts and have been through the global configuration but can't find anything obvious.
Has anyone come across this problem before, or have any ideas on what would be the next step?
Thanks, Shaun? ?
______________________________________________________________________ This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________
______________________________________________________________________ This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________
______________________________________________________________________ This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________

Hi,
by first.. UPDATE your install ;)
Many changes related to notification since your revision.
if you update, pls make and attach debug output for:
./test_alert.php -d -c <id>
where <id> is contact number at left of your contact on "Contacts" page.
(you can send this debug to me outside ML)
Shaun Stokes wrote:
There were a lot of changes related to Syslog and Notifications on 19/20 June 2017, this is around the time this feature stopped working for us, it seems plausible that one of these could be the culprit.
r8668 | adama | 2017-06-20 21:48:12 +0200 (Tue, 20 Jun 2017) | 2 lines [FIX] Minor rearrangement of syslog notification queuer.
r8667 | sid3windr | 2017-06-20 10:48:41 +0200 (Tue, 20 Jun 2017) | 1 line [FIX] Fix email backend trying to mail [device_id]@[observium_hostname] when sending alert notifications
r8666 | mike | 2017-06-19 21:06:33 +0200 (Mon, 19 Jun 2017) | 2 lines [TRIVIAL] Fixed syslog notifications (according to latest changes).
r8665 | adama | 2017-06-19 02:09:00 +0200 (Mon, 19 Jun 2017) | 2 lines [FIX] Remove "alerted" field in syslog alertlog
*From:* observium observium-bounces@observium.org on behalf of Richard Wigzell richard.wigzell@it.ox.ac.uk *Sent:* 25 January 2018 15:07 *To:* observium@observium.org *Subject:* Re: [Observium] Observium - Unable to generate email notifications from Syslog Alerts
We have the exact same issue but haven’t been able to resolve it.
cid:image001.jpg@01CE9215.D39244B0 https://www.it.ox.ac.uk/
*Richard Wigzell* Technical Lead, Storage Platforms IT Services | University of Oxford
13 Banbury Road, Oxford OX2 6NN
*T:* 01865 616282 www.it.ox.ac.uk https://www.it.ox.ac.uk/
*From:*observium [mailto:observium-bounces@observium.org] *On Behalf Of *Shaun Stokes *Sent:* 25 January 2018 11:10 *To:* observium@observium.org *Subject:* [Observium] Observium - Unable to generate email notifications from Syslog Alerts
Hi All,
Observium Professional 17.11.8960 (29th November 2017)
OS: Linux 3.16.0-4-amd64 [amd64] (Debian 8.7)
Apache: 2.4.10 (Debian)
PHP: 5.6.30-0+deb8u1 (OPcache: ENABLED)
Python: 2.7.9
MySQL: 5.5.54-0+deb8u1 (extension: mysqli 5.5.54)
SNMP: NET-SNMP 5.7.2.1
RRDtool: 1.4.8 (rrdcached 1.4.8: unix:/var/run/rrdcached.sock)
We are unable to generate email notifications from Syslog Alerts.
We are able to generate email notifications from Alert Checkers for the same contacts, and we are able to generate Syslog Alerts on Observium from the Syslog Rules but the associated Contacts do not receive email notifications for those Syslog Alerts.
Essentially everything works except for the last step 'Generating Notifications' following the documentation:
http://docs.observium.org/alerting_syslog/
We've run a Rebuild via the Web UI, have re-created the Syslog Rules and Contacts and have been through the global configuration but can't find anything obvious.
Has anyone come across this problem before, or have any ideas on what would be the next step?
Thanks,
Shaun
This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________
This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________
This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________ _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

Hi Mike,
Updated to latest revision.
Device alert notifications working as does './test_alert.php -d -c <id>' but Syslog alert notifications still not working.
Please let me know if you have any other ideas.
Thanks, Shaun ________________________________________ From: observium observium-bounces@observium.org on behalf of Mike Stupalov mike@observium.org Sent: 05 February 2018 13:23 To: Observium Subject: Re: [Observium] Observium - Unable to generate email notifications from Syslog Alerts
Hi,
by first.. UPDATE your install ;)
Many changes related to notification since your revision.
if you update, pls make and attach debug output for:
./test_alert.php -d -c <id>
where <id> is contact number at left of your contact on "Contacts" page.
(you can send this debug to me outside ML)
Shaun Stokes wrote:
There were a lot of changes related to Syslog and Notifications on 19/20 June 2017, this is around the time this feature stopped working for us, it seems plausible that one of these could be the culprit.
r8668 | adama | 2017-06-20 21:48:12 +0200 (Tue, 20 Jun 2017) | 2 lines [FIX] Minor rearrangement of syslog notification queuer.
r8667 | sid3windr | 2017-06-20 10:48:41 +0200 (Tue, 20 Jun 2017) | 1 line [FIX] Fix email backend trying to mail [device_id]@[observium_hostname] when sending alert notifications
r8666 | mike | 2017-06-19 21:06:33 +0200 (Mon, 19 Jun 2017) | 2 lines [TRIVIAL] Fixed syslog notifications (according to latest changes).
r8665 | adama | 2017-06-19 02:09:00 +0200 (Mon, 19 Jun 2017) | 2 lines [FIX] Remove "alerted" field in syslog alertlog
*From:* observium observium-bounces@observium.org on behalf of Richard Wigzell richard.wigzell@it.ox.ac.uk *Sent:* 25 January 2018 15:07 *To:* observium@observium.org *Subject:* Re: [Observium] Observium - Unable to generate email notifications from Syslog Alerts
We have the exact same issue but haven’t been able to resolve it.
cid:image001.jpg@01CE9215.D39244B0 https://www.it.ox.ac.uk/
*Richard Wigzell* Technical Lead, Storage Platforms IT Services | University of Oxford
13 Banbury Road, Oxford OX2 6NN
*T:* 01865 616282 www.it.ox.ac.uk https://www.it.ox.ac.uk/
*From:*observium [mailto:observium-bounces@observium.org] *On Behalf Of *Shaun Stokes *Sent:* 25 January 2018 11:10 *To:* observium@observium.org *Subject:* [Observium] Observium - Unable to generate email notifications from Syslog Alerts
Hi All,
Observium Professional 17.11.8960 (29th November 2017)
OS: Linux 3.16.0-4-amd64 [amd64] (Debian 8.7)
Apache: 2.4.10 (Debian)
PHP: 5.6.30-0+deb8u1 (OPcache: ENABLED)
Python: 2.7.9
MySQL: 5.5.54-0+deb8u1 (extension: mysqli 5.5.54)
SNMP: NET-SNMP 5.7.2.1
RRDtool: 1.4.8 (rrdcached 1.4.8: unix:/var/run/rrdcached.sock)
We are unable to generate email notifications from Syslog Alerts.
We are able to generate email notifications from Alert Checkers for the same contacts, and we are able to generate Syslog Alerts on Observium from the Syslog Rules but the associated Contacts do not receive email notifications for those Syslog Alerts.
Essentially everything works except for the last step 'Generating Notifications' following the documentation:
http://docs.observium.org/alerting_syslog/
We've run a Rebuild via the Web UI, have re-created the Syslog Rules and Contacts and have been through the global configuration but can't find anything obvious.
Has anyone come across this problem before, or have any ideas on what would be the next step?
Thanks,
Shaun
This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________
This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________
This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________ _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
-- Mike Stupalov Observium Limited, http://observium.org _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
______________________________________________________________________ This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________
______________________________________________________________________ This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________

Hi Richard,
This is now working for us (with some help from Mike).
We've updated to the latest revision but the fix was to kill all process ids that were returned when we run 'ps aux | grep syslog.php', then restart rsyslog service.
Hope this helps.
Thanks,
Shaun
________________________________ From: observium observium-bounces@observium.org on behalf of Richard Wigzell richard.wigzell@it.ox.ac.uk Sent: 25 January 2018 15:07 To: observium@observium.org Subject: Re: [Observium] Observium - Unable to generate email notifications from Syslog Alerts
We have the exact same issue but haven't been able to resolve it.
[cid:image001.jpg@01CE9215.D39244B0]https://www.it.ox.ac.uk/
Richard Wigzell Technical Lead, Storage Platforms IT Services | University of Oxford 13 Banbury Road, Oxford OX2 6NN
T: 01865 616282 www.it.ox.ac.ukhttps://www.it.ox.ac.uk/
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Shaun Stokes Sent: 25 January 2018 11:10 To: observium@observium.org Subject: [Observium] Observium - Unable to generate email notifications from Syslog Alerts
Hi All,
Observium Professional 17.11.8960 (29th November 2017) OS: Linux 3.16.0-4-amd64 [amd64] (Debian 8.7) Apache: 2.4.10 (Debian) PHP: 5.6.30-0+deb8u1 (OPcache: ENABLED) Python: 2.7.9 MySQL: 5.5.54-0+deb8u1 (extension: mysqli 5.5.54) SNMP: NET-SNMP 5.7.2.1 RRDtool: 1.4.8 (rrdcached 1.4.8: unix:/var/run/rrdcached.sock)
We are unable to generate email notifications from Syslog Alerts.
We are able to generate email notifications from Alert Checkers for the same contacts, and we are able to generate Syslog Alerts on Observium from the Syslog Rules but the associated Contacts do not receive email notifications for those Syslog Alerts.
Essentially everything works except for the last step 'Generating Notifications' following the documentation: http://docs.observium.org/alerting_syslog/
We've run a Rebuild via the Web UI, have re-created the Syslog Rules and Contacts and have been through the global configuration but can't find anything obvious.
Has anyone come across this problem before, or have any ideas on what would be the next step?
Thanks, Shaun? ?
______________________________________________________________________ This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________
______________________________________________________________________ This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________
______________________________________________________________________ This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________

Hello Shaun,
We will do the same and let everyone know the results.
Many thanks.
[cid:image001.jpg@01CE9215.D39244B0]https://www.it.ox.ac.uk/
Richard Wigzell Technical Lead, Storage Platforms IT Services | University of Oxford 13 Banbury Road, Oxford OX2 6NN
T: 01865 616282 www.it.ox.ac.ukhttps://www.it.ox.ac.uk/
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Shaun Stokes Sent: 07 February 2018 13:39 To: Observium observium@observium.org Subject: Re: [Observium] Observium - Unable to generate email notifications from Syslog Alerts
Hi Richard,
This is now working for us (with some help from Mike).
We've updated to the latest revision but the fix was to kill all process ids that were returned when we run 'ps aux | grep syslog.php', then restart rsyslog service.
Hope this helps.
Thanks,
Shaun
________________________________ From: observium <observium-bounces@observium.orgmailto:observium-bounces@observium.org> on behalf of Richard Wigzell <richard.wigzell@it.ox.ac.ukmailto:richard.wigzell@it.ox.ac.uk> Sent: 25 January 2018 15:07 To: observium@observium.orgmailto:observium@observium.org Subject: Re: [Observium] Observium - Unable to generate email notifications from Syslog Alerts
We have the exact same issue but haven’t been able to resolve it.
[cid:image001.jpg@01CE9215.D39244B0]https://www.it.ox.ac.uk/
Richard Wigzell Technical Lead, Storage Platforms IT Services | University of Oxford 13 Banbury Road, Oxford OX2 6NN
T: 01865 616282 www.it.ox.ac.ukhttps://www.it.ox.ac.uk/
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Shaun Stokes Sent: 25 January 2018 11:10 To: observium@observium.orgmailto:observium@observium.org Subject: [Observium] Observium - Unable to generate email notifications from Syslog Alerts
Hi All,
Observium Professional 17.11.8960 (29th November 2017) OS: Linux 3.16.0-4-amd64 [amd64] (Debian 8.7) Apache: 2.4.10 (Debian) PHP: 5.6.30-0+deb8u1 (OPcache: ENABLED) Python: 2.7.9 MySQL: 5.5.54-0+deb8u1 (extension: mysqli 5.5.54) SNMP: NET-SNMP 5.7.2.1 RRDtool: 1.4.8 (rrdcached 1.4.8: unix:/var/run/rrdcached.sock)
We are unable to generate email notifications from Syslog Alerts.
We are able to generate email notifications from Alert Checkers for the same contacts, and we are able to generate Syslog Alerts on Observium from the Syslog Rules but the associated Contacts do not receive email notifications for those Syslog Alerts.
Essentially everything works except for the last step 'Generating Notifications' following the documentation: http://docs.observium.org/alerting_syslog/
We've run a Rebuild via the Web UI, have re-created the Syslog Rules and Contacts and have been through the global configuration but can't find anything obvious.
Has anyone come across this problem before, or have any ideas on what would be the next step?
Thanks, Shaun
______________________________________________________________________ This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________
______________________________________________________________________ This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________
______________________________________________________________________ This message has been checked for all known viruses by MessageLabs Virus Scanning Service. ______________________________________________________________________
participants (4)
-
Mike Stupalov
-
Richard Wigzell
-
Sean Pedersen
-
Shaun Stokes