Observium stopped firing alerts to contacts
Hi all,
I am having an issues where suddenly observium has stopped sending alerts to contacts. I have had PagerDuty up and running for almost a year and suddenly today we had a device go down and pager duty didn’t fire an alarm. As you can see below the device alarmed but there is not alert send message or the recovery.
[cid:image007.png@01D94138.0CF0EC30]
I have the device grouped in a group aptly named Access Switch. The alarm is configured as follows:
[cid:image008.png@01D94138.67223790]
I also have the pagerduty contact configured for this service. I have blanked the api key for security reasons.
[cid:image009.png@01D94138.E250F230]
Is there a process that handles contacts the needs to be restarted or am I missing something. No changes to my knowledge have been made to the configuration. As I stated earlier this has been working for close to a year. Any help would be appreciated.
Thanks, Andrew
[Woods Hole Oceanographic Institution]https://www.whoi.edu/ Andrew Caplice | Senior Network Engineer Information Services p: (508) 289 2472tel:(508)%20289%200000 c: (774) 722-2633 tel:(508)%20289%200000 acaplice@whoi.eduhttp://www.whoi.edu [facebook]https://www.facebook.com/WoodsHoleOcean [twitter]https://twitter.com/WHOI [instagram]https://www.instagram.com/whoi.ocean/ [YouTube]https://www.youtube.com/user/WoodsHoleOceanInst [linked in]https://www.linkedin.com/school/woods-hole-oceanographic-institution/
good I wanted to know how I can do to put alerts to the Observium to know if a device is turned off or runs out of internet, and I made the SMTP configuration but I have not been able to check if it is working.
[cid:image001.png@01D950F2.8FBAC4F0]
De: Adam Armstrong via observium observium@lists.observium.org Enviado el: viernes, 3 de marzo de 2023 13:04 Para: Observium observium@lists.observium.org CC: Adam Armstrong adama@observium.org Asunto: [Observium] Re: Observium stopped firing alerts to contacts
No, we don’t have any persistent processes.
Alerts are sent by alerted.php which is executed by poller-wrapper.py
You can test alert sending with test_alert.php
Adam. Sent from my iPhone
On 3 Mar 2023, at 16:42, Andrew Caplice via observium <observium@lists.observium.orgmailto:observium@lists.observium.org> wrote: Hi all,
I am having an issues where suddenly observium has stopped sending alerts to contacts. I have had PagerDuty up and running for almost a year and suddenly today we had a device go down and pager duty didn’t fire an alarm. As you can see below the device alarmed but there is not alert send message or the recovery.
<image007.png>
I have the device grouped in a group aptly named Access Switch. The alarm is configured as follows:
<image008.png>
I also have the pagerduty contact configured for this service. I have blanked the api key for security reasons.
<image009.png>
Is there a process that handles contacts the needs to be restarted or am I missing something. No changes to my knowledge have been made to the configuration. As I stated earlier this has been working for close to a year. Any help would be appreciated.
Thanks, Andrew
https://us-east-2.protection.sophos.com?d=whoi.edu&u=aHR0cHM6Ly93d3cud2hvaS5lZHUv&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=TS80QWo0Q0crbUMvMzZ0bE1KaHdxaGVEQTJHamxPR3JiejVpdDkxTUFMND0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck <image001.png> https://us-east-2.protection.sophos.com?d=whoi.edu&u=aHR0cHM6Ly93d3cud2hvaS5lZHUv&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=TS80QWo0Q0crbUMvMzZ0bE1KaHdxaGVEQTJHamxPR3JiejVpdDkxTUFMND0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck
Andrew Caplice | Senior Network Engineer Information Services p: (508) 289 2472tel:(508)%20289%200000 c: (774) 722-2633 tel:(508)%20289%200000 acaplice@whoi.eduhttps://us-east-2.protection.sophos.com?d=whoi.edu&u=aHR0cDovL3d3dy53aG9pLmVkdQ==&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=TmpTaHhpSkhIMHh1L2VWQmtxVVlTRFA5aE1WVmw0RWVCWUFiUVBUY3dTQT0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck https://us-east-2.protection.sophos.com?d=facebook.com&u=aHR0cHM6Ly93d3cuZmFjZWJvb2suY29tL1dvb2RzSG9sZU9jZWFu&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=bGN4YlRSOE4ydytOMnF0Y01MeWFsZnNvU1h3b09IbmxGcGZISXdmUnRqbz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck <image002.png> https://us-east-2.protection.sophos.com?d=facebook.com&u=aHR0cHM6Ly93d3cuZmFjZWJvb2suY29tL1dvb2RzSG9sZU9jZWFu&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=bGN4YlRSOE4ydytOMnF0Y01MeWFsZnNvU1h3b09IbmxGcGZISXdmUnRqbz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck
https://us-east-2.protection.sophos.com?d=twitter.com&u=aHR0cHM6Ly90d2l0dGVyLmNvbS9XSE9J&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=dlFrRE9qY2toOUJWSlpFbEZtMWdaUjBtcm55bVQwVjV6MUx6QVpna3dkcz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck <image003.png> https://us-east-2.protection.sophos.com?d=twitter.com&u=aHR0cHM6Ly90d2l0dGVyLmNvbS9XSE9J&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=dlFrRE9qY2toOUJWSlpFbEZtMWdaUjBtcm55bVQwVjV6MUx6QVpna3dkcz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck
https://us-east-2.protection.sophos.com?d=instagram.com&u=aHR0cHM6Ly93d3cuaW5zdGFncmFtLmNvbS93aG9pLm9jZWFuLw==&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=YmkrQ1FxeWlTUDBMTU5lcktYMnBOK1lnc2hsUHBDV0s5T0hsekt5U2pTaz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck <image004.png> https://us-east-2.protection.sophos.com?d=instagram.com&u=aHR0cHM6Ly93d3cuaW5zdGFncmFtLmNvbS93aG9pLm9jZWFuLw==&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=YmkrQ1FxeWlTUDBMTU5lcktYMnBOK1lnc2hsUHBDV0s5T0hsekt5U2pTaz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck
https://us-east-2.protection.sophos.com?d=youtube.com&u=aHR0cHM6Ly93d3cueW91dHViZS5jb20vdXNlci9Xb29kc0hvbGVPY2Vhbkluc3Q=&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=anNQWnAxdGdOZWtKL1B0Z3VYbDk0ZjFBbTB6VmlOR05vd25XOG1jbkNYdz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck <image005.png> https://us-east-2.protection.sophos.com?d=youtube.com&u=aHR0cHM6Ly93d3cueW91dHViZS5jb20vdXNlci9Xb29kc0hvbGVPY2Vhbkluc3Q=&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=anNQWnAxdGdOZWtKL1B0Z3VYbDk0ZjFBbTB6VmlOR05vd25XOG1jbkNYdz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck
https://us-east-2.protection.sophos.com?d=linkedin.com&u=aHR0cHM6Ly93d3cubGlua2VkaW4uY29tL3NjaG9vbC93b29kcy1ob2xlLW9jZWFub2dyYXBoaWMtaW5zdGl0dXRpb24v&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=bHpwRXl4NS82Q3NVNjNSYUhML3phYVB2b1MzWE93MDBqUnhsenRJWVliaz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck <image006.png> https://us-east-2.protection.sophos.com?d=linkedin.com&u=aHR0cHM6Ly93d3cubGlua2VkaW4uY29tL3NjaG9vbC93b29kcy1ob2xlLW9jZWFub2dyYXBoaWMtaW5zdGl0dXRpb24v&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=bHpwRXl4NS82Q3NVNjNSYUhML3phYVB2b1MzWE93MDBqUnhsenRJWVliaz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck
_______________________________________________ observium mailing list -- observium@lists.observium.orgmailto:observium@lists.observium.org To unsubscribe send an email to observium-leave@lists.observium.orgmailto:observium-leave@lists.observium.org
I'd generally recommend *not* using SMTP. Most problems with email notification come from people trying to use SMTP delivery. It's much more reliable to use a local MTA (just install postfix and answer the questions, that'll be sufficient on Debian/Ubuntu).
adam.
Oscar Sánchez Mora via observium wrote on 07/03/2023 18:45:
good I wanted to know how I can do to put alerts to the Observium to know if a device is turned off or runs out of internet, and I made the SMTP configuration but I have not been able to check if it is working.
*De:*Adam Armstrong via observium observium@lists.observium.org *Enviado el:* viernes, 3 de marzo de 2023 13:04 *Para:* Observium observium@lists.observium.org *CC:* Adam Armstrong adama@observium.org *Asunto:* [Observium] Re: Observium stopped firing alerts to contacts
No, we don’t have any persistent processes.
Alerts are sent by alerted.php which is executed by poller-wrapper.py
You can test alert sending with test_alert.php
Adam.
Sent from my iPhone
On 3 Mar 2023, at 16:42, Andrew Caplice via observium <observium@lists.observium.org <mailto:observium@lists.observium.org>> wrote: Hi all, I am having an issues where suddenly observium has stopped sending alerts to contacts. I have had PagerDuty up and running for almost a year and suddenly today we had a device go down and pager duty didn’t fire an alarm. As you can see below the device alarmed but there is not alert send message or the recovery. <image007.png> I have the device grouped in a group aptly named Access Switch. The alarm is configured as follows: <image008.png> I also have the pagerduty contact configured for this service. I have blanked the api key for security reasons. <image009.png> Is there a process that handles contacts the needs to be restarted or am I missing something. No changes to my knowledge have been made to the configuration. As I stated earlier this has been working for close to a year. Any help would be appreciated. Thanks, Andrew <https://us-east-2.protection.sophos.com?d=whoi.edu&u=aHR0cHM6Ly93d3cud2hvaS5lZHUv&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=TS80QWo0Q0crbUMvMzZ0bE1KaHdxaGVEQTJHamxPR3JiejVpdDkxTUFMND0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck> <image001.png> <https://us-east-2.protection.sophos.com?d=whoi.edu&u=aHR0cHM6Ly93d3cud2hvaS5lZHUv&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=TS80QWo0Q0crbUMvMzZ0bE1KaHdxaGVEQTJHamxPR3JiejVpdDkxTUFMND0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck> *Andrew Caplice *| Senior Network Engineer Information Services p: (508) 289 2472 <tel:%28508%29%20289%200000> c: (774) 722-2633 <tel:%28508%29%20289%200000> *acaplice@whoi.edu* <https://us-east-2.protection.sophos.com?d=whoi.edu&u=aHR0cDovL3d3dy53aG9pLmVkdQ==&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=TmpTaHhpSkhIMHh1L2VWQmtxVVlTRFA5aE1WVmw0RWVCWUFiUVBUY3dTQT0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck> <https://us-east-2.protection.sophos.com?d=facebook.com&u=aHR0cHM6Ly93d3cuZmFjZWJvb2suY29tL1dvb2RzSG9sZU9jZWFu&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=bGN4YlRSOE4ydytOMnF0Y01MeWFsZnNvU1h3b09IbmxGcGZISXdmUnRqbz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck> <image002.png> <https://us-east-2.protection.sophos.com?d=facebook.com&u=aHR0cHM6Ly93d3cuZmFjZWJvb2suY29tL1dvb2RzSG9sZU9jZWFu&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=bGN4YlRSOE4ydytOMnF0Y01MeWFsZnNvU1h3b09IbmxGcGZISXdmUnRqbz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck> <https://us-east-2.protection.sophos.com?d=twitter.com&u=aHR0cHM6Ly90d2l0dGVyLmNvbS9XSE9J&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=dlFrRE9qY2toOUJWSlpFbEZtMWdaUjBtcm55bVQwVjV6MUx6QVpna3dkcz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck> <image003.png> <https://us-east-2.protection.sophos.com?d=twitter.com&u=aHR0cHM6Ly90d2l0dGVyLmNvbS9XSE9J&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=dlFrRE9qY2toOUJWSlpFbEZtMWdaUjBtcm55bVQwVjV6MUx6QVpna3dkcz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck> <https://us-east-2.protection.sophos.com?d=instagram.com&u=aHR0cHM6Ly93d3cuaW5zdGFncmFtLmNvbS93aG9pLm9jZWFuLw==&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=YmkrQ1FxeWlTUDBMTU5lcktYMnBOK1lnc2hsUHBDV0s5T0hsekt5U2pTaz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck> <image004.png> <https://us-east-2.protection.sophos.com?d=instagram.com&u=aHR0cHM6Ly93d3cuaW5zdGFncmFtLmNvbS93aG9pLm9jZWFuLw==&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=YmkrQ1FxeWlTUDBMTU5lcktYMnBOK1lnc2hsUHBDV0s5T0hsekt5U2pTaz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck> <https://us-east-2.protection.sophos.com?d=youtube.com&u=aHR0cHM6Ly93d3cueW91dHViZS5jb20vdXNlci9Xb29kc0hvbGVPY2Vhbkluc3Q=&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=anNQWnAxdGdOZWtKL1B0Z3VYbDk0ZjFBbTB6VmlOR05vd25XOG1jbkNYdz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck> <image005.png> <https://us-east-2.protection.sophos.com?d=youtube.com&u=aHR0cHM6Ly93d3cueW91dHViZS5jb20vdXNlci9Xb29kc0hvbGVPY2Vhbkluc3Q=&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=anNQWnAxdGdOZWtKL1B0Z3VYbDk0ZjFBbTB6VmlOR05vd25XOG1jbkNYdz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck> <https://us-east-2.protection.sophos.com?d=linkedin.com&u=aHR0cHM6Ly93d3cubGlua2VkaW4uY29tL3NjaG9vbC93b29kcy1ob2xlLW9jZWFub2dyYXBoaWMtaW5zdGl0dXRpb24v&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=bHpwRXl4NS82Q3NVNjNSYUhML3phYVB2b1MzWE93MDBqUnhsenRJWVliaz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck> <image006.png> <https://us-east-2.protection.sophos.com?d=linkedin.com&u=aHR0cHM6Ly93d3cubGlua2VkaW4uY29tL3NjaG9vbC93b29kcy1ob2xlLW9jZWFub2dyYXBoaWMtaW5zdGl0dXRpb24v&p=m&i=NjMyZGM2MjkyMGY2ZTAxMjdiY2Q3NDJj&t=bHpwRXl4NS82Q3NVNjNSYUhML3phYVB2b1MzWE93MDBqUnhsenRJWVliaz0=&h=4b1413a2e4b449af855a234b5b98ab24&s=AVNPUEhUT0NFTkNSWVBUSVaOzYUnaqpIdvDtaMzyfhTGnAX500dhcZs0xYklJwC1eQGLFNxdx0G86sjAFzyoPck> _______________________________________________ observium mailing list -- observium@lists.observium.org <mailto:observium@lists.observium.org> To unsubscribe send an email to observium-leave@lists.observium.org <mailto:observium-leave@lists.observium.org>
observium mailing list -- observium@lists.observium.org To unsubscribe send an email to observium-leave@lists.observium.org
participants (3)
-
Adam Armstrong -
Andrew Caplice -
Oscar Sánchez Mora