Salvo of false alerts
This happens when a device fails to correctly respond to an SNMP walk.
adam.
Sent from Mailbird [http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm...] On 30/11/2016 13:37:12, Youssef BENGELLOUN - ZAHR ybzahr@prodware.fr wrote: Dear Community, I received a salvo of false alerts from an HP switch for all its’ interfaces : Alert checker is pretty standard, it raises an alarm if an interface using over 90% of its’ bandwidth : If I look at the eventlogs from the device, I can see a number of events (common to all interfaces) indicating the following : 2016-11-30 14:11:20 Interface changed: [ifAlias] X.X.X.X' -> NULL; [ifHighSpeed] '1000' -> NULL; [ifPromiscuousMode] 'true' -> NULL; [ifConnectorPresent] 'true' -> NULL then, a few minutes later after the poller runs a new cycle : 2016-11-30 14:16:05 Interface changed: [ifAlias] '' -> 'X.X.X.X'; [ifHighSpeed] '' -> '1000'; [ifPromiscuousMode] '' -> 'true'; [ifConnectorPresent] '' -> 'true' In the end, all alarms cleared out. I’m curious to understand why observium would react in such a way. Any one could shed some light ? Thank you. Youssef BENGELLOUN - ZAHR - Consultant Expert Prodware France T : +33 979 999 000 - F : +33 988 814 001 - ybzahr@prodware.fr [mailto:ybzahr@prodware.fr]
Web : prodware.fr [http://www.prodware.fr] [http://twitter.com/Prodware/%5D%C2%A0 [http://www.facebook.com/Prodware/%5D%C2%A0 [https://www.linkedin.com/company/prodwarefrance%5D%C2%A0 [https://www.youtube.com/c/ProdwareFrance%5D%C2%A0 [http://www.viadeo.com/fr/company/prodware%5D%C2%A0 [http://www.prodware.fr/social-network/]
I'm not sure, why would an SNMP walk from your server across your network to your device fail?
adam.
Sent from Mailbird [http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm...] On 30/11/2016 13:46:13, Youssef BENGELLOUN - ZAHR ybzahr@prodware.fr wrote: Dear Adam, Shortly after that, observium detected that device went down due to SNMP being unreachable. Which isn’t true… Question is : considering that the device is basically doing nothing, why would the SNMP walk fail ? Thank you for the feedback. Best regards. Youssef BENGELLOUN - ZAHR - Consultant Expert Prodware France T : +33 979 999 000 - F : +33 988 814 001 - ybzahr@prodware.fr [mailto:ybzahr@prodware.fr]
Web : prodware.fr [http://www.prodware.fr] [http://twitter.com/Prodware/%5D%C2%A0 [http://www.facebook.com/Prodware/%5D%C2%A0 [https://www.linkedin.com/company/prodwarefrance%5D%C2%A0 [https://www.youtube.com/c/ProdwareFrance%5D%C2%A0 [http://www.viadeo.com/fr/company/prodware%5D%C2%A0 [http://www.prodware.fr/social-network/] De : observium observium-bounces@observium.org au nom de Adam Armstrong adama@memetic.org Répondre à : Observium Network Observation System observium@observium.org Date : mercredi 30 novembre 2016 14:42 À : "observium@observium.org" observium@observium.org Objet : Re: [Observium] Salvo of false alerts This happens when a device fails to correctly respond to an SNMP walk. adam. Sent from Mailbird [http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm...] On 30/11/2016 13:37:12, Youssef BENGELLOUN - ZAHR ybzahr@prodware.fr wrote: Dear Community, I received a salvo of false alerts from an HP switch for all its’ interfaces : Alert checker is pretty standard, it raises an alarm if an interface using over 90% of its’ bandwidth : If I look at the eventlogs from the device, I can see a number of events (common to all interfaces) indicating the following : 2016-11-30 14:11:20 Interface changed: [ifAlias] X.X.X.X' -> NULL; [ifHighSpeed] '1000' -> NULL; [ifPromiscuousMode] 'true' -> NULL; [ifConnectorPresent] 'true' -> NULL then, a few minutes later after the poller runs a new cycle : 2016-11-30 14:16:05 Interface changed: [ifAlias] '' -> 'X.X.X.X'; [ifHighSpeed] '' -> '1000'; [ifPromiscuousMode] '' -> 'true'; [ifConnectorPresent] '' -> 'true' In the end, all alarms cleared out. I’m curious to understand why observium would react in such a way. Any one could shed some light ? Thank you. Youssef BENGELLOUN - ZAHR - Consultant Expert Prodware France T : +33 979 999 000 - F : +33 988 814 001 - ybzahr@prodware.fr [mailto:ybzahr@prodware.fr] Web : prodware.fr [http://www.prodware.fr] [http://twitter.com/Prodware/%5D%C2%A0 [http://www.facebook.com/Prodware/%5D%C2%A0 [https://www.linkedin.com/company/prodwarefrance%5D%C2%A0 [https://www.youtube.com/c/ProdwareFrance%5D%C2%A0 [http://www.viadeo.com/fr/company/prodware%5D%C2%A0 [http://www.prodware.fr/social-network/]
On 30 Nov 2016, at 14:50, Adam Armstrong adama@memetic.org wrote:
I'm not sure, why would an SNMP walk from your server across your network to your device fail?
a
FWIIW: we have quite a few few D-Link DGS(Smart) and HP E-series switches (J9xxxA) that die when hit by more than a few HTTP or SNMP-gets/second; or, if the credentials are wrong and it needs to do a syslog() UDP to report such - reproducibly die when the next request arrives prior it having sent the log message.
Dw
This happens alot on devices who are bad at SNMP, sometimes they just seem to fail to respond to certain tables. In your case the interface speed gets changed which causes the %-load on the interface calculation to be wrong and triggers the alarm. My solution for traffic alarms like this is to use the "delay"-option in the alarm checker, then the alarm will only trigger if a devices interface is over the threshold for 2(or more) polls in a row.
/Markus
2016-11-30 14:50 GMT+01:00 Adam Armstrong adama@memetic.org:
I'm not sure, why would an SNMP walk from your server across your network to your device fail?
adam.
Sent from Mailbird http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm_campaign=sent-from-mailbird
On 30/11/2016 13:46:13, Youssef BENGELLOUN - ZAHR ybzahr@prodware.fr wrote:
Dear Adam,
Shortly after that, observium detected that device went down due to SNMP being unreachable. Which isn’t true…
Question is : considering that the device is basically doing nothing, why would the SNMP walk fail ?
Thank you for the feedback.
Best regards.
*Youssef BENGELLOUN - ZAHR* - Consultant Expert Prodware France T : +33 979 999 000 - F : +33 988 814 001 - ybzahr@prodware.fr
Web : prodware.fr http://www.prodware.fr
http://twitter.com/Prodware/ http://www.facebook.com/Prodware/ https://www.linkedin.com/company/prodwarefrance https://www.youtube.com/c/ProdwareFrance http://www.viadeo.com/fr/company/prodware http://www.prodware.fr/social-network/
*De : *observium observium-bounces@observium.org au nom de Adam Armstrong adama@memetic.org *Répondre à : *Observium Network Observation System < observium@observium.org> *Date : *mercredi 30 novembre 2016 14:42 *À : *"observium@observium.org" observium@observium.org *Objet : *Re: [Observium] Salvo of false alerts
This happens when a device fails to correctly respond to an SNMP walk.
adam.
Sent from Mailbird http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm_campaign=sent-from-mailbird
On 30/11/2016 13:37:12, Youssef BENGELLOUN - ZAHR ybzahr@prodware.fr wrote:
Dear Community,
I received a salvo of false alerts from an HP switch for all its’ interfaces :
Alert checker is pretty standard, it raises an alarm if an interface using over 90% of its’ bandwidth :
If I look at the eventlogs from the device, I can see a number of events (common to all interfaces) indicating the following :
2016-11-30 14:11:20 Interface changed: [ifAlias] X.X.X.X' -> NULL; [ifHighSpeed] '1000' -> NULL; [ifPromiscuousMode] 'true' -> NULL; [ifConnectorPresent] 'true' -> NULL
then, a few minutes later after the poller runs a new cycle :
2016-11-30 14:16:05 Interface changed: [ifAlias] '' -> 'X.X.X.X'; [ifHighSpeed] '' -> '1000'; [ifPromiscuousMode] '' -> 'true'; [ifConnectorPresent] '' -> 'true'
In the end, all alarms cleared out.
I’m curious to understand why observium would react in such a way. Any one could shed some light ?
Thank you.
*Youssef BENGELLOUN - ZAHR* - Consultant Expert Prodware France T : +33 979 999 000 - F : +33 988 814 001 - ybzahr@prodware.fr
Web : prodware.fr http://www.prodware.fr
http://twitter.com/Prodware/ http://www.facebook.com/Prodware/ https://www.linkedin.com/company/prodwarefrance https://www.youtube.com/c/ProdwareFrance http://www.viadeo.com/fr/company/prodware http://www.prodware.fr/social-network/
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Yes, you should never have an interface percentage load alert without a delay, really. It just doesn't make sense!
adam.
Sent from Mailbird [http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm...] On 30/11/2016 14:11:13, Markus Klock markus@best-practice.se wrote: This happens alot on devices who are bad at SNMP, sometimes they just seem to fail to respond to certain tables. In your case the interface speed gets changed which causes the %-load on the interface calculation to be wrong and triggers the alarm. My solution for traffic alarms like this is to use the "delay"-option in the alarm checker, then the alarm will only trigger if a devices interface is over the threshold for 2(or more) polls in a row.
/Markus
2016-11-30 14:50 GMT+01:00 Adam Armstrong <adama@memetic.org [mailto:adama@memetic.org]>:
I'm not sure, why would an SNMP walk from your server across your network to your device fail?
adam.
Sent from Mailbird [http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm...] On 30/11/2016 13:46:13, Youssef BENGELLOUN - ZAHR <ybzahr@prodware.fr [mailto:ybzahr@prodware.fr]> wrote: Dear Adam, Shortly after that, observium detected that device went down due to SNMP being unreachable. Which isn’t true… Question is : considering that the device is basically doing nothing, why would the SNMP walk fail ? Thank you for the feedback. Best regards. Youssef BENGELLOUN - ZAHR - Consultant Expert Prodware France T : +33 979 999 000 - F : +33 988 814 001 - ybzahr@prodware.fr [mailto:ybzahr@prodware.fr]
Web : prodware.fr [http://www.prodware.fr] [http://twitter.com/Prodware/] [http://www.facebook.com/Prodware/] [https://www.linkedin.com/company/prodwarefrance] [https://www.youtube.com/c/ProdwareFrance] [http://www.viadeo.com/fr/company/prodware] [http://www.prodware.fr/social-network/] De : observium <observium-bounces@observium.org [mailto:observium-bounces@observium.org]> au nom de Adam Armstrong <adama@memetic.org [mailto:adama@memetic.org]> Répondre à : Observium Network Observation System <observium@observium.org [mailto:observium@observium.org]> Date : mercredi 30 novembre 2016 14:42 À : "observium@observium.org [mailto:observium@observium.org]" <observium@observium.org [mailto:observium@observium.org]> Objet : Re: [Observium] Salvo of false alerts This happens when a device fails to correctly respond to an SNMP walk. adam. Sent from Mailbird [http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm...] On 30/11/2016 13:37:12, Youssef BENGELLOUN - ZAHR <ybzahr@prodware.fr [mailto:ybzahr@prodware.fr]> wrote: Dear Community, I received a salvo of false alerts from an HP switch for all its’ interfaces : Alert checker is pretty standard, it raises an alarm if an interface using over 90% of its’ bandwidth : If I look at the eventlogs from the device, I can see a number of events (common to all interfaces) indicating the following : 2016-11-30 14:11:20 Interface changed: [ifAlias] X.X.X.X' -> NULL; [ifHighSpeed] '1000' -> NULL; [ifPromiscuousMode] 'true' -> NULL; [ifConnectorPresent] 'true' -> NULL then, a few minutes later after the poller runs a new cycle : 2016-11-30 14:16:05 Interface changed: [ifAlias] '' -> 'X.X.X.X'; [ifHighSpeed] '' -> '1000'; [ifPromiscuousMode] '' -> 'true'; [ifConnectorPresent] '' -> 'true' In the end, all alarms cleared out. I’m curious to understand why observium would react in such a way. Any one could shed some light ? Thank you. Youssef BENGELLOUN - ZAHR - Consultant Expert Prodware France T : +33 979 999 000 - F : +33 988 814 001 - ybzahr@prodware.fr [mailto:ybzahr@prodware.fr] Web : prodware.fr [http://www.prodware.fr] [http://twitter.com/Prodware/] [http://www.facebook.com/Prodware/] [https://www.linkedin.com/company/prodwarefrance] [https://www.youtube.com/c/ProdwareFrance] [http://www.viadeo.com/fr/company/prodware] [http://www.prodware.fr/social-network/]
_______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
_______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Because interfaces are at 100% when they're transferring data.
adam.
Sent from Mailbird [http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm...] On 30/11/2016 14:16:21, Youssef BENGELLOUN - ZAHR ybzahr@prodware.fr wrote: Hello, Why wouldn’t make sense ? Y. Youssef BENGELLOUN - ZAHR - Consultant Expert Prodware France T : +33 979 999 000 - F : +33 988 814 001 - ybzahr@prodware.fr [mailto:ybzahr@prodware.fr]
Web : prodware.fr [http://www.prodware.fr] [http://twitter.com/Prodware/%5D%C2%A0 [http://www.facebook.com/Prodware/%5D%C2%A0 [https://www.linkedin.com/company/prodwarefrance%5D%C2%A0 [https://www.youtube.com/c/ProdwareFrance%5D%C2%A0 [http://www.viadeo.com/fr/company/prodware%5D%C2%A0 [http://www.prodware.fr/social-network/] De : observium observium-bounces@observium.org au nom de Adam Armstrong adama@memetic.org Répondre à : Observium Network Observation System observium@observium.org Date : mercredi 30 novembre 2016 15:14 À : "observium@observium.org" observium@observium.org Objet : Re: [Observium] Salvo of false alerts Yes, you should never have an interface percentage load alert without a delay, really. It just doesn't make sense! adam. Sent from Mailbird [http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm...] On 30/11/2016 14:11:13, Markus Klock markus@best-practice.se wrote: This happens alot on devices who are bad at SNMP, sometimes they just seem to fail to respond to certain tables. In your case the interface speed gets changed which causes the %-load on the interface calculation to be wrong and triggers the alarm. My solution for traffic alarms like this is to use the "delay"-option in the alarm checker, then the alarm will only trigger if a devices interface is over the threshold for 2(or more) polls in a row. /Markus 2016-11-30 14:50 GMT+01:00 Adam Armstrong <adama@memetic.org [mailto:adama@memetic.org]>: I'm not sure, why would an SNMP walk from your server across your network to your device fail? adam. Sent from Mailbird [http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm...] On 30/11/2016 13:46:13, Youssef BENGELLOUN - ZAHR <ybzahr@prodware.fr [mailto:ybzahr@prodware.fr]> wrote: Dear Adam, Shortly after that, observium detected that device went down due to SNMP being unreachable. Which isn’t true… Question is : considering that the device is basically doing nothing, why would the SNMP walk fail ? Thank you for the feedback. Best regards. Youssef BENGELLOUN - ZAHR - Consultant Expert Prodware France T : +33 979 999 000 - F : +33 988 814 001 - ybzahr@prodware.fr [mailto:ybzahr@prodware.fr] Erreur ! Le nom du fichier n’a pas été spécifié. Web : prodware.fr [http://www.prodware.fr] Erreur ! Le nom du fichier n’a pas été spécifié. [http://twitter.com/Prodware/%5D%C2%A0 Erreur ! Le nom du fichier n’a pas été spécifié. [http://www.facebook.com/Prodware/%5D%C2%A0 Erreur ! Le nom du fichier n’a pas été spécifié. [https://www.linkedin.com/company/prodwarefrance%5D%C2%A0 Erreur ! Le nom du fichier n’a pas été spécifié. [https://www.youtube.com/c/ProdwareFrance%5D%C2%A0 Erreur ! Le nom du fichier n’a pas été spécifié. [http://www.viadeo.com/fr/company/prodware%5D%C2%A0 Erreur ! Le nom du fichier n’a pas été spécifié. [http://www.prodware.fr/social-network/] De : observium <observium-bounces@observium.org [mailto:observium-bounces@observium.org]> au nom de Adam Armstrong <adama@memetic.org [mailto:adama@memetic.org]> Répondre à : Observium Network Observation System <observium@observium.org [mailto:observium@observium.org]> Date : mercredi 30 novembre 2016 14:42 À : "observium@observium.org [mailto:observium@observium.org]" <observium@observium.org [mailto:observium@observium.org]> Objet : Re: [Observium] Salvo of false alerts This happens when a device fails to correctly respond to an SNMP walk. adam. Sent from Mailbird [http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm...] On 30/11/2016 13:37:12, Youssef BENGELLOUN - ZAHR <ybzahr@prodware.fr [mailto:ybzahr@prodware.fr]> wrote: Dear Community, I received a salvo of false alerts from an HP switch for all its’ interfaces : Erreur ! Le nom du fichier n’a pas été spécifié. Alert checker is pretty standard, it raises an alarm if an interface using over 90% of its’ bandwidth : Erreur ! Le nom du fichier n’a pas été spécifié. Erreur ! Le nom du fichier n’a pas été spécifié. If I look at the eventlogs from the device, I can see a number of events (common to all interfaces) indicating the following : 2016-11-30 14:11:20 Interface changed: [ifAlias] X.X.X.X' -> NULL; [ifHighSpeed] '1000' -> NULL; [ifPromiscuousMode] 'true' -> NULL; [ifConnectorPresent] 'true' -> NULL then, a few minutes later after the poller runs a new cycle : 2016-11-30 14:16:05 Interface changed: [ifAlias] '' -> 'X.X.X.X'; [ifHighSpeed] '' -> '1000'; [ifPromiscuousMode] '' -> 'true'; [ifConnectorPresent] '' -> 'true' In the end, all alarms cleared out. I’m curious to understand why observium would react in such a way. Any one could shed some light ? Thank you. Youssef BENGELLOUN - ZAHR - Consultant Expert Prodware France T : +33 979 999 000 - F : +33 988 814 001 - ybzahr@prodware.fr [mailto:ybzahr@prodware.fr] Erreur ! Le nom du fichier n’a pas été spécifié. Web : prodware.fr [http://www.prodware.fr] Erreur ! Le nom du fichier n’a pas été spécifié. [http://twitter.com/Prodware/%5D%C2%A0 Erreur ! Le nom du fichier n’a pas été spécifié. [http://www.facebook.com/Prodware/%5D%C2%A0 Erreur ! Le nom du fichier n’a pas été spécifié. [https://www.linkedin.com/company/prodwarefrance%5D%C2%A0 Erreur ! Le nom du fichier n’a pas été spécifié. [https://www.youtube.com/c/ProdwareFrance%5D%C2%A0 Erreur ! Le nom du fichier n’a pas été spécifié. [http://www.viadeo.com/fr/company/prodware%5D%C2%A0 Erreur ! Le nom du fichier n’a pas été spécifié. [http://www.prodware.fr/social-network/]
_______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium] _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
participants (4)
-
Adam Armstrong -
Dirk-Willem van Gulik -
Markus Klock -
Youssef BENGELLOUN - ZAHR