This is something that comes up every few months.
In reality we don't actually know if anything is up or down, we only know that something /was/ up or down at some point in the past.
This makes deciding whether to tell you about something that has been detected NOW based on the state of something some minutes in the past somewhat unreliable.
10 superfluous alerts are better than missing 1 legitimate alert, imo.
adam. On 2019-01-31 14:04:35, Jeff Waddell via observium observium@observium.org wrote: Maybe doing something relational -
Like Y devices are dependent on X device - therefore if X device is down, then it is highly probably Y would be down as well - I am sure there is some draw back I am not thinking about here
Thoughts for down the road
On Thu, Jan 31, 2019 at 6:56 AM Adam Armstrong via observium <observium@observium.org [mailto:observium@observium.org]> wrote:
Hi Joey,
Hmm. This was something we thought about when I was first building the alerting system, but I never really came up with a good solution. Suppressing things at random seems like a bad idea.
adam. On 2019-01-31 01:13:11, Joey Stanford via observium <observium@observium.org [mailto:observium@observium.org]> wrote: Hi All,
I’m looking for advice on how to deal with alert storms. For example, we lost a UPS last night at a critical hub which cause Observium not to be able to reach most of our networks (and of course, all the alternative OSPF routes were powered via that UPS). Consequently we had several hundred alerts sent out via email and telegram. Is there any interesting way to deal with this that anyone can recommend?
Thanks,
Joey _______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
_______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
_______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium