We are having a similar issue for some of our devices. In our environment we have tracked it down to be a result of DNS query negative caching in our M$ based DNS servers. This happens to devices with DNS-names NOT in the AD DNS-zone but in zones on other DNS-servers to which the AD DNS-servers do a conditional forwarding to.
We haven't solved this annoyance yet... But we have put some AD-folks on it...
Don't know if this is your issue but it could give you a hint on where to look.
Regards, Tobias
---- Timothy Kaufman skrev ----
We have an SE installation with about 260 devices being checked. We have alerting setup for various services or checks. Often we see false positives on the SNMP device down alert, for example devices will report SNMP down when they are not really down. After a second or even third SNMP check we see a recovery and the uptime shows no outage.
I also verify this by doing and snmpwalk from observium while the "alert" is active.
Is there anything we can check or begin to look to find the cause. The server is not overloaded, with either memory or processor.
We are running 25 poller wrappers with a complete polling time of about 300 seconds.
Thanks,
Tim