Re: [Observium] OBS-2925 - Ping Only Hosts / OBS_SNMP_SKIP flag / snmp_skip attribute
I know where you're at... but,
0) No refactoring required - this is part of the point of the patch ( I would not have even attempted this if it wasn't so easy... ) and my long writeup for Mike given he hadn't actually looked at JIRA yet. Uses existing code/architecture - bit flag and device attributes - to mark a device as "don't do any SNMP to this device right now"
1) Observium uses fping...
2) Smokeping is horrible - can provide great value sure, but horrible to use, and again, extra tool. I'm a long way from unfamiliar with it and have both used it many years ago, and considered it for our current environments. Not useful to us.
3) Smokeping is not integrated to Observium if the device is not monitored by SNMP - this is fine when it's just you or a small team of people who understand this and are happy to flick between interfaces and how to work with Smokeping. Not great when you have a larger team and you really just need one interface on a monitor in your operations room that tells everyone everything that's up/down/problematic right now. Particularly not fine when your level 1's can't really login on CLI and edit config files etc.
4) IP SLA's - vendor specific thing. I love them in the right environment. Fantastic tool and capability to have in a network. We have zero Cisco in this situation and don't want to add a device specifically for this.
5) Your description sounds like it's just you in this environment... and if email fails, or you're hit by a bus, no one else knows there's a problem let alone how your alerting solution works and how to maintain it. Also, manual investigation/access to history. By having Observium do the ping we get to leverage the alerting capabilities inherent to it, e.g. email/webhook/PagerDuty/OpsGenie/VictorOps etc etc etc., as well as the webUI for visual alert notification and access to graphs/history.
It's actually a simpler and cleaner solution; unless you value seeing the min/max/average/jitter values that IP SLA or Smokeping can help you see. In which case it's not what you need.
-Colin
Email: cstubbs @ gmail . com
On Sun, 3 Mar 2019 at 22:44, Ryan Huff ryanhuff@outlook.com wrote:
I really do understand this Colin, on a deeper level than you might imagine ;). I have no vested interest either way personally, I’m just offering some unsolicited thoughts...
I always looked at this specific topic like asking a 10lbs. (4.5kg or 0.7149 stones) sledge hammer to drive a nail. Sure, it can do it but it’ll probably be messy and ugly because a sledge hammer was never meant to drive nails, and when you use tools in ways they weren’t really designed for, things can get messy. Just use a hammer instead.
I have done three things to achieve icmp only monitoring and also stick within the base code (as a developer, I hate mucking with base code or re-inventing wheels when I want to do different things). Icmp-only monitoring to me, always just seemed silly (not in purpose) to try and work into an existing and established application that didn’t already do it.
It’s just too easy of a problem to solve otherwise, than to go to the time/expense of refactoring an application IMHO.
- I use RTT SLAs to monitor next hop devices (which OBS is fabulous at
picking up), assuming the device hopped from is SLA capable and OBS monitored.
- I use Fping (Smokeping) integrated with OBS. Using the Smokeping web
wrapper makes a great RTT only tool, and you also have the OBS integration for when it makes sense to use on OBS devices.
- I have a small (very small) bash script that reads a text file of IP
address and launches one thread per device to run continuous pings. Not a very scalable script, but like you, I only need it for a very few limited cases. The script uses sendmail to email me if it can’t reach something.
Thanks,
Ryan
On Mar 3, 2019, at 06:35, Colin Stubbs via observium < observium@observium.org> wrote:
Hi Mike,
They're in the JIRA ticket, but sure, here's some inserted to this email.
I'm aware of Adam/your preference to reject or simply ignore the regular requests for this kind of feature, and the confusion it causes new users considering Observium for use in their organisations when they find out they can't do the most basic of monitoring to something using Observium.
A great many of your users would gain a lot of value from being able to add a few ping only hosts.
Why build, manage and integrate (to other tools) another ping based monitoring system when all you need is to monitor a handful of devices with ping only; simply because those devices fail to offer SNMP for some reason?
Anything that does have a working SNMP service can and should be monitored with SNMP to gain greater insight into what's happening on it... but not everything does.
No one in their right mind should attempt to use Observium as a system which monitors massive numbers of DNS names or IP's with ping only, there are better systems out there for that, but there are so many situations in a which an Observium install can and should be able to monitor some devices ping only.
I've associated the ticket with the two previous JIRA requests for this feature; which Adam rejected.
They were just feature requests as I recall; no code proposed. Totally understandable you guys didn't want to work on the feature when it's not really what you feel Observium is for.
Rejecting this now when you have just been gifted working code, that modifies things fairly minimally, and simply leverages the ping information that's already occurring to each and every device (that ping is not disabled for), and uses an existing method to mark devices so that further SNMP based polling does not occur... would make no sense.
That said; I totally understand if you want me to fix/add/improve things before considering it further - simply communicate what's necessary and I'll do that.
So, for anyone else on the list, who wants the feature; now is your best chance to add your voice and explain why this kind of feature is of use to you. Because you're probably not going to get it otherwise.
We've been using this internally since early 2017... I just got around to updating Observium to the latest and greatest (reasons, terrible terrible reasons...), and had to update my patch.
But, I'd really prefer not to have to maintain a patch, and I definitely don't want to have to publish it so others can use it and cause Observium Ltd further dramas because of it.
How many hosts do we monitor ping only?
Six. A grand total of six. Everything else has SNMP in some form; and we want as much info as we can extract from them via it - which is why we love Observium. Nothing else compares for SNMP based monitoring IMHO.
Most of those ping only devices are next hop service provider devices just outside our network, for which they won't give us any kind of SNMP access. Ping'ing them offers a small (very small!) degree of insight into link latency/loss from us to them (manual comparison I must admit), and/or potential device problems for the thing that's currently responding for that IP.
Hey look... a perfect use case for this feature...
The others are devices that we use but which simply don't offer SNMP because vendor X didn't bother to add net-snmp to their standard build for CentOS/Ubuntu/whatever weird and wonderful Linux distro they built them on; when they really should have. Don't care much just need to know they're on the network in some way and want a long track record of response time to them.
Hey look... a perfect use case for this feature...
Not surprisingly we don't want to have to operate and maintain a separate system for six IP's. Alerting can be flappy because Observium/fping is only sending a single ICMP echo - we're OK with that and don't even really want more pings which would lead to average values anyway - and we've worked around flaps with delays and automatic suppression of duplicate alerts in our alert management and escalation tool.
The six are monitored by IP; the DNS hostname test below was simply to double check there was no issue with this and a DNS name. Just a hostname that I've never managed to forget and use a lot to ping test connectivity out.
Also, I just realised that the "Skip ping" option is no longer automatically hidden in device settings when "Skip SNMP" is checked; I'll add an updated diff in JIRA shortly after I look at the JS for that again.
<Capture.PNG>
<observium_ping_only_screens3.PNG>
<observium_ping_only_screens4.png>
<observium_ping_only_screens5.PNG>
On Sun, 3 Mar 2019 at 18:07, Mike Stupalov mike@observium.org wrote:
Ohh my dear..
Which really benefit to use Observium for ping_only hosts?
Pls show (screenshots) how this devices displayed on your install.
Colin Stubbs via observium wrote on 03/03/2019 07:43:
For the benefit of anyone on the list who doesn't use JIRA... and also so that others who support (want/need) this feature can comment.
https://jira.observium.org/browse/OBS-2925 https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjira.observium.org%2Fbrowse%2FOBS-2925&data=02%7C01%7C%7C80ff8602df2d450722c308d69fcc50ed%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872097241069291&sdata=1k%2BgXuWMIfh6wZPTkpyusp0hIbIn24X1thOcHPEyfAU%3D&reserved=0
Attached patch defines OBS_SNMP_SKIP flag and uses snmp_skip device attribute, similar to OBS_PING_SKIP and ping_skip, in order to have hosts that are ping only.
Ping only hosts can still have the Observium Unix Agent installed (tested), and other poller modules such as IPMI enabled (untested).
Tested:
- Add/remove/rename ping only hosts via CLI
- Add/remove ping only hosts via webUI
- View/interact ping only hosts via webUI - SNMP specific
features/menus etc are hidden while skip SNMP is enabled 4. Alerting - device_status equals 0 && device_status_type equals ping - will trigger alerts for host down/recovery events 5. Shifting a previously SNMP contactable host to ping only by ticking skip SNMP box - old SNMP graphs/etc are maintained and remain available - remove skip SNMP box and SNMP polling begins again
Things I know kind of don't work right now:
- Location override - poller/discovery doesn't seem to perform
geocoding and whatever else is happening there
Things that could be improved:
- Unix Agent poller module etc is enabled by default for all hosts,
for ping only hosts, perhaps it should be disabled by default? Will improve performance by reducing the number of processes that hang while the 10s default connect timeout happens.
Totally untested:
- Use of autodiscovery SNMP skip - should work in theory, unsure if
those parts of the patch should actually be used though. Some people out there may actually want to add anything that does respond to ping and can be found thru adjacency and routing protocol info etc??
Patch generated from recent trunk, touches files as below,
[root@desktop observium]# diff -r -u observium-trunk root | grep -v ^Only > ping_only_hosts.diff [root@desktop observium]# cd root [root@desktop root]# svn status M add_device.php M html/pages/addhost.inc.php M html/pages/device/edit/device.inc.php M html/pages/device/edit.inc.php M html/pages/device/graphs.inc.php M html/pages/device/perf.inc.php M includes/config-variables.inc.php M includes/defaults.inc.php M includes/definitions.inc.php M includes/discovery/functions.inc.php M includes/functions.inc.php M includes/polling/functions.inc.php M poller.php M rename_device.php [root@desktop root]# svn info | grep ^Revision Revision: 9704 [root@desktop root]# [root@desktop observium]# mv ping_only_hosts.diff ping_only_hosts_r9704.diff
-Colin
Email: cstubbs @ gmail . com
observium mailing listobservium@observium.orghttp://postman.memetic.org/cgi-bin/mailman/listinfo/observium https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.memetic.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fobservium&data=02%7C01%7C%7C80ff8602df2d450722c308d69fcc50ed%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872097241079296&sdata=dtBb4A1H1r%2BZKOcp6ZU8ECUbt5LHMmVjEiud9SdAkfA%3D&reserved=0
-- Mike Stupalov Observium Limited, http://observium.org https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fobservium.org&data=02%7C01%7C%7C80ff8602df2d450722c308d69fcc50ed%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872097241089301&sdata=4hJ48eiAfK7PLOYpeonGgaVqcrgdimtUHkaCPgxA%2FV8%3D&reserved=0
observium mailing list observium@observium.org
https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.mem...
Naaa, mine is a big use case/implementation; it’s just 3 out of a bagillion devices that this needs done for, its not my personal email, it’s a distro. We could easily do an JSON post to any manner of transports, but “mail();” was so much simpler.
All we need to know is when it’s down so someone can go kick the hamster wheel, that’s “ping only”, monitoring to me.
If you’re looking for graphs / SLA type metrics too, then I’d argue you’re already looking for more than “ping only” monitoring....
I see what you’re saying though; OBS has such a great altogether GUI experience, it’d be nice!
-Ryan
On Mar 3, 2019, at 08:15, Colin Stubbs <cstubbs@gmail.commailto:cstubbs@gmail.com> wrote:
I know where you're at... but,
0) No refactoring required - this is part of the point of the patch ( I would not have even attempted this if it wasn't so easy... ) and my long writeup for Mike given he hadn't actually looked at JIRA yet. Uses existing code/architecture - bit flag and device attributes - to mark a device as "don't do any SNMP to this device right now"
1) Observium uses fping...
2) Smokeping is horrible - can provide great value sure, but horrible to use, and again, extra tool. I'm a long way from unfamiliar with it and have both used it many years ago, and considered it for our current environments. Not useful to us.
3) Smokeping is not integrated to Observium if the device is not monitored by SNMP - this is fine when it's just you or a small team of people who understand this and are happy to flick between interfaces and how to work with Smokeping. Not great when you have a larger team and you really just need one interface on a monitor in your operations room that tells everyone everything that's up/down/problematic right now. Particularly not fine when your level 1's can't really login on CLI and edit config files etc.
4) IP SLA's - vendor specific thing. I love them in the right environment. Fantastic tool and capability to have in a network. We have zero Cisco in this situation and don't want to add a device specifically for this.
5) Your description sounds like it's just you in this environment... and if email fails, or you're hit by a bus, no one else knows there's a problem let alone how your alerting solution works and how to maintain it. Also, manual investigation/access to history. By having Observium do the ping we get to leverage the alerting capabilities inherent to it, e.g. email/webhook/PagerDuty/OpsGenie/VictorOps etc etc etc., as well as the webUI for visual alert notification and access to graphs/history.
It's actually a simpler and cleaner solution; unless you value seeing the min/max/average/jitter values that IP SLA or Smokeping can help you see. In which case it's not what you need.
-Colin
Email: cstubbs @ gmail . com
On Sun, 3 Mar 2019 at 22:44, Ryan Huff <ryanhuff@outlook.commailto:ryanhuff@outlook.com> wrote: I really do understand this Colin, on a deeper level than you might imagine ;). I have no vested interest either way personally, I’m just offering some unsolicited thoughts...
I always looked at this specific topic like asking a 10lbs. (4.5kg or 0.7149 stones) sledge hammer to drive a nail. Sure, it can do it but it’ll probably be messy and ugly because a sledge hammer was never meant to drive nails, and when you use tools in ways they weren’t really designed for, things can get messy. Just use a hammer instead.
I have done three things to achieve icmp only monitoring and also stick within the base code (as a developer, I hate mucking with base code or re-inventing wheels when I want to do different things). Icmp-only monitoring to me, always just seemed silly (not in purpose) to try and work into an existing and established application that didn’t already do it.
It’s just too easy of a problem to solve otherwise, than to go to the time/expense of refactoring an application IMHO.
- I use RTT SLAs to monitor next hop devices (which OBS is fabulous at picking up), assuming the device hopped from is SLA capable and OBS monitored.
- I use Fping (Smokeping) integrated with OBS. Using the Smokeping web wrapper makes a great RTT only tool, and you also have the OBS integration for when it makes sense to use on OBS devices.
- I have a small (very small) bash script that reads a text file of IP address and launches one thread per device to run continuous pings. Not a very scalable script, but like you, I only need it for a very few limited cases. The script uses sendmail to email me if it can’t reach something.
Thanks,
Ryan
On Mar 3, 2019, at 06:35, Colin Stubbs via observium <observium@observium.orgmailto:observium@observium.org> wrote:
Hi Mike,
They're in the JIRA ticket, but sure, here's some inserted to this email.
I'm aware of Adam/your preference to reject or simply ignore the regular requests for this kind of feature, and the confusion it causes new users considering Observium for use in their organisations when they find out they can't do the most basic of monitoring to something using Observium.
A great many of your users would gain a lot of value from being able to add a few ping only hosts.
Why build, manage and integrate (to other tools) another ping based monitoring system when all you need is to monitor a handful of devices with ping only; simply because those devices fail to offer SNMP for some reason?
Anything that does have a working SNMP service can and should be monitored with SNMP to gain greater insight into what's happening on it... but not everything does.
No one in their right mind should attempt to use Observium as a system which monitors massive numbers of DNS names or IP's with ping only, there are better systems out there for that, but there are so many situations in a which an Observium install can and should be able to monitor some devices ping only.
I've associated the ticket with the two previous JIRA requests for this feature; which Adam rejected.
They were just feature requests as I recall; no code proposed. Totally understandable you guys didn't want to work on the feature when it's not really what you feel Observium is for.
Rejecting this now when you have just been gifted working code, that modifies things fairly minimally, and simply leverages the ping information that's already occurring to each and every device (that ping is not disabled for), and uses an existing method to mark devices so that further SNMP based polling does not occur... would make no sense.
That said; I totally understand if you want me to fix/add/improve things before considering it further - simply communicate what's necessary and I'll do that.
So, for anyone else on the list, who wants the feature; now is your best chance to add your voice and explain why this kind of feature is of use to you. Because you're probably not going to get it otherwise.
We've been using this internally since early 2017... I just got around to updating Observium to the latest and greatest (reasons, terrible terrible reasons...), and had to update my patch.
But, I'd really prefer not to have to maintain a patch, and I definitely don't want to have to publish it so others can use it and cause Observium Ltd further dramas because of it.
How many hosts do we monitor ping only?
Six. A grand total of six. Everything else has SNMP in some form; and we want as much info as we can extract from them via it - which is why we love Observium. Nothing else compares for SNMP based monitoring IMHO.
Most of those ping only devices are next hop service provider devices just outside our network, for which they won't give us any kind of SNMP access. Ping'ing them offers a small (very small!) degree of insight into link latency/loss from us to them (manual comparison I must admit), and/or potential device problems for the thing that's currently responding for that IP.
Hey look... a perfect use case for this feature...
The others are devices that we use but which simply don't offer SNMP because vendor X didn't bother to add net-snmp to their standard build for CentOS/Ubuntu/whatever weird and wonderful Linux distro they built them on; when they really should have. Don't care much just need to know they're on the network in some way and want a long track record of response time to them.
Hey look... a perfect use case for this feature...
Not surprisingly we don't want to have to operate and maintain a separate system for six IP's. Alerting can be flappy because Observium/fping is only sending a single ICMP echo - we're OK with that and don't even really want more pings which would lead to average values anyway - and we've worked around flaps with delays and automatic suppression of duplicate alerts in our alert management and escalation tool.
The six are monitored by IP; the DNS hostname test below was simply to double check there was no issue with this and a DNS name. Just a hostname that I've never managed to forget and use a lot to ping test connectivity out.
Also, I just realised that the "Skip ping" option is no longer automatically hidden in device settings when "Skip SNMP" is checked; I'll add an updated diff in JIRA shortly after I look at the JS for that again.
<Capture.PNG>
<observium_ping_only_screens3.PNG>
<observium_ping_only_screens4.png>
<observium_ping_only_screens5.PNG>
On Sun, 3 Mar 2019 at 18:07, Mike Stupalov <mike@observium.orgmailto:mike@observium.org> wrote: Ohh my dear..
Which really benefit to use Observium for ping_only hosts?
Pls show (screenshots) how this devices displayed on your install.
Colin Stubbs via observium wrote on 03/03/2019 07:43:
For the benefit of anyone on the list who doesn't use JIRA... and also so that others who support (want/need) this feature can comment.
https://jira.observium.org/browse/OBS-2925https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjira.observium.org%2Fbrowse%2FOBS-2925&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503009597&sdata=J5gbdkhaj1DVqk7F0gvDAk%2B6S25XfYziaa7xh8v5%2Bgc%3D&reserved=0
Attached patch defines OBS_SNMP_SKIP flag and uses snmp_skip device attribute, similar to OBS_PING_SKIP and ping_skip, in order to have hosts that are ping only.
Ping only hosts can still have the Observium Unix Agent installed (tested), and other poller modules such as IPMI enabled (untested).
Tested:
1. Add/remove/rename ping only hosts via CLI 2. Add/remove ping only hosts via webUI 3. View/interact ping only hosts via webUI - SNMP specific features/menus etc are hidden while skip SNMP is enabled 4. Alerting - device_status equals 0 && device_status_type equals ping - will trigger alerts for host down/recovery events 5. Shifting a previously SNMP contactable host to ping only by ticking skip SNMP box - old SNMP graphs/etc are maintained and remain available - remove skip SNMP box and SNMP polling begins again
Things I know kind of don't work right now:
1. Location override - poller/discovery doesn't seem to perform geocoding and whatever else is happening there
Things that could be improved:
1. Unix Agent poller module etc is enabled by default for all hosts, for ping only hosts, perhaps it should be disabled by default? Will improve performance by reducing the number of processes that hang while the 10s default connect timeout happens.
Totally untested:
1. Use of autodiscovery SNMP skip - should work in theory, unsure if those parts of the patch should actually be used though. Some people out there may actually want to add anything that does respond to ping and can be found thru adjacency and routing protocol info etc??
Patch generated from recent trunk, touches files as below,
[root@desktop observium]# diff -r -u observium-trunk root | grep -v ^Only > ping_only_hosts.diff [root@desktop observium]# cd root [root@desktop root]# svn status M add_device.php M html/pages/addhost.inc.php M html/pages/device/edit/device.inc.php M html/pages/device/edit.inc.php M html/pages/device/graphs.inc.php M html/pages/device/perf.inc.php M includes/config-variables.inc.php M includes/defaults.inc.php M includes/definitions.inc.php M includes/discovery/functions.inc.php M includes/functions.inc.php M includes/polling/functions.inc.php M poller.php M rename_device.php [root@desktop root]# svn info | grep ^Revision Revision: 9704 [root@desktop root]# [root@desktop observium]# mv ping_only_hosts.diff ping_only_hosts_r9704.diff
-Colin
Email: cstubbs @ gmail . com
_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observiumhttps://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.memetic.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fobservium&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503019596&sdata=E967lJ4buMEKYKMMawVJYnlaC6MYm2evhD8JZDqESXs%3D&reserved=0
-- Mike Stupalov Observium Limited, http://observium.orghttps://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fobservium.org&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503029613&sdata=bclwJLX8WOiFp4nyfLA2PxXvl40er5OPquD%2FA3XsY3Q%3D&reserved=0
_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.mem...https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.memetic.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fobservium&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503039618&sdata=DH9UrWMzx08fgOTm%2FH4FgeOtYBt4P2K%2BzNIiqtqxIPs%3D&reserved=0
I’ll add my $0.02. In my implementation, we monitor 790 devices, and all of them have SNMP on them. There are a grand total of TWO devices that I’d like to add as ping-only because I don’t have an SNMP string on them… our two uplink internet providers.
Right now we rely on the BGP idle notification to tell us one of the internet uplinks is down. I’ve also built IP SLA on each of our border routers doing ICMP to the next hop IP on the internet side, and of course OBS will graph the results of the IP SLA… and I’m sure there’s probably a way to do an alert notice if the SLA shows a problem, but not all routers are going to have IP SLA capability.
So… to a limited extent, I’d *like* to see the ability to add ping-only devices, but I’m not crippled by not having the capability.
…Ron
--
Ron Marosko, Jr.
. . . . . . . . . . . . . . . . . . . . . . . . . .
CCIE No. 4526 (R/S) Emeritus, NN5DX, RM1517-ARIN
Senior Network Engineer
DirectBytes, LLC.
1108A W. Dickinson Blvd.
Fort Stockton, TX 79735 USA
o: +1 432 336 5600 x115
c: +1 432 290 6344
e: ron@directbytes.net
pgp pubkey: 0x58AB8B5C http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x778B3B0258AB8B5C
"To know me is to fly with me."
From: observium observium-bounces@observium.org On Behalf Of Ryan Huff via observium Sent: Sunday, March 3, 2019 7:44 AM To: cstubbs@gmail.com Cc: Ryan Huff ryanhuff@outlook.com; Observium observium@observium.org Subject: Re: [Observium] OBS-2925 - Ping Only Hosts / OBS_SNMP_SKIP flag / snmp_skip attribute
Naaa, mine is a big use case/implementation; it’s just 3 out of a bagillion devices that this needs done for, its not my personal email, it’s a distro. We could easily do an JSON post to any manner of transports, but “mail();” was so much simpler.
All we need to know is when it’s down so someone can go kick the hamster wheel, that’s “ping only”, monitoring to me.
If you’re looking for graphs / SLA type metrics too, then I’d argue you’re already looking for more than “ping only” monitoring....
I see what you’re saying though; OBS has such a great altogether GUI experience, it’d be nice!
-Ryan
On Mar 3, 2019, at 08:15, Colin Stubbs <cstubbs@gmail.com mailto:cstubbs@gmail.com > wrote:
I know where you're at... but,
0) No refactoring required - this is part of the point of the patch ( I would not have even attempted this if it wasn't so easy... ) and my long writeup for Mike given he hadn't actually looked at JIRA yet. Uses existing code/architecture - bit flag and device attributes - to mark a device as "don't do any SNMP to this device right now"
1) Observium uses fping...
2) Smokeping is horrible - can provide great value sure, but horrible to use, and again, extra tool. I'm a long way from unfamiliar with it and have both used it many years ago, and considered it for our current environments. Not useful to us.
3) Smokeping is not integrated to Observium if the device is not monitored by SNMP - this is fine when it's just you or a small team of people who understand this and are happy to flick between interfaces and how to work with Smokeping. Not great when you have a larger team and you really just need one interface on a monitor in your operations room that tells everyone everything that's up/down/problematic right now. Particularly not fine when your level 1's can't really login on CLI and edit config files etc.
4) IP SLA's - vendor specific thing. I love them in the right environment. Fantastic tool and capability to have in a network. We have zero Cisco in this situation and don't want to add a device specifically for this.
5) Your description sounds like it's just you in this environment... and if email fails, or you're hit by a bus, no one else knows there's a problem let alone how your alerting solution works and how to maintain it. Also, manual investigation/access to history. By having Observium do the ping we get to leverage the alerting capabilities inherent to it, e.g. email/webhook/PagerDuty/OpsGenie/VictorOps etc etc etc., as well as the webUI for visual alert notification and access to graphs/history.
It's actually a simpler and cleaner solution; unless you value seeing the min/max/average/jitter values that IP SLA or Smokeping can help you see. In which case it's not what you need.
-Colin
Email: cstubbs @ gmail . com
On Sun, 3 Mar 2019 at 22:44, Ryan Huff <ryanhuff@outlook.com mailto:ryanhuff@outlook.com > wrote:
I really do understand this Colin, on a deeper level than you might imagine ;). I have no vested interest either way personally, I’m just offering some unsolicited thoughts...
I always looked at this specific topic like asking a 10lbs. (4.5kg or 0.7149 stones) sledge hammer to drive a nail. Sure, it can do it but it’ll probably be messy and ugly because a sledge hammer was never meant to drive nails, and when you use tools in ways they weren’t really designed for, things can get messy. Just use a hammer instead.
I have done three things to achieve icmp only monitoring and also stick within the base code (as a developer, I hate mucking with base code or re-inventing wheels when I want to do different things). Icmp-only monitoring to me, always just seemed silly (not in purpose) to try and work into an existing and established application that didn’t already do it.
It’s just too easy of a problem to solve otherwise, than to go to the time/expense of refactoring an application IMHO.
- I use RTT SLAs to monitor next hop devices (which OBS is fabulous at picking up), assuming the device hopped from is SLA capable and OBS monitored.
- I use Fping (Smokeping) integrated with OBS. Using the Smokeping web wrapper makes a great RTT only tool, and you also have the OBS integration for when it makes sense to use on OBS devices.
- I have a small (very small) bash script that reads a text file of IP address and launches one thread per device to run continuous pings. Not a very scalable script, but like you, I only need it for a very few limited cases. The script uses sendmail to email me if it can’t reach something.
Thanks,
Ryan
On Mar 3, 2019, at 06:35, Colin Stubbs via observium <observium@observium.org mailto:observium@observium.org > wrote:
Hi Mike,
They're in the JIRA ticket, but sure, here's some inserted to this email.
I'm aware of Adam/your preference to reject or simply ignore the regular requests for this kind of feature, and the confusion it causes new users considering Observium for use in their organisations when they find out they can't do the most basic of monitoring to something using Observium.
A great many of your users would gain a lot of value from being able to add a few ping only hosts.
Why build, manage and integrate (to other tools) another ping based monitoring system when all you need is to monitor a handful of devices with ping only; simply because those devices fail to offer SNMP for some reason?
Anything that does have a working SNMP service can and should be monitored with SNMP to gain greater insight into what's happening on it... but not everything does.
No one in their right mind should attempt to use Observium as a system which monitors massive numbers of DNS names or IP's with ping only, there are better systems out there for that, but there are so many situations in a which an Observium install can and should be able to monitor some devices ping only.
I've associated the ticket with the two previous JIRA requests for this feature; which Adam rejected.
They were just feature requests as I recall; no code proposed. Totally understandable you guys didn't want to work on the feature when it's not really what you feel Observium is for.
Rejecting this now when you have just been gifted working code, that modifies things fairly minimally, and simply leverages the ping information that's already occurring to each and every device (that ping is not disabled for), and uses an existing method to mark devices so that further SNMP based polling does not occur... would make no sense.
That said; I totally understand if you want me to fix/add/improve things before considering it further - simply communicate what's necessary and I'll do that.
So, for anyone else on the list, who wants the feature; now is your best chance to add your voice and explain why this kind of feature is of use to you. Because you're probably not going to get it otherwise.
We've been using this internally since early 2017... I just got around to updating Observium to the latest and greatest (reasons, terrible terrible reasons...), and had to update my patch.
But, I'd really prefer not to have to maintain a patch, and I definitely don't want to have to publish it so others can use it and cause Observium Ltd further dramas because of it.
How many hosts do we monitor ping only?
Six. A grand total of six. Everything else has SNMP in some form; and we want as much info as we can extract from them via it - which is why we love Observium. Nothing else compares for SNMP based monitoring IMHO.
Most of those ping only devices are next hop service provider devices just outside our network, for which they won't give us any kind of SNMP access. Ping'ing them offers a small (very small!) degree of insight into link latency/loss from us to them (manual comparison I must admit), and/or potential device problems for the thing that's currently responding for that IP.
Hey look... a perfect use case for this feature...
The others are devices that we use but which simply don't offer SNMP because vendor X didn't bother to add net-snmp to their standard build for CentOS/Ubuntu/whatever weird and wonderful Linux distro they built them on; when they really should have. Don't care much just need to know they're on the network in some way and want a long track record of response time to them.
Hey look... a perfect use case for this feature...
Not surprisingly we don't want to have to operate and maintain a separate system for six IP's. Alerting can be flappy because Observium/fping is only sending a single ICMP echo - we're OK with that and don't even really want more pings which would lead to average values anyway - and we've worked around flaps with delays and automatic suppression of duplicate alerts in our alert management and escalation tool.
The six are monitored by IP; the DNS hostname test below was simply to double check there was no issue with this and a DNS name. Just a hostname that I've never managed to forget and use a lot to ping test connectivity out.
Also, I just realised that the "Skip ping" option is no longer automatically hidden in device settings when "Skip SNMP" is checked; I'll add an updated diff in JIRA shortly after I look at the JS for that again.
<Capture.PNG>
<observium_ping_only_screens3.PNG>
<observium_ping_only_screens4.png>
<observium_ping_only_screens5.PNG>
On Sun, 3 Mar 2019 at 18:07, Mike Stupalov <mike@observium.org mailto:mike@observium.org > wrote:
Ohh my dear..
Which really benefit to use Observium for ping_only hosts?
Pls show (screenshots) how this devices displayed on your install.
Colin Stubbs via observium wrote on 03/03/2019 07:43:
For the benefit of anyone on the list who doesn't use JIRA... and also so that others who support (want/need) this feature can comment.
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjira.observium.org%2Fbrowse%2FOBS-2925&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503009597&sdata=J5gbdkhaj1DVqk7F0gvDAk%2B6S25XfYziaa7xh8v5%2Bgc%3D&reserved=0 MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be https://jira.observium.org/browse/OBS-2925
Attached patch defines OBS_SNMP_SKIP flag and uses snmp_skip device attribute, similar to OBS_PING_SKIP and ping_skip, in order to have hosts that are ping only.
Ping only hosts can still have the Observium Unix Agent installed (tested), and other poller modules such as IPMI enabled (untested).
Tested:
1. Add/remove/rename ping only hosts via CLI 2. Add/remove ping only hosts via webUI 3. View/interact ping only hosts via webUI - SNMP specific features/menus etc are hidden while skip SNMP is enabled 4. Alerting - device_status equals 0 && device_status_type equals ping - will trigger alerts for host down/recovery events 5. Shifting a previously SNMP contactable host to ping only by ticking skip SNMP box - old SNMP graphs/etc are maintained and remain available - remove skip SNMP box and SNMP polling begins again
Things I know kind of don't work right now:
1. Location override - poller/discovery doesn't seem to perform geocoding and whatever else is happening there
Things that could be improved:
1. Unix Agent poller module etc is enabled by default for all hosts, for ping only hosts, perhaps it should be disabled by default? Will improve performance by reducing the number of processes that hang while the 10s default connect timeout happens.
Totally untested:
1. Use of autodiscovery SNMP skip - should work in theory, unsure if those parts of the patch should actually be used though. Some people out there may actually want to add anything that does respond to ping and can be found thru adjacency and routing protocol info etc??
Patch generated from recent trunk, touches files as below,
[root@desktop observium]# diff -r -u observium-trunk root | grep -v ^Only > ping_only_hosts.diff [root@desktop observium]# cd root [root@desktop root]# svn status M add_device.php M html/pages/addhost.inc.php M html/pages/device/edit/device.inc.php M html/pages/device/edit.inc.php M html/pages/device/graphs.inc.php M html/pages/device/perf.inc.php M includes/config-variables.inc.php M includes/defaults.inc.php M includes/definitions.inc.php M includes/discovery/functions.inc.php M includes/functions.inc.php M includes/polling/functions.inc.php M poller.php M rename_device.php [root@desktop root]# svn info | grep ^Revision Revision: 9704 [root@desktop root]# [root@desktop observium]# mv ping_only_hosts.diff ping_only_hosts_r9704.diff
-Colin
Email: cstubbs @ gmail . com
_______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.memetic.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fobservium&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503019596&sdata=E967lJ4buMEKYKMMawVJYnlaC6MYm2evhD8JZDqESXs%3D&reserved=0 MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
My usual answer to this is that this causes too much weirdness in the overall system to have "devices" which have zero information attached to them, so "just ping" should probably be a separate thing, perhaps done by a separate poller process. Just a list of hosts to ping.
This probably isn't very difficult to implement, and may be useful for other things (pinging external things to monitor availability a la IP SLA, but without an external device. I'm not sure how good a job we would do of this though. I think fping may help. Just a single fping run should supply the data for all pinged devices, right?
One of the problems with donated code is that it invariably gets abandoned by its original creators and then leaves us to maintain it to handle a 0.1% corner case. This has happened to us more than once in the past, and is now why we're resistant to implementing odd things that impact other parts of code and don't seem to have wide utility.
adam. On 2019-03-03 16:52:44, Ron Marosko via observium observium@observium.org wrote: I’ll add my $0.02. In my implementation, we monitor 790 devices, and all of them have SNMP on them. There are a grand total of TWO devices that I’d like to add as ping-only because I don’t have an SNMP string on them… our two uplink internet providers. Right now we rely on the BGP idle notification to tell us one of the internet uplinks is down. I’ve also built IP SLA on each of our border routers doing ICMP to the next hop IP on the internet side, and of course OBS will graph the results of the IP SLA… and I’m sure there’s probably a way to do an alert notice if the SLA shows a problem, but not all routers are going to have IP SLA capability. So… to a limited extent, I’d *like* to see the ability to add ping-only devices, but I’m not crippled by not having the capability. …Ron -- Ron Marosko, Jr. . . . . . . . . . . . . . . . . . . . . . . . . . . CCIE No. 4526 (R/S) Emeritus, NN5DX, RM1517-ARIN Senior Network Engineer DirectBytes, LLC. 1108A W. Dickinson Blvd. Fort Stockton, TX 79735 USA o: +1 432 336 5600 x115 c: +1 432 290 6344 e: ron@directbytes.net pgp pubkey: 0x58AB8B5C [http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x778B3B0258AB8B5C] "To know me is to fly with me." From: observium observium-bounces@observium.org On Behalf Of Ryan Huff via observium Sent: Sunday, March 3, 2019 7:44 AM To: cstubbs@gmail.com Cc: Ryan Huff ryanhuff@outlook.com; Observium observium@observium.org Subject: Re: [Observium] OBS-2925 - Ping Only Hosts / OBS_SNMP_SKIP flag / snmp_skip attribute Naaa, mine is a big use case/implementation; it’s just 3 out of a bagillion devices that this needs done for, its not my personal email, it’s a distro. We could easily do an JSON post to any manner of transports, but “mail();” was so much simpler. All we need to know is when it’s down so someone can go kick the hamster wheel, that’s “ping only”, monitoring to me. If you’re looking for graphs / SLA type metrics too, then I’d argue you’re already looking for more than “ping only” monitoring.... I see what you’re saying though; OBS has such a great altogether GUI experience, it’d be nice!
-Ryan
On Mar 3, 2019, at 08:15, Colin Stubbs <cstubbs@gmail.com [mailto:cstubbs@gmail.com]> wrote: I know where you're at... but, 0) No refactoring required - this is part of the point of the patch ( I would not have even attempted this if it wasn't so easy... ) and my long writeup for Mike given he hadn't actually looked at JIRA yet. Uses existing code/architecture - bit flag and device attributes - to mark a device as "don't do any SNMP to this device right now" 1) Observium uses fping... 2) Smokeping is horrible - can provide great value sure, but horrible to use, and again, extra tool. I'm a long way from unfamiliar with it and have both used it many years ago, and considered it for our current environments. Not useful to us. 3) Smokeping is not integrated to Observium if the device is not monitored by SNMP - this is fine when it's just you or a small team of people who understand this and are happy to flick between interfaces and how to work with Smokeping. Not great when you have a larger team and you really just need one interface on a monitor in your operations room that tells everyone everything that's up/down/problematic right now. Particularly not fine when your level 1's can't really login on CLI and edit config files etc. 4) IP SLA's - vendor specific thing. I love them in the right environment. Fantastic tool and capability to have in a network. We have zero Cisco in this situation and don't want to add a device specifically for this. 5) Your description sounds like it's just you in this environment... and if email fails, or you're hit by a bus, no one else knows there's a problem let alone how your alerting solution works and how to maintain it. Also, manual investigation/access to history. By having Observium do the ping we get to leverage the alerting capabilities inherent to it, e.g. email/webhook/PagerDuty/OpsGenie/VictorOps etc etc etc., as well as the webUI for visual alert notification and access to graphs/history. It's actually a simpler and cleaner solution; unless you value seeing the min/max/average/jitter values that IP SLA or Smokeping can help you see. In which case it's not what you need.
-Colin
Email: cstubbs @ gmail . com On Sun, 3 Mar 2019 at 22:44, Ryan Huff <ryanhuff@outlook.com [mailto:ryanhuff@outlook.com]> wrote: I really do understand this Colin, on a deeper level than you might imagine ;). I have no vested interest either way personally, I’m just offering some unsolicited thoughts... I always looked at this specific topic like asking a 10lbs. (4.5kg or 0.7149 stones) sledge hammer to drive a nail. Sure, it can do it but it’ll probably be messy and ugly because a sledge hammer was never meant to drive nails, and when you use tools in ways they weren’t really designed for, things can get messy. Just use a hammer instead. I have done three things to achieve icmp only monitoring and also stick within the base code (as a developer, I hate mucking with base code or re-inventing wheels when I want to do different things). Icmp-only monitoring to me, always just seemed silly (not in purpose) to try and work into an existing and established application that didn’t already do it. It’s just too easy of a problem to solve otherwise, than to go to the time/expense of refactoring an application IMHO. - I use RTT SLAs to monitor next hop devices (which OBS is fabulous at picking up), assuming the device hopped from is SLA capable and OBS monitored. - I use Fping (Smokeping) integrated with OBS. Using the Smokeping web wrapper makes a great RTT only tool, and you also have the OBS integration for when it makes sense to use on OBS devices. - I have a small (very small) bash script that reads a text file of IP address and launches one thread per device to run continuous pings. Not a very scalable script, but like you, I only need it for a very few limited cases. The script uses sendmail to email me if it can’t reach something. Thanks, Ryan On Mar 3, 2019, at 06:35, Colin Stubbs via observium <observium@observium.org [mailto:observium@observium.org]> wrote: Hi Mike, They're in the JIRA ticket, but sure, here's some inserted to this email. I'm aware of Adam/your preference to reject or simply ignore the regular requests for this kind of feature, and the confusion it causes new users considering Observium for use in their organisations when they find out they can't do the most basic of monitoring to something using Observium. A great many of your users would gain a lot of value from being able to add a few ping only hosts. Why build, manage and integrate (to other tools) another ping based monitoring system when all you need is to monitor a handful of devices with ping only; simply because those devices fail to offer SNMP for some reason? Anything that does have a working SNMP service can and should be monitored with SNMP to gain greater insight into what's happening on it... but not everything does. No one in their right mind should attempt to use Observium as a system which monitors massive numbers of DNS names or IP's with ping only, there are better systems out there for that, but there are so many situations in a which an Observium install can and should be able to monitor some devices ping only. I've associated the ticket with the two previous JIRA requests for this feature; which Adam rejected. They were just feature requests as I recall; no code proposed. Totally understandable you guys didn't want to work on the feature when it's not really what you feel Observium is for. Rejecting this now when you have just been gifted working code, that modifies things fairly minimally, and simply leverages the ping information that's already occurring to each and every device (that ping is not disabled for), and uses an existing method to mark devices so that further SNMP based polling does not occur... would make no sense. That said; I totally understand if you want me to fix/add/improve things before considering it further - simply communicate what's necessary and I'll do that. So, for anyone else on the list, who wants the feature; now is your best chance to add your voice and explain why this kind of feature is of use to you. Because you're probably not going to get it otherwise. We've been using this internally since early 2017... I just got around to updating Observium to the latest and greatest (reasons, terrible terrible reasons...), and had to update my patch. But, I'd really prefer not to have to maintain a patch, and I definitely don't want to have to publish it so others can use it and cause Observium Ltd further dramas because of it. How many hosts do we monitor ping only? Six. A grand total of six. Everything else has SNMP in some form; and we want as much info as we can extract from them via it - which is why we love Observium. Nothing else compares for SNMP based monitoring IMHO. Most of those ping only devices are next hop service provider devices just outside our network, for which they won't give us any kind of SNMP access. Ping'ing them offers a small (very small!) degree of insight into link latency/loss from us to them (manual comparison I must admit), and/or potential device problems for the thing that's currently responding for that IP. Hey look... a perfect use case for this feature... The others are devices that we use but which simply don't offer SNMP because vendor X didn't bother to add net-snmp to their standard build for CentOS/Ubuntu/whatever weird and wonderful Linux distro they built them on; when they really should have. Don't care much just need to know they're on the network in some way and want a long track record of response time to them. Hey look... a perfect use case for this feature... Not surprisingly we don't want to have to operate and maintain a separate system for six IP's. Alerting can be flappy because Observium/fping is only sending a single ICMP echo - we're OK with that and don't even really want more pings which would lead to average values anyway - and we've worked around flaps with delays and automatic suppression of duplicate alerts in our alert management and escalation tool. The six are monitored by IP; the DNS hostname test below was simply to double check there was no issue with this and a DNS name. Just a hostname that I've never managed to forget and use a lot to ping test connectivity out. Also, I just realised that the "Skip ping" option is no longer automatically hidden in device settings when "Skip SNMP" is checked; I'll add an updated diff in JIRA shortly after I look at the JS for that again. <Capture.PNG> <observium_ping_only_screens3.PNG> <observium_ping_only_screens4.png> <observium_ping_only_screens5.PNG> On Sun, 3 Mar 2019 at 18:07, Mike Stupalov <mike@observium.org [mailto:mike@observium.org]> wrote: Ohh my dear..
Which really benefit to use Observium for ping_only hosts?
Pls show (screenshots) how this devices displayed on your install.
Colin Stubbs via observium wrote on 03/03/2019 07:43:
For the benefit of anyone on the list who doesn't use JIRA... and also so that others who support (want/need) this feature can comment. MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be https://jira.observium.org/browse/OBS-2925 [https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjira.obser...] Attached patch defines OBS_SNMP_SKIP flag and uses snmp_skip device attribute, similar to OBS_PING_SKIP and ping_skip, in order to have hosts that are ping only. Ping only hosts can still have the Observium Unix Agent installed (tested), and other poller modules such as IPMI enabled (untested). Tested: * Add/remove/rename ping only hosts via CLI * Add/remove ping only hosts via webUI * View/interact ping only hosts via webUI - SNMP specific features/menus etc are hidden while skip SNMP is enabled * Alerting - device_status equals 0 && device_status_type equals ping - will trigger alerts for host down/recovery events * Shifting a previously SNMP contactable host to ping only by ticking skip SNMP box - old SNMP graphs/etc are maintained and remain available - remove skip SNMP box and SNMP polling begins again Things I know kind of don't work right now: * Location override - poller/discovery doesn't seem to perform geocoding and whatever else is happening there Things that could be improved: * Unix Agent poller module etc is enabled by default for all hosts, for ping only hosts, perhaps it should be disabled by default? Will improve performance by reducing the number of processes that hang while the 10s default connect timeout happens. Totally untested: * Use of autodiscovery SNMP skip - should work in theory, unsure if those parts of the patch should actually be used though. Some people out there may actually want to add anything that does respond to ping and can be found thru adjacency and routing protocol info etc?? Patch generated from recent trunk, touches files as below, [root@desktop observium]# diff -r -u observium-trunk root | grep -v ^Only > ping_only_hosts.diff [root@desktop observium]# cd root [root@desktop root]# svn status M add_device.php M html/pages/addhost.inc.php M html/pages/device/edit/device.inc.php M html/pages/device/edit.inc.php M html/pages/device/graphs.inc.php M html/pages/device/perf.inc.php M includes/config-variables.inc.php M includes/defaults.inc.php M includes/definitions.inc.php M includes/discovery/functions.inc.php M includes/functions.inc.php M includes/polling/functions.inc.php M poller.php M rename_device.php [root@desktop root]# svn info | grep ^Revision Revision: 9704 [root@desktop root]# [root@desktop observium]# mv ping_only_hosts.diff ping_only_hosts_r9704.diff
-Colin
Email: cstubbs @ gmail . com _______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.mem...] -- Mike Stupalov Observium Limited, MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be http://observium.org [https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fobservium.o...] _______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.mem... [https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.mem...]
Yes a single fping can be run with multiple targets like this: fping <target1> <target2> <target3> And there is a flag to get the output machine-readable to be easy to parse. /Markus
Den sön 3 mars 2019 kl 18:09 skrev Adam Armstrong via observium < observium@observium.org>:
My usual answer to this is that this causes too much weirdness in the overall system to have "devices" which have zero information attached to them, so "just ping" should probably be a separate thing, perhaps done by a separate poller process. Just a list of hosts to ping.
This probably isn't very difficult to implement, and may be useful for other things (pinging external things to monitor availability a la IP SLA, but without an external device. I'm not sure how good a job we would do of this though. I think fping may help. Just a single fping run should supply the data for all pinged devices, right?
One of the problems with donated code is that it invariably gets abandoned by its original creators and then leaves us to maintain it to handle a 0.1% corner case. This has happened to us more than once in the past, and is now why we're resistant to implementing odd things that impact other parts of code and don't seem to have wide utility.
adam.
On 2019-03-03 16:52:44, Ron Marosko via observium observium@observium.org wrote:
I’ll add my $0.02. In my implementation, we monitor 790 devices, and all of them have SNMP on them. There are a grand total of TWO devices that I’d like to add as ping-only because I don’t have an SNMP string on them… our two uplink internet providers.
Right now we rely on the BGP idle notification to tell us one of the internet uplinks is down. I’ve also built IP SLA on each of our border routers doing ICMP to the next hop IP on the internet side, and of course OBS will graph the results of the IP SLA… and I’m sure there’s probably a way to do an alert notice if the SLA shows a problem, but not all routers are going to have IP SLA capability.
So… to a limited extent, I’d **like** to see the ability to add ping-only devices, but I’m not crippled by not having the capability.
…Ron
--
*Ron Marosko, Jr.*
. . . . . . . . . . . . . . . . . . . . . . . . . .
CCIE No. 4526 (R/S) Emeritus, NN5DX, RM1517-ARIN
Senior Network Engineer
*DirectBytes, LLC.*
1108A W. Dickinson Blvd.
Fort Stockton, TX 79735 USA
*o:* *+1 432 336 5600 x115*
*c:* +1 432 290 6344
*e:** ron@directbytes.net ron@directbytes.net*
*pgp pubkey:* *0x58AB8B5C http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x778B3B0258AB8B5C*
*"To know me is to fly with me."*
*From:* observium observium-bounces@observium.org *On Behalf Of *Ryan Huff via observium *Sent:* Sunday, March 3, 2019 7:44 AM *To:* cstubbs@gmail.com *Cc:* Ryan Huff ryanhuff@outlook.com; Observium <observium@observium.org
*Subject:* Re: [Observium] OBS-2925 - Ping Only Hosts / OBS_SNMP_SKIP flag / snmp_skip attribute
Naaa, mine is a big use case/implementation; it’s just 3 out of a bagillion devices that this needs done for, its not my personal email, it’s a distro. We could easily do an JSON post to any manner of transports, but “mail();” was so much simpler.
All we need to know is when it’s down so someone can go kick the hamster wheel, that’s “ping only”, monitoring to me.
If you’re looking for graphs / SLA type metrics too, then I’d argue you’re already looking for more than “ping only” monitoring....
I see what you’re saying though; OBS has such a great altogether GUI experience, it’d be nice!
-Ryan
On Mar 3, 2019, at 08:15, Colin Stubbs cstubbs@gmail.com wrote:
I know where you're at... but,
- No refactoring required - this is part of the point of the patch ( I
would not have even attempted this if it wasn't so easy... ) and my long writeup for Mike given he hadn't actually looked at JIRA yet. Uses existing code/architecture - bit flag and device attributes - to mark a device as "don't do any SNMP to this device right now"
Observium uses fping...
Smokeping is horrible - can provide great value sure, but horrible to
use, and again, extra tool. I'm a long way from unfamiliar with it and have both used it many years ago, and considered it for our current environments. Not useful to us.
- Smokeping is not integrated to Observium if the device is not monitored
by SNMP - this is fine when it's just you or a small team of people who understand this and are happy to flick between interfaces and how to work with Smokeping. Not great when you have a larger team and you really just need one interface on a monitor in your operations room that tells everyone everything that's up/down/problematic right now. Particularly not fine when your level 1's can't really login on CLI and edit config files etc.
- IP SLA's - vendor specific thing. I love them in the right environment.
Fantastic tool and capability to have in a network. We have zero Cisco in this situation and don't want to add a device specifically for this.
- Your description sounds like it's just you in this environment... and
if email fails, or you're hit by a bus, no one else knows there's a problem let alone how your alerting solution works and how to maintain it. Also, manual investigation/access to history. By having Observium do the ping we get to leverage the alerting capabilities inherent to it, e.g. email/webhook/PagerDuty/OpsGenie/VictorOps etc etc etc., as well as the webUI for visual alert notification and access to graphs/history.
It's actually a simpler and cleaner solution; unless you value seeing the min/max/average/jitter values that IP SLA or Smokeping can help you see. In which case it's not what you need.
-Colin
Email: cstubbs @ gmail . com
On Sun, 3 Mar 2019 at 22:44, Ryan Huff ryanhuff@outlook.com wrote:
I really do understand this Colin, on a deeper level than you might imagine ;). I have no vested interest either way personally, I’m just offering some unsolicited thoughts...
I always looked at this specific topic like asking a 10lbs. (4.5kg or 0.7149 stones) sledge hammer to drive a nail. Sure, it can do it but it’ll probably be messy and ugly because a sledge hammer was never meant to drive nails, and when you use tools in ways they weren’t really designed for, things can get messy. Just use a hammer instead.
I have done three things to achieve icmp only monitoring and also stick within the base code (as a developer, I hate mucking with base code or re-inventing wheels when I want to do different things). Icmp-only monitoring to me, always just seemed silly (not in purpose) to try and work into an existing and established application that didn’t already do it.
It’s just too easy of a problem to solve otherwise, than to go to the time/expense of refactoring an application IMHO.
- I use RTT SLAs to monitor next hop devices (which OBS is fabulous at
picking up), assuming the device hopped from is SLA capable and OBS monitored.
- I use Fping (Smokeping) integrated with OBS. Using the Smokeping web
wrapper makes a great RTT only tool, and you also have the OBS integration for when it makes sense to use on OBS devices.
- I have a small (very small) bash script that reads a text file of IP
address and launches one thread per device to run continuous pings. Not a very scalable script, but like you, I only need it for a very few limited cases. The script uses sendmail to email me if it can’t reach something.
Thanks,
Ryan
On Mar 3, 2019, at 06:35, Colin Stubbs via observium < observium@observium.org> wrote:
Hi Mike,
They're in the JIRA ticket, but sure, here's some inserted to this email.
I'm aware of Adam/your preference to reject or simply ignore the regular requests for this kind of feature, and the confusion it causes new users considering Observium for use in their organisations when they find out they can't do the most basic of monitoring to something using Observium.
A great many of your users would gain a lot of value from being able to add a few ping only hosts.
Why build, manage and integrate (to other tools) another ping based monitoring system when all you need is to monitor a handful of devices with ping only; simply because those devices fail to offer SNMP for some reason?
Anything that does have a working SNMP service can and should be monitored with SNMP to gain greater insight into what's happening on it... but not everything does.
No one in their right mind should attempt to use Observium as a system which monitors massive numbers of DNS names or IP's with ping only, there are better systems out there for that, but there are so many situations in a which an Observium install can and should be able to monitor some devices ping only.
I've associated the ticket with the two previous JIRA requests for this feature; which Adam rejected.
They were just feature requests as I recall; no code proposed. Totally understandable you guys didn't want to work on the feature when it's not really what you feel Observium is for.
Rejecting this now when you have just been gifted working code, that modifies things fairly minimally, and simply leverages the ping information that's already occurring to each and every device (that ping is not disabled for), and uses an existing method to mark devices so that further SNMP based polling does not occur... would make no sense.
That said; I totally understand if you want me to fix/add/improve things before considering it further - simply communicate what's necessary and I'll do that.
So, for anyone else on the list, who wants the feature; now is your best chance to add your voice and explain why this kind of feature is of use to you. Because you're probably not going to get it otherwise.
We've been using this internally since early 2017... I just got around to updating Observium to the latest and greatest (reasons, terrible terrible reasons...), and had to update my patch.
But, I'd really prefer not to have to maintain a patch, and I definitely don't want to have to publish it so others can use it and cause Observium Ltd further dramas because of it.
How many hosts do we monitor ping only?
Six. A grand total of six. Everything else has SNMP in some form; and we want as much info as we can extract from them via it - which is why we love Observium. Nothing else compares for SNMP based monitoring IMHO.
Most of those ping only devices are next hop service provider devices just outside our network, for which they won't give us any kind of SNMP access. Ping'ing them offers a small (very small!) degree of insight into link latency/loss from us to them (manual comparison I must admit), and/or potential device problems for the thing that's currently responding for that IP.
Hey look... a perfect use case for this feature...
The others are devices that we use but which simply don't offer SNMP because vendor X didn't bother to add net-snmp to their standard build for CentOS/Ubuntu/whatever weird and wonderful Linux distro they built them on; when they really should have. Don't care much just need to know they're on the network in some way and want a long track record of response time to them.
Hey look... a perfect use case for this feature...
Not surprisingly we don't want to have to operate and maintain a separate system for six IP's. Alerting can be flappy because Observium/fping is only sending a single ICMP echo - we're OK with that and don't even really want more pings which would lead to average values anyway - and we've worked around flaps with delays and automatic suppression of duplicate alerts in our alert management and escalation tool.
The six are monitored by IP; the DNS hostname test below was simply to double check there was no issue with this and a DNS name. Just a hostname that I've never managed to forget and use a lot to ping test connectivity out.
Also, I just realised that the "Skip ping" option is no longer automatically hidden in device settings when "Skip SNMP" is checked; I'll add an updated diff in JIRA shortly after I look at the JS for that again.
<Capture.PNG>
<observium_ping_only_screens3.PNG>
<observium_ping_only_screens4.png>
<observium_ping_only_screens5.PNG>
On Sun, 3 Mar 2019 at 18:07, Mike Stupalov mike@observium.org wrote:
Ohh my dear..
Which really benefit to use Observium for ping_only hosts?
Pls show (screenshots) how this devices displayed on your install.
Colin Stubbs via observium wrote on 03/03/2019 07:43:
For the benefit of anyone on the list who doesn't use JIRA... and also so that others who support (want/need) this feature can comment.
*MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* https://jira.observium.org/browse/OBS-2925 https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjira.observium.org%2Fbrowse%2FOBS-2925&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503009597&sdata=J5gbdkhaj1DVqk7F0gvDAk%2B6S25XfYziaa7xh8v5%2Bgc%3D&reserved=0
Attached patch defines OBS_SNMP_SKIP flag and uses snmp_skip device attribute, similar to OBS_PING_SKIP and ping_skip, in order to have hosts that are ping only.
Ping only hosts can still have the Observium Unix Agent installed (tested), and other poller modules such as IPMI enabled (untested).
Tested:
- Add/remove/rename ping only hosts via CLI
- Add/remove ping only hosts via webUI
- View/interact ping only hosts via webUI - SNMP specific
features/menus etc are hidden while skip SNMP is enabled 4. Alerting - device_status equals 0 && device_status_type equals ping
- will trigger alerts for host down/recovery events
- Shifting a previously SNMP contactable host to ping only by ticking
skip SNMP box - old SNMP graphs/etc are maintained and remain available - remove skip SNMP box and SNMP polling begins again
Things I know kind of don't work right now:
- Location override - poller/discovery doesn't seem to perform
geocoding and whatever else is happening there
Things that could be improved:
- Unix Agent poller module etc is enabled by default for all hosts,
for ping only hosts, perhaps it should be disabled by default? Will improve performance by reducing the number of processes that hang while the 10s default connect timeout happens.
Totally untested:
- Use of autodiscovery SNMP skip - should work in theory, unsure if
those parts of the patch should actually be used though. Some people out there may actually want to add anything that does respond to ping and can be found thru adjacency and routing protocol info etc??
Patch generated from recent trunk, touches files as below,
[root@desktop observium]# diff -r -u observium-trunk root | grep -v ^Only
ping_only_hosts.diff
[root@desktop observium]# cd root [root@desktop root]# svn status M add_device.php M html/pages/addhost.inc.php M html/pages/device/edit/device.inc.php M html/pages/device/edit.inc.php M html/pages/device/graphs.inc.php M html/pages/device/perf.inc.php M includes/config-variables.inc.php M includes/defaults.inc.php M includes/definitions.inc.php M includes/discovery/functions.inc.php M includes/functions.inc.php M includes/polling/functions.inc.php M poller.php M rename_device.php [root@desktop root]# svn info | grep ^Revision Revision: 9704 [root@desktop root]# [root@desktop observium]# mv ping_only_hosts.diff ping_only_hosts_r9704.diff
-Colin
Email: cstubbs @ gmail . com
observium mailing list
observium@observium.org
*MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* http://postman.memetic.org/cgi-bin/mailman/listinfo/observium https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.memetic.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fobservium&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503019596&sdata=E967lJ4buMEKYKMMawVJYnlaC6MYm2evhD8JZDqESXs%3D&reserved=0
-- Mike Stupalov Observium Limited, *MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* http://observium.org https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fobservium.org&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503029613&sdata=bclwJLX8WOiFp4nyfLA2PxXvl40er5OPquD%2FA3XsY3Q%3D&reserved=0
observium mailing list observium@observium.org *MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.mem... https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.memetic.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fobservium&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503039618&sdata=DH9UrWMzx08fgOTm%2FH4FgeOtYBt4P2K%2BzNIiqtqxIPs%3D&reserved=0
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
I think the patch is great btw, it adds a feature a few people will have use of in a logical way without much hustle. Merge it for now and start looking at creating a more complete IP SLA-funktion. Ping-only checks probably can and should be run more often than every 5min and it would be nice for uplink/ISP-monitoring and stuff like that.
/Markus
Den sön 3 mars 2019 kl 17:33 skrev Markus Klock markus@best-practice.se:
Yes a single fping can be run with multiple targets like this: fping <target1> <target2> <target3> And there is a flag to get the output machine-readable to be easy to parse. /Markus
Den sön 3 mars 2019 kl 18:09 skrev Adam Armstrong via observium < observium@observium.org>:
My usual answer to this is that this causes too much weirdness in the overall system to have "devices" which have zero information attached to them, so "just ping" should probably be a separate thing, perhaps done by a separate poller process. Just a list of hosts to ping.
This probably isn't very difficult to implement, and may be useful for other things (pinging external things to monitor availability a la IP SLA, but without an external device. I'm not sure how good a job we would do of this though. I think fping may help. Just a single fping run should supply the data for all pinged devices, right?
One of the problems with donated code is that it invariably gets abandoned by its original creators and then leaves us to maintain it to handle a 0.1% corner case. This has happened to us more than once in the past, and is now why we're resistant to implementing odd things that impact other parts of code and don't seem to have wide utility.
adam.
On 2019-03-03 16:52:44, Ron Marosko via observium < observium@observium.org> wrote:
I’ll add my $0.02. In my implementation, we monitor 790 devices, and all of them have SNMP on them. There are a grand total of TWO devices that I’d like to add as ping-only because I don’t have an SNMP string on them… our two uplink internet providers.
Right now we rely on the BGP idle notification to tell us one of the internet uplinks is down. I’ve also built IP SLA on each of our border routers doing ICMP to the next hop IP on the internet side, and of course OBS will graph the results of the IP SLA… and I’m sure there’s probably a way to do an alert notice if the SLA shows a problem, but not all routers are going to have IP SLA capability.
So… to a limited extent, I’d **like** to see the ability to add ping-only devices, but I’m not crippled by not having the capability.
…Ron
--
*Ron Marosko, Jr.*
. . . . . . . . . . . . . . . . . . . . . . . . . .
CCIE No. 4526 (R/S) Emeritus, NN5DX, RM1517-ARIN
Senior Network Engineer
*DirectBytes, LLC.*
1108A W. Dickinson Blvd.
Fort Stockton, TX 79735 USA
*o:* *+1 432 336 5600 x115*
*c:* +1 432 290 6344
*e:** ron@directbytes.net ron@directbytes.net*
*pgp pubkey:* *0x58AB8B5C http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x778B3B0258AB8B5C*
*"To know me is to fly with me."*
*From:* observium observium-bounces@observium.org *On Behalf Of *Ryan Huff via observium *Sent:* Sunday, March 3, 2019 7:44 AM *To:* cstubbs@gmail.com *Cc:* Ryan Huff ryanhuff@outlook.com; Observium < observium@observium.org> *Subject:* Re: [Observium] OBS-2925 - Ping Only Hosts / OBS_SNMP_SKIP flag / snmp_skip attribute
Naaa, mine is a big use case/implementation; it’s just 3 out of a bagillion devices that this needs done for, its not my personal email, it’s a distro. We could easily do an JSON post to any manner of transports, but “mail();” was so much simpler.
All we need to know is when it’s down so someone can go kick the hamster wheel, that’s “ping only”, monitoring to me.
If you’re looking for graphs / SLA type metrics too, then I’d argue you’re already looking for more than “ping only” monitoring....
I see what you’re saying though; OBS has such a great altogether GUI experience, it’d be nice!
-Ryan
On Mar 3, 2019, at 08:15, Colin Stubbs cstubbs@gmail.com wrote:
I know where you're at... but,
- No refactoring required - this is part of the point of the patch ( I
would not have even attempted this if it wasn't so easy... ) and my long writeup for Mike given he hadn't actually looked at JIRA yet. Uses existing code/architecture - bit flag and device attributes - to mark a device as "don't do any SNMP to this device right now"
Observium uses fping...
Smokeping is horrible - can provide great value sure, but horrible to
use, and again, extra tool. I'm a long way from unfamiliar with it and have both used it many years ago, and considered it for our current environments. Not useful to us.
- Smokeping is not integrated to Observium if the device is not
monitored by SNMP - this is fine when it's just you or a small team of people who understand this and are happy to flick between interfaces and how to work with Smokeping. Not great when you have a larger team and you really just need one interface on a monitor in your operations room that tells everyone everything that's up/down/problematic right now. Particularly not fine when your level 1's can't really login on CLI and edit config files etc.
- IP SLA's - vendor specific thing. I love them in the right
environment. Fantastic tool and capability to have in a network. We have zero Cisco in this situation and don't want to add a device specifically for this.
- Your description sounds like it's just you in this environment... and
if email fails, or you're hit by a bus, no one else knows there's a problem let alone how your alerting solution works and how to maintain it. Also, manual investigation/access to history. By having Observium do the ping we get to leverage the alerting capabilities inherent to it, e.g. email/webhook/PagerDuty/OpsGenie/VictorOps etc etc etc., as well as the webUI for visual alert notification and access to graphs/history.
It's actually a simpler and cleaner solution; unless you value seeing the min/max/average/jitter values that IP SLA or Smokeping can help you see. In which case it's not what you need.
-Colin
Email: cstubbs @ gmail . com
On Sun, 3 Mar 2019 at 22:44, Ryan Huff ryanhuff@outlook.com wrote:
I really do understand this Colin, on a deeper level than you might imagine ;). I have no vested interest either way personally, I’m just offering some unsolicited thoughts...
I always looked at this specific topic like asking a 10lbs. (4.5kg or 0.7149 stones) sledge hammer to drive a nail. Sure, it can do it but it’ll probably be messy and ugly because a sledge hammer was never meant to drive nails, and when you use tools in ways they weren’t really designed for, things can get messy. Just use a hammer instead.
I have done three things to achieve icmp only monitoring and also stick within the base code (as a developer, I hate mucking with base code or re-inventing wheels when I want to do different things). Icmp-only monitoring to me, always just seemed silly (not in purpose) to try and work into an existing and established application that didn’t already do it.
It’s just too easy of a problem to solve otherwise, than to go to the time/expense of refactoring an application IMHO.
- I use RTT SLAs to monitor next hop devices (which OBS is fabulous at
picking up), assuming the device hopped from is SLA capable and OBS monitored.
- I use Fping (Smokeping) integrated with OBS. Using the Smokeping web
wrapper makes a great RTT only tool, and you also have the OBS integration for when it makes sense to use on OBS devices.
- I have a small (very small) bash script that reads a text file of IP
address and launches one thread per device to run continuous pings. Not a very scalable script, but like you, I only need it for a very few limited cases. The script uses sendmail to email me if it can’t reach something.
Thanks,
Ryan
On Mar 3, 2019, at 06:35, Colin Stubbs via observium < observium@observium.org> wrote:
Hi Mike,
They're in the JIRA ticket, but sure, here's some inserted to this email.
I'm aware of Adam/your preference to reject or simply ignore the regular requests for this kind of feature, and the confusion it causes new users considering Observium for use in their organisations when they find out they can't do the most basic of monitoring to something using Observium.
A great many of your users would gain a lot of value from being able to add a few ping only hosts.
Why build, manage and integrate (to other tools) another ping based monitoring system when all you need is to monitor a handful of devices with ping only; simply because those devices fail to offer SNMP for some reason?
Anything that does have a working SNMP service can and should be monitored with SNMP to gain greater insight into what's happening on it... but not everything does.
No one in their right mind should attempt to use Observium as a system which monitors massive numbers of DNS names or IP's with ping only, there are better systems out there for that, but there are so many situations in a which an Observium install can and should be able to monitor some devices ping only.
I've associated the ticket with the two previous JIRA requests for this feature; which Adam rejected.
They were just feature requests as I recall; no code proposed. Totally understandable you guys didn't want to work on the feature when it's not really what you feel Observium is for.
Rejecting this now when you have just been gifted working code, that modifies things fairly minimally, and simply leverages the ping information that's already occurring to each and every device (that ping is not disabled for), and uses an existing method to mark devices so that further SNMP based polling does not occur... would make no sense.
That said; I totally understand if you want me to fix/add/improve things before considering it further - simply communicate what's necessary and I'll do that.
So, for anyone else on the list, who wants the feature; now is your best chance to add your voice and explain why this kind of feature is of use to you. Because you're probably not going to get it otherwise.
We've been using this internally since early 2017... I just got around to updating Observium to the latest and greatest (reasons, terrible terrible reasons...), and had to update my patch.
But, I'd really prefer not to have to maintain a patch, and I definitely don't want to have to publish it so others can use it and cause Observium Ltd further dramas because of it.
How many hosts do we monitor ping only?
Six. A grand total of six. Everything else has SNMP in some form; and we want as much info as we can extract from them via it - which is why we love Observium. Nothing else compares for SNMP based monitoring IMHO.
Most of those ping only devices are next hop service provider devices just outside our network, for which they won't give us any kind of SNMP access. Ping'ing them offers a small (very small!) degree of insight into link latency/loss from us to them (manual comparison I must admit), and/or potential device problems for the thing that's currently responding for that IP.
Hey look... a perfect use case for this feature...
The others are devices that we use but which simply don't offer SNMP because vendor X didn't bother to add net-snmp to their standard build for CentOS/Ubuntu/whatever weird and wonderful Linux distro they built them on; when they really should have. Don't care much just need to know they're on the network in some way and want a long track record of response time to them.
Hey look... a perfect use case for this feature...
Not surprisingly we don't want to have to operate and maintain a separate system for six IP's. Alerting can be flappy because Observium/fping is only sending a single ICMP echo - we're OK with that and don't even really want more pings which would lead to average values anyway - and we've worked around flaps with delays and automatic suppression of duplicate alerts in our alert management and escalation tool.
The six are monitored by IP; the DNS hostname test below was simply to double check there was no issue with this and a DNS name. Just a hostname that I've never managed to forget and use a lot to ping test connectivity out.
Also, I just realised that the "Skip ping" option is no longer automatically hidden in device settings when "Skip SNMP" is checked; I'll add an updated diff in JIRA shortly after I look at the JS for that again.
<Capture.PNG>
<observium_ping_only_screens3.PNG>
<observium_ping_only_screens4.png>
<observium_ping_only_screens5.PNG>
On Sun, 3 Mar 2019 at 18:07, Mike Stupalov mike@observium.org wrote:
Ohh my dear..
Which really benefit to use Observium for ping_only hosts?
Pls show (screenshots) how this devices displayed on your install.
Colin Stubbs via observium wrote on 03/03/2019 07:43:
For the benefit of anyone on the list who doesn't use JIRA... and also so that others who support (want/need) this feature can comment.
*MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* https://jira.observium.org/browse/OBS-2925 https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjira.observium.org%2Fbrowse%2FOBS-2925&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503009597&sdata=J5gbdkhaj1DVqk7F0gvDAk%2B6S25XfYziaa7xh8v5%2Bgc%3D&reserved=0
Attached patch defines OBS_SNMP_SKIP flag and uses snmp_skip device attribute, similar to OBS_PING_SKIP and ping_skip, in order to have hosts that are ping only.
Ping only hosts can still have the Observium Unix Agent installed (tested), and other poller modules such as IPMI enabled (untested).
Tested:
- Add/remove/rename ping only hosts via CLI
- Add/remove ping only hosts via webUI
- View/interact ping only hosts via webUI - SNMP specific
features/menus etc are hidden while skip SNMP is enabled 4. Alerting - device_status equals 0 && device_status_type equals ping - will trigger alerts for host down/recovery events 5. Shifting a previously SNMP contactable host to ping only by ticking skip SNMP box - old SNMP graphs/etc are maintained and remain available - remove skip SNMP box and SNMP polling begins again
Things I know kind of don't work right now:
- Location override - poller/discovery doesn't seem to perform
geocoding and whatever else is happening there
Things that could be improved:
- Unix Agent poller module etc is enabled by default for all hosts,
for ping only hosts, perhaps it should be disabled by default? Will improve performance by reducing the number of processes that hang while the 10s default connect timeout happens.
Totally untested:
- Use of autodiscovery SNMP skip - should work in theory, unsure if
those parts of the patch should actually be used though. Some people out there may actually want to add anything that does respond to ping and can be found thru adjacency and routing protocol info etc??
Patch generated from recent trunk, touches files as below,
[root@desktop observium]# diff -r -u observium-trunk root | grep -v ^Only > ping_only_hosts.diff [root@desktop observium]# cd root [root@desktop root]# svn status M add_device.php M html/pages/addhost.inc.php M html/pages/device/edit/device.inc.php M html/pages/device/edit.inc.php M html/pages/device/graphs.inc.php M html/pages/device/perf.inc.php M includes/config-variables.inc.php M includes/defaults.inc.php M includes/definitions.inc.php M includes/discovery/functions.inc.php M includes/functions.inc.php M includes/polling/functions.inc.php M poller.php M rename_device.php [root@desktop root]# svn info | grep ^Revision Revision: 9704 [root@desktop root]# [root@desktop observium]# mv ping_only_hosts.diff ping_only_hosts_r9704.diff
-Colin
Email: cstubbs @ gmail . com
observium mailing list
observium@observium.org
*MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* http://postman.memetic.org/cgi-bin/mailman/listinfo/observium https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.memetic.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fobservium&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503019596&sdata=E967lJ4buMEKYKMMawVJYnlaC6MYm2evhD8JZDqESXs%3D&reserved=0
-- Mike Stupalov Observium Limited, *MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* http://observium.org https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fobservium.org&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503029613&sdata=bclwJLX8WOiFp4nyfLA2PxXvl40er5OPquD%2FA3XsY3Q%3D&reserved=0
observium mailing list observium@observium.org *MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.mem... https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.memetic.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fobservium&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503039618&sdata=DH9UrWMzx08fgOTm%2FH4FgeOtYBt4P2K%2BzNIiqtqxIPs%3D&reserved=0
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Around 8% of our assets are ping-only monitoring in our manufacturing environment, where the vendors either won't provide SNMP characteristics, or the SNMP stack is so completely off kilter and pointless that to implement it would not be worth the time, especially given that we just want to know its there and powered up.
Currently managing this behind a second non-observium monitoring instance, but would really rather centralize behind Observium. Colin's contribution is most interesting...
Philippe
On Mon, Mar 4, 2019 at 2:08 AM Markus Klock via observium < observium@observium.org> wrote:
I think the patch is great btw, it adds a feature a few people will have use of in a logical way without much hustle. Merge it for now and start looking at creating a more complete IP SLA-funktion. Ping-only checks probably can and should be run more often than every 5min and it would be nice for uplink/ISP-monitoring and stuff like that.
/Markus
Den sön 3 mars 2019 kl 17:33 skrev Markus Klock markus@best-practice.se:
Yes a single fping can be run with multiple targets like this: fping <target1> <target2> <target3> And there is a flag to get the output machine-readable to be easy to parse. /Markus
Den sön 3 mars 2019 kl 18:09 skrev Adam Armstrong via observium < observium@observium.org>:
My usual answer to this is that this causes too much weirdness in the overall system to have "devices" which have zero information attached to them, so "just ping" should probably be a separate thing, perhaps done by a separate poller process. Just a list of hosts to ping.
This probably isn't very difficult to implement, and may be useful for other things (pinging external things to monitor availability a la IP SLA, but without an external device. I'm not sure how good a job we would do of this though. I think fping may help. Just a single fping run should supply the data for all pinged devices, right?
One of the problems with donated code is that it invariably gets abandoned by its original creators and then leaves us to maintain it to handle a 0.1% corner case. This has happened to us more than once in the past, and is now why we're resistant to implementing odd things that impact other parts of code and don't seem to have wide utility.
adam.
On 2019-03-03 16:52:44, Ron Marosko via observium < observium@observium.org> wrote:
I’ll add my $0.02. In my implementation, we monitor 790 devices, and all of them have SNMP on them. There are a grand total of TWO devices that I’d like to add as ping-only because I don’t have an SNMP string on them… our two uplink internet providers.
Right now we rely on the BGP idle notification to tell us one of the internet uplinks is down. I’ve also built IP SLA on each of our border routers doing ICMP to the next hop IP on the internet side, and of course OBS will graph the results of the IP SLA… and I’m sure there’s probably a way to do an alert notice if the SLA shows a problem, but not all routers are going to have IP SLA capability.
So… to a limited extent, I’d **like** to see the ability to add ping-only devices, but I’m not crippled by not having the capability.
…Ron
--
*Ron Marosko, Jr.*
. . . . . . . . . . . . . . . . . . . . . . . . . .
CCIE No. 4526 (R/S) Emeritus, NN5DX, RM1517-ARIN
Senior Network Engineer
*DirectBytes, LLC.*
1108A W. Dickinson Blvd.
Fort Stockton, TX 79735 USA
*o:* *+1 432 336 5600 x115*
*c:* +1 432 290 6344
*e:** ron@directbytes.net ron@directbytes.net*
*pgp pubkey:* *0x58AB8B5C http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x778B3B0258AB8B5C*
*"To know me is to fly with me."*
*From:* observium observium-bounces@observium.org *On Behalf Of *Ryan Huff via observium *Sent:* Sunday, March 3, 2019 7:44 AM *To:* cstubbs@gmail.com *Cc:* Ryan Huff ryanhuff@outlook.com; Observium < observium@observium.org> *Subject:* Re: [Observium] OBS-2925 - Ping Only Hosts / OBS_SNMP_SKIP flag / snmp_skip attribute
Naaa, mine is a big use case/implementation; it’s just 3 out of a bagillion devices that this needs done for, its not my personal email, it’s a distro. We could easily do an JSON post to any manner of transports, but “mail();” was so much simpler.
All we need to know is when it’s down so someone can go kick the hamster wheel, that’s “ping only”, monitoring to me.
If you’re looking for graphs / SLA type metrics too, then I’d argue you’re already looking for more than “ping only” monitoring....
I see what you’re saying though; OBS has such a great altogether GUI experience, it’d be nice!
-Ryan
On Mar 3, 2019, at 08:15, Colin Stubbs cstubbs@gmail.com wrote:
I know where you're at... but,
- No refactoring required - this is part of the point of the patch ( I
would not have even attempted this if it wasn't so easy... ) and my long writeup for Mike given he hadn't actually looked at JIRA yet. Uses existing code/architecture - bit flag and device attributes - to mark a device as "don't do any SNMP to this device right now"
Observium uses fping...
Smokeping is horrible - can provide great value sure, but horrible to
use, and again, extra tool. I'm a long way from unfamiliar with it and have both used it many years ago, and considered it for our current environments. Not useful to us.
- Smokeping is not integrated to Observium if the device is not
monitored by SNMP - this is fine when it's just you or a small team of people who understand this and are happy to flick between interfaces and how to work with Smokeping. Not great when you have a larger team and you really just need one interface on a monitor in your operations room that tells everyone everything that's up/down/problematic right now. Particularly not fine when your level 1's can't really login on CLI and edit config files etc.
- IP SLA's - vendor specific thing. I love them in the right
environment. Fantastic tool and capability to have in a network. We have zero Cisco in this situation and don't want to add a device specifically for this.
- Your description sounds like it's just you in this environment... and
if email fails, or you're hit by a bus, no one else knows there's a problem let alone how your alerting solution works and how to maintain it. Also, manual investigation/access to history. By having Observium do the ping we get to leverage the alerting capabilities inherent to it, e.g. email/webhook/PagerDuty/OpsGenie/VictorOps etc etc etc., as well as the webUI for visual alert notification and access to graphs/history.
It's actually a simpler and cleaner solution; unless you value seeing the min/max/average/jitter values that IP SLA or Smokeping can help you see. In which case it's not what you need.
-Colin
Email: cstubbs @ gmail . com
On Sun, 3 Mar 2019 at 22:44, Ryan Huff ryanhuff@outlook.com wrote:
I really do understand this Colin, on a deeper level than you might imagine ;). I have no vested interest either way personally, I’m just offering some unsolicited thoughts...
I always looked at this specific topic like asking a 10lbs. (4.5kg or 0.7149 stones) sledge hammer to drive a nail. Sure, it can do it but it’ll probably be messy and ugly because a sledge hammer was never meant to drive nails, and when you use tools in ways they weren’t really designed for, things can get messy. Just use a hammer instead.
I have done three things to achieve icmp only monitoring and also stick within the base code (as a developer, I hate mucking with base code or re-inventing wheels when I want to do different things). Icmp-only monitoring to me, always just seemed silly (not in purpose) to try and work into an existing and established application that didn’t already do it.
It’s just too easy of a problem to solve otherwise, than to go to the time/expense of refactoring an application IMHO.
- I use RTT SLAs to monitor next hop devices (which OBS is fabulous at
picking up), assuming the device hopped from is SLA capable and OBS monitored.
- I use Fping (Smokeping) integrated with OBS. Using the Smokeping web
wrapper makes a great RTT only tool, and you also have the OBS integration for when it makes sense to use on OBS devices.
- I have a small (very small) bash script that reads a text file of IP
address and launches one thread per device to run continuous pings. Not a very scalable script, but like you, I only need it for a very few limited cases. The script uses sendmail to email me if it can’t reach something.
Thanks,
Ryan
On Mar 3, 2019, at 06:35, Colin Stubbs via observium < observium@observium.org> wrote:
Hi Mike,
They're in the JIRA ticket, but sure, here's some inserted to this email.
I'm aware of Adam/your preference to reject or simply ignore the regular requests for this kind of feature, and the confusion it causes new users considering Observium for use in their organisations when they find out they can't do the most basic of monitoring to something using Observium.
A great many of your users would gain a lot of value from being able to add a few ping only hosts.
Why build, manage and integrate (to other tools) another ping based monitoring system when all you need is to monitor a handful of devices with ping only; simply because those devices fail to offer SNMP for some reason?
Anything that does have a working SNMP service can and should be monitored with SNMP to gain greater insight into what's happening on it... but not everything does.
No one in their right mind should attempt to use Observium as a system which monitors massive numbers of DNS names or IP's with ping only, there are better systems out there for that, but there are so many situations in a which an Observium install can and should be able to monitor some devices ping only.
I've associated the ticket with the two previous JIRA requests for this feature; which Adam rejected.
They were just feature requests as I recall; no code proposed. Totally understandable you guys didn't want to work on the feature when it's not really what you feel Observium is for.
Rejecting this now when you have just been gifted working code, that modifies things fairly minimally, and simply leverages the ping information that's already occurring to each and every device (that ping is not disabled for), and uses an existing method to mark devices so that further SNMP based polling does not occur... would make no sense.
That said; I totally understand if you want me to fix/add/improve things before considering it further - simply communicate what's necessary and I'll do that.
So, for anyone else on the list, who wants the feature; now is your best chance to add your voice and explain why this kind of feature is of use to you. Because you're probably not going to get it otherwise.
We've been using this internally since early 2017... I just got around to updating Observium to the latest and greatest (reasons, terrible terrible reasons...), and had to update my patch.
But, I'd really prefer not to have to maintain a patch, and I definitely don't want to have to publish it so others can use it and cause Observium Ltd further dramas because of it.
How many hosts do we monitor ping only?
Six. A grand total of six. Everything else has SNMP in some form; and we want as much info as we can extract from them via it - which is why we love Observium. Nothing else compares for SNMP based monitoring IMHO.
Most of those ping only devices are next hop service provider devices just outside our network, for which they won't give us any kind of SNMP access. Ping'ing them offers a small (very small!) degree of insight into link latency/loss from us to them (manual comparison I must admit), and/or potential device problems for the thing that's currently responding for that IP.
Hey look... a perfect use case for this feature...
The others are devices that we use but which simply don't offer SNMP because vendor X didn't bother to add net-snmp to their standard build for CentOS/Ubuntu/whatever weird and wonderful Linux distro they built them on; when they really should have. Don't care much just need to know they're on the network in some way and want a long track record of response time to them.
Hey look... a perfect use case for this feature...
Not surprisingly we don't want to have to operate and maintain a separate system for six IP's. Alerting can be flappy because Observium/fping is only sending a single ICMP echo - we're OK with that and don't even really want more pings which would lead to average values anyway
- and we've worked around flaps with delays and automatic suppression of
duplicate alerts in our alert management and escalation tool.
The six are monitored by IP; the DNS hostname test below was simply to double check there was no issue with this and a DNS name. Just a hostname that I've never managed to forget and use a lot to ping test connectivity out.
Also, I just realised that the "Skip ping" option is no longer automatically hidden in device settings when "Skip SNMP" is checked; I'll add an updated diff in JIRA shortly after I look at the JS for that again.
<Capture.PNG>
<observium_ping_only_screens3.PNG>
<observium_ping_only_screens4.png>
<observium_ping_only_screens5.PNG>
On Sun, 3 Mar 2019 at 18:07, Mike Stupalov mike@observium.org wrote:
Ohh my dear..
Which really benefit to use Observium for ping_only hosts?
Pls show (screenshots) how this devices displayed on your install.
Colin Stubbs via observium wrote on 03/03/2019 07:43:
For the benefit of anyone on the list who doesn't use JIRA... and also so that others who support (want/need) this feature can comment.
*MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* https://jira.observium.org/browse/OBS-2925 https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjira.observium.org%2Fbrowse%2FOBS-2925&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503009597&sdata=J5gbdkhaj1DVqk7F0gvDAk%2B6S25XfYziaa7xh8v5%2Bgc%3D&reserved=0
Attached patch defines OBS_SNMP_SKIP flag and uses snmp_skip device attribute, similar to OBS_PING_SKIP and ping_skip, in order to have hosts that are ping only.
Ping only hosts can still have the Observium Unix Agent installed (tested), and other poller modules such as IPMI enabled (untested).
Tested:
- Add/remove/rename ping only hosts via CLI
- Add/remove ping only hosts via webUI
- View/interact ping only hosts via webUI - SNMP specific
features/menus etc are hidden while skip SNMP is enabled 4. Alerting - device_status equals 0 && device_status_type equals ping - will trigger alerts for host down/recovery events 5. Shifting a previously SNMP contactable host to ping only by ticking skip SNMP box - old SNMP graphs/etc are maintained and remain available - remove skip SNMP box and SNMP polling begins again
Things I know kind of don't work right now:
- Location override - poller/discovery doesn't seem to perform
geocoding and whatever else is happening there
Things that could be improved:
- Unix Agent poller module etc is enabled by default for all hosts,
for ping only hosts, perhaps it should be disabled by default? Will improve performance by reducing the number of processes that hang while the 10s default connect timeout happens.
Totally untested:
- Use of autodiscovery SNMP skip - should work in theory, unsure if
those parts of the patch should actually be used though. Some people out there may actually want to add anything that does respond to ping and can be found thru adjacency and routing protocol info etc??
Patch generated from recent trunk, touches files as below,
[root@desktop observium]# diff -r -u observium-trunk root | grep -v ^Only > ping_only_hosts.diff [root@desktop observium]# cd root [root@desktop root]# svn status M add_device.php M html/pages/addhost.inc.php M html/pages/device/edit/device.inc.php M html/pages/device/edit.inc.php M html/pages/device/graphs.inc.php M html/pages/device/perf.inc.php M includes/config-variables.inc.php M includes/defaults.inc.php M includes/definitions.inc.php M includes/discovery/functions.inc.php M includes/functions.inc.php M includes/polling/functions.inc.php M poller.php M rename_device.php [root@desktop root]# svn info | grep ^Revision Revision: 9704 [root@desktop root]# [root@desktop observium]# mv ping_only_hosts.diff ping_only_hosts_r9704.diff
-Colin
Email: cstubbs @ gmail . com
observium mailing list
observium@observium.org
*MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* http://postman.memetic.org/cgi-bin/mailman/listinfo/observium https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.memetic.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fobservium&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503019596&sdata=E967lJ4buMEKYKMMawVJYnlaC6MYm2evhD8JZDqESXs%3D&reserved=0
-- Mike Stupalov Observium Limited, *MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* http://observium.org https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fobservium.org&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503029613&sdata=bclwJLX8WOiFp4nyfLA2PxXvl40er5OPquD%2FA3XsY3Q%3D&reserved=0
observium mailing list observium@observium.org *MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.mem... https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.memetic.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fobservium&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503039618&sdata=DH9UrWMzx08fgOTm%2FH4FgeOtYBt4P2K%2BzNIiqtqxIPs%3D&reserved=0
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hello
That would be great because we have too some old devices that can't be added to Observium because of snmp / reply /discovery but will work with custom OID. So they will be added as a device with ping and add them the custom OID-s.
More greater would be that in the graphs of custom OID implement formulas.
Regards
From: observium observium-bounces@observium.org On Behalf Of Adam Armstrong via observium Sent: Sunday, March 3, 2019 6:09 PM To: Ron Marosko via observium observium@observium.org Cc: Adam Armstrong adama@memetic.org Subject: Re: [Observium] OBS-2925 - Ping Only Hosts / OBS_SNMP_SKIP flag / snmp_skip attribute
My usual answer to this is that this causes too much weirdness in the overall system to have "devices" which have zero information attached to them, so "just ping" should probably be a separate thing, perhaps done by a separate poller process. Just a list of hosts to ping.
This probably isn't very difficult to implement, and may be useful for other things (pinging external things to monitor availability a la IP SLA, but without an external device. I'm not sure how good a job we would do of this though. I think fping may help. Just a single fping run should supply the data for all pinged devices, right?
One of the problems with donated code is that it invariably gets abandoned by its original creators and then leaves us to maintain it to handle a 0.1% corner case. This has happened to us more than once in the past, and is now why we're resistant to implementing odd things that impact other parts of code and don't seem to have wide utility.
adam.
On 2019-03-03 16:52:44, Ron Marosko via observium <observium@observium.org mailto:observium@observium.org > wrote:
I’ll add my $0.02. In my implementation, we monitor 790 devices, and all of them have SNMP on them. There are a grand total of TWO devices that I’d like to add as ping-only because I don’t have an SNMP string on them… our two uplink internet providers.
Right now we rely on the BGP idle notification to tell us one of the internet uplinks is down. I’ve also built IP SLA on each of our border routers doing ICMP to the next hop IP on the internet side, and of course OBS will graph the results of the IP SLA… and I’m sure there’s probably a way to do an alert notice if the SLA shows a problem, but not all routers are going to have IP SLA capability.
So… to a limited extent, I’d *like* to see the ability to add ping-only devices, but I’m not crippled by not having the capability.
…Ron
--
Ron Marosko, Jr.
. . . . . . . . . . . . . . . . . . . . . . . . . .
CCIE No. 4526 (R/S) Emeritus, NN5DX, RM1517-ARIN
Senior Network Engineer
DirectBytes, LLC.
1108A W. Dickinson Blvd.
Fort Stockton, TX 79735 USA
o: +1 432 336 5600 x115
c: +1 432 290 6344
e: ron@directbytes.net mailto:ron@directbytes.net
pgp pubkey: 0x58AB8B5C http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x778B3B0258AB8B5C
"To know me is to fly with me."
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Ryan Huff via observium Sent: Sunday, March 3, 2019 7:44 AM To: cstubbs@gmail.com mailto:cstubbs@gmail.com Cc: Ryan Huff <ryanhuff@outlook.com mailto:ryanhuff@outlook.com >; Observium <observium@observium.org mailto:observium@observium.org > Subject: Re: [Observium] OBS-2925 - Ping Only Hosts / OBS_SNMP_SKIP flag / snmp_skip attribute
Naaa, mine is a big use case/implementation; it’s just 3 out of a bagillion devices that this needs done for, its not my personal email, it’s a distro. We could easily do an JSON post to any manner of transports, but “mail();” was so much simpler.
All we need to know is when it’s down so someone can go kick the hamster wheel, that’s “ping only”, monitoring to me.
If you’re looking for graphs / SLA type metrics too, then I’d argue you’re already looking for more than “ping only” monitoring....
I see what you’re saying though; OBS has such a great altogether GUI experience, it’d be nice!
-Ryan
On Mar 3, 2019, at 08:15, Colin Stubbs <cstubbs@gmail.com mailto:cstubbs@gmail.com > wrote:
I know where you're at... but,
0) No refactoring required - this is part of the point of the patch ( I would not have even attempted this if it wasn't so easy... ) and my long writeup for Mike given he hadn't actually looked at JIRA yet. Uses existing code/architecture - bit flag and device attributes - to mark a device as "don't do any SNMP to this device right now"
1) Observium uses fping...
2) Smokeping is horrible - can provide great value sure, but horrible to use, and again, extra tool. I'm a long way from unfamiliar with it and have both used it many years ago, and considered it for our current environments. Not useful to us.
3) Smokeping is not integrated to Observium if the device is not monitored by SNMP - this is fine when it's just you or a small team of people who understand this and are happy to flick between interfaces and how to work with Smokeping. Not great when you have a larger team and you really just need one interface on a monitor in your operations room that tells everyone everything that's up/down/problematic right now. Particularly not fine when your level 1's can't really login on CLI and edit config files etc.
4) IP SLA's - vendor specific thing. I love them in the right environment. Fantastic tool and capability to have in a network. We have zero Cisco in this situation and don't want to add a device specifically for this.
5) Your description sounds like it's just you in this environment... and if email fails, or you're hit by a bus, no one else knows there's a problem let alone how your alerting solution works and how to maintain it. Also, manual investigation/access to history. By having Observium do the ping we get to leverage the alerting capabilities inherent to it, e.g. email/webhook/PagerDuty/OpsGenie/VictorOps etc etc etc., as well as the webUI for visual alert notification and access to graphs/history.
It's actually a simpler and cleaner solution; unless you value seeing the min/max/average/jitter values that IP SLA or Smokeping can help you see. In which case it's not what you need.
-Colin
Email: cstubbs @ gmail . com
On Sun, 3 Mar 2019 at 22:44, Ryan Huff <ryanhuff@outlook.com mailto:ryanhuff@outlook.com > wrote:
I really do understand this Colin, on a deeper level than you might imagine ;). I have no vested interest either way personally, I’m just offering some unsolicited thoughts...
I always looked at this specific topic like asking a 10lbs. (4.5kg or 0.7149 stones) sledge hammer to drive a nail. Sure, it can do it but it’ll probably be messy and ugly because a sledge hammer was never meant to drive nails, and when you use tools in ways they weren’t really designed for, things can get messy. Just use a hammer instead.
I have done three things to achieve icmp only monitoring and also stick within the base code (as a developer, I hate mucking with base code or re-inventing wheels when I want to do different things). Icmp-only monitoring to me, always just seemed silly (not in purpose) to try and work into an existing and established application that didn’t already do it.
It’s just too easy of a problem to solve otherwise, than to go to the time/expense of refactoring an application IMHO.
- I use RTT SLAs to monitor next hop devices (which OBS is fabulous at picking up), assuming the device hopped from is SLA capable and OBS monitored.
- I use Fping (Smokeping) integrated with OBS. Using the Smokeping web wrapper makes a great RTT only tool, and you also have the OBS integration for when it makes sense to use on OBS devices.
- I have a small (very small) bash script that reads a text file of IP address and launches one thread per device to run continuous pings. Not a very scalable script, but like you, I only need it for a very few limited cases. The script uses sendmail to email me if it can’t reach something.
Thanks,
Ryan
On Mar 3, 2019, at 06:35, Colin Stubbs via observium <observium@observium.org mailto:observium@observium.org > wrote:
Hi Mike,
They're in the JIRA ticket, but sure, here's some inserted to this email.
I'm aware of Adam/your preference to reject or simply ignore the regular requests for this kind of feature, and the confusion it causes new users considering Observium for use in their organisations when they find out they can't do the most basic of monitoring to something using Observium.
A great many of your users would gain a lot of value from being able to add a few ping only hosts.
Why build, manage and integrate (to other tools) another ping based monitoring system when all you need is to monitor a handful of devices with ping only; simply because those devices fail to offer SNMP for some reason?
Anything that does have a working SNMP service can and should be monitored with SNMP to gain greater insight into what's happening on it... but not everything does.
No one in their right mind should attempt to use Observium as a system which monitors massive numbers of DNS names or IP's with ping only, there are better systems out there for that, but there are so many situations in a which an Observium install can and should be able to monitor some devices ping only.
I've associated the ticket with the two previous JIRA requests for this feature; which Adam rejected.
They were just feature requests as I recall; no code proposed. Totally understandable you guys didn't want to work on the feature when it's not really what you feel Observium is for.
Rejecting this now when you have just been gifted working code, that modifies things fairly minimally, and simply leverages the ping information that's already occurring to each and every device (that ping is not disabled for), and uses an existing method to mark devices so that further SNMP based polling does not occur... would make no sense.
That said; I totally understand if you want me to fix/add/improve things before considering it further - simply communicate what's necessary and I'll do that.
So, for anyone else on the list, who wants the feature; now is your best chance to add your voice and explain why this kind of feature is of use to you. Because you're probably not going to get it otherwise.
We've been using this internally since early 2017... I just got around to updating Observium to the latest and greatest (reasons, terrible terrible reasons...), and had to update my patch.
But, I'd really prefer not to have to maintain a patch, and I definitely don't want to have to publish it so others can use it and cause Observium Ltd further dramas because of it.
How many hosts do we monitor ping only?
Six. A grand total of six. Everything else has SNMP in some form; and we want as much info as we can extract from them via it - which is why we love Observium. Nothing else compares for SNMP based monitoring IMHO.
Most of those ping only devices are next hop service provider devices just outside our network, for which they won't give us any kind of SNMP access. Ping'ing them offers a small (very small!) degree of insight into link latency/loss from us to them (manual comparison I must admit), and/or potential device problems for the thing that's currently responding for that IP.
Hey look... a perfect use case for this feature...
The others are devices that we use but which simply don't offer SNMP because vendor X didn't bother to add net-snmp to their standard build for CentOS/Ubuntu/whatever weird and wonderful Linux distro they built them on; when they really should have. Don't care much just need to know they're on the network in some way and want a long track record of response time to them.
Hey look... a perfect use case for this feature...
Not surprisingly we don't want to have to operate and maintain a separate system for six IP's. Alerting can be flappy because Observium/fping is only sending a single ICMP echo - we're OK with that and don't even really want more pings which would lead to average values anyway - and we've worked around flaps with delays and automatic suppression of duplicate alerts in our alert management and escalation tool.
The six are monitored by IP; the DNS hostname test below was simply to double check there was no issue with this and a DNS name. Just a hostname that I've never managed to forget and use a lot to ping test connectivity out.
Also, I just realised that the "Skip ping" option is no longer automatically hidden in device settings when "Skip SNMP" is checked; I'll add an updated diff in JIRA shortly after I look at the JS for that again.
<Capture.PNG>
<observium_ping_only_screens3.PNG>
<observium_ping_only_screens4.png>
<observium_ping_only_screens5.PNG>
On Sun, 3 Mar 2019 at 18:07, Mike Stupalov <mike@observium.org mailto:mike@observium.org > wrote:
Ohh my dear..
Which really benefit to use Observium for ping_only hosts?
Pls show (screenshots) how this devices displayed on your install.
Colin Stubbs via observium wrote on 03/03/2019 07:43:
For the benefit of anyone on the list who doesn't use JIRA... and also so that others who support (want/need) this feature can comment.
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjira.observium.org%2Fbrowse%2FOBS-2925&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503009597&sdata=J5gbdkhaj1DVqk7F0gvDAk%2B6S25XfYziaa7xh8v5%2Bgc%3D&reserved=0 MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be https://jira.observium.org/browse/OBS-2925
Attached patch defines OBS_SNMP_SKIP flag and uses snmp_skip device attribute, similar to OBS_PING_SKIP and ping_skip, in order to have hosts that are ping only.
Ping only hosts can still have the Observium Unix Agent installed (tested), and other poller modules such as IPMI enabled (untested).
Tested:
1. Add/remove/rename ping only hosts via CLI 2. Add/remove ping only hosts via webUI 3. View/interact ping only hosts via webUI - SNMP specific features/menus etc are hidden while skip SNMP is enabled 4. Alerting - device_status equals 0 && device_status_type equals ping - will trigger alerts for host down/recovery events 5. Shifting a previously SNMP contactable host to ping only by ticking skip SNMP box - old SNMP graphs/etc are maintained and remain available - remove skip SNMP box and SNMP polling begins again
Things I know kind of don't work right now:
1. Location override - poller/discovery doesn't seem to perform geocoding and whatever else is happening there
Things that could be improved:
1. Unix Agent poller module etc is enabled by default for all hosts, for ping only hosts, perhaps it should be disabled by default? Will improve performance by reducing the number of processes that hang while the 10s default connect timeout happens.
Totally untested:
1. Use of autodiscovery SNMP skip - should work in theory, unsure if those parts of the patch should actually be used though. Some people out there may actually want to add anything that does respond to ping and can be found thru adjacency and routing protocol info etc??
Patch generated from recent trunk, touches files as below,
[root@desktop observium]# diff -r -u observium-trunk root | grep -v ^Only > ping_only_hosts.diff [root@desktop observium]# cd root [root@desktop root]# svn status M add_device.php M html/pages/addhost.inc.php M html/pages/device/edit/device.inc.php M html/pages/device/edit.inc.php M html/pages/device/graphs.inc.php M html/pages/device/perf.inc.php M includes/config-variables.inc.php M includes/defaults.inc.php M includes/definitions.inc.php M includes/discovery/functions.inc.php M includes/functions.inc.php M includes/polling/functions.inc.php M poller.php M rename_device.php [root@desktop root]# svn info | grep ^Revision Revision: 9704 [root@desktop root]# [root@desktop observium]# mv ping_only_hosts.diff ping_only_hosts_r9704.diff
-Colin
Email: cstubbs @ gmail . com
_______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.memetic.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fobservium&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503019596&sdata=E967lJ4buMEKYKMMawVJYnlaC6MYm2evhD8JZDqESXs%3D&reserved=0 MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hello all,
Just a little comment to mention that i will be also VERY interested with such feature. My context : arround 1500 devices (mostly routers and switches), with full SNMP stats that are usefull, but i would like to add 500 devices more with only availability check and history of that without consuming unnecessary system ressources for SNMP stats that i don't really care of ...
Tarik 5 mars 2019 10:01 "Mirditor Kuçaj via observium" <observium@observium.org (mailto:observium@observium.org?to=%22Mirditor%20Ku%C3%A7aj%20via%20observium%22%20observium@observium.org)> a écrit: Hello
That would be great because we have too some old devices that can't be added to Observium because of snmp / reply /discovery but will work with custom OID. So they will be added as a device with ping and add them the custom OID-s.
More greater would be that in the graphs of custom OID implement formulas.
Regards From: observium <observium-bounces@observium.org (mailto:observium-bounces@observium.org)> On Behalf Of Adam Armstrong via observium Sent: Sunday, March 3, 2019 6:09 PM To: Ron Marosko via observium <observium@observium.org (mailto:observium@observium.org)> Cc: Adam Armstrong <adama@memetic.org (mailto:adama@memetic.org)> Subject: Re: [Observium] OBS-2925 - Ping Only Hosts / OBS_SNMP_SKIP flag / snmp_skip attribute My usual answer to this is that this causes too much weirdness in the overall system to have "devices" which have zero information attached to them, so "just ping" should probably be a separate thing, perhaps done by a separate poller process. Just a list of hosts to ping. This probably isn't very difficult to implement, and may be useful for other things (pinging external things to monitor availability a la IP SLA, but without an external device. I'm not sure how good a job we would do of this though. I think fping may help. Just a single fping run should supply the data for all pinged devices, right? One of the problems with donated code is that it invariably gets abandoned by its original creators and then leaves us to maintain it to handle a 0.1% corner case. This has happened to us more than once in the past, and is now why we're resistant to implementing odd things that impact other parts of code and don't seem to have wide utility. adam.
On 2019-03-03 16:52:44, Ron Marosko via observium <observium@observium.org (mailto:observium@observium.org)> wrote: I’ll add my $0.02. In my implementation, we monitor 790 devices, and all of them have SNMP on them. There are a grand total of TWO devices that I’d like to add as ping-only because I don’t have an SNMP string on them… our two uplink internet providers.
Right now we rely on the BGP idle notification to tell us one of the internet uplinks is down. I’ve also built IP SLA on each of our border routers doing ICMP to the next hop IP on the internet side, and of course OBS will graph the results of the IP SLA… and I’m sure there’s probably a way to do an alert notice if the SLA shows a problem, but not all routers are going to have IP SLA capability.
So… to a limited extent, I’d *like* to see the ability to add ping-only devices, but I’m not crippled by not having the capability.
…Ron
--
Ron Marosko, Jr.
. . . . . . . . . . . . . . . . . . . . . . . . . .
CCIE No. 4526 (R/S) Emeritus, NN5DX, RM1517-ARIN
Senior Network Engineer
DirectBytes, LLC.
1108A W. Dickinson Blvd.
Fort Stockton, TX 79735 USA
o: +1 432 336 5600 x115
c: +1 432 290 6344
e: ron@directbytes.net (mailto:ron@directbytes.net)
pgp pubkey: 0x58AB8B5C (http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x778B3B0258AB8B5C)
"To know me is to fly with me." From: observium <observium-bounces@observium.org (mailto:observium-bounces@observium.org)> On Behalf Of Ryan Huff via observium Sent: Sunday, March 3, 2019 7:44 AM To: cstubbs@gmail.com (mailto:cstubbs@gmail.com) Cc: Ryan Huff <ryanhuff@outlook.com (mailto:ryanhuff@outlook.com)>; Observium <observium@observium.org (mailto:observium@observium.org)> Subject: Re: [Observium] OBS-2925 - Ping Only Hosts / OBS_SNMP_SKIP flag / snmp_skip attribute
Naaa, mine is a big use case/implementation; it’s just 3 out of a bagillion devices that this needs done for, its not my personal email, it’s a distro. We could easily do an JSON post to any manner of transports, but “mail();” was so much simpler. All we need to know is when it’s down so someone can go kick the hamster wheel, that’s “ping only”, monitoring to me. If you’re looking for graphs / SLA type metrics too, then I’d argue you’re already looking for more than “ping only” monitoring.... I see what you’re saying though; OBS has such a great altogether GUI experience, it’d be nice! -Ryan On Mar 3, 2019, at 08:15, Colin Stubbs <cstubbs@gmail.com (mailto:cstubbs@gmail.com)> wrote: I know where you're at... but, 0) No refactoring required - this is part of the point of the patch ( I would not have even attempted this if it wasn't so easy... ) and my long writeup for Mike given he hadn't actually looked at JIRA yet. Uses existing code/architecture - bit flag and device attributes - to mark a device as "don't do any SNMP to this device right now" 1) Observium uses fping... 2) Smokeping is horrible - can provide great value sure, but horrible to use, and again, extra tool. I'm a long way from unfamiliar with it and have both used it many years ago, and considered it for our current environments. Not useful to us. 3) Smokeping is not integrated to Observium if the device is not monitored by SNMP - this is fine when it's just you or a small team of people who understand this and are happy to flick between interfaces and how to work with Smokeping. Not great when you have a larger team and you really just need one interface on a monitor in your operations room that tells everyone everything that's up/down/problematic right now. Particularly not fine when your level 1's can't really login on CLI and edit config files etc. 4) IP SLA's - vendor specific thing. I love them in the right environment. Fantastic tool and capability to have in a network. We have zero Cisco in this situation and don't want to add a device specifically for this. 5) Your description sounds like it's just you in this environment... and if email fails, or you're hit by a bus, no one else knows there's a problem let alone how your alerting solution works and how to maintain it. Also, manual investigation/access to history. By having Observium do the ping we get to leverage the alerting capabilities inherent to it, e.g. email/webhook/PagerDuty/OpsGenie/VictorOps etc etc etc., as well as the webUI for visual alert notification and access to graphs/history. It's actually a simpler and cleaner solution; unless you value seeing the min/max/average/jitter values that IP SLA or Smokeping can help you see. In which case it's not what you need. -Colin
Email: cstubbs @ gmail . com On Sun, 3 Mar 2019 at 22:44, Ryan Huff <ryanhuff@outlook.com (mailto:ryanhuff@outlook.com)> wrote: I really do understand this Colin, on a deeper level than you might imagine ;). I have no vested interest either way personally, I’m just offering some unsolicited thoughts... I always looked at this specific topic like asking a 10lbs. (4.5kg or 0.7149 stones) sledge hammer to drive a nail. Sure, it can do it but it’ll probably be messy and ugly because a sledge hammer was never meant to drive nails, and when you use tools in ways they weren’t really designed for, things can get messy. Just use a hammer instead. I have done three things to achieve icmp only monitoring and also stick within the base code (as a developer, I hate mucking with base code or re-inventing wheels when I want to do different things). Icmp-only monitoring to me, always just seemed silly (not in purpose) to try and work into an existing and established application that didn’t already do it. It’s just too easy of a problem to solve otherwise, than to go to the time/expense of refactoring an application IMHO. - I use RTT SLAs to monitor next hop devices (which OBS is fabulous at picking up), assuming the device hopped from is SLA capable and OBS monitored. - I use Fping (Smokeping) integrated with OBS. Using the Smokeping web wrapper makes a great RTT only tool, and you also have the OBS integration for when it makes sense to use on OBS devices. - I have a small (very small) bash script that reads a text file of IP address and launches one thread per device to run continuous pings. Not a very scalable script, but like you, I only need it for a very few limited cases. The script uses sendmail to email me if it can’t reach something. Thanks, Ryan On Mar 3, 2019, at 06:35, Colin Stubbs via observium <observium@observium.org (mailto:observium@observium.org)> wrote: Hi Mike, They're in the JIRA ticket, but sure, here's some inserted to this email. I'm aware of Adam/your preference to reject or simply ignore the regular requests for this kind of feature, and the confusion it causes new users considering Observium for use in their organisations when they find out they can't do the most basic of monitoring to something using Observium. A great many of your users would gain a lot of value from being able to add a few ping only hosts. Why build, manage and integrate (to other tools) another ping based monitoring system when all you need is to monitor a handful of devices with ping only; simply because those devices fail to offer SNMP for some reason? Anything that does have a working SNMP service can and should be monitored with SNMP to gain greater insight into what's happening on it... but not everything does. No one in their right mind should attempt to use Observium as a system which monitors massive numbers of DNS names or IP's with ping only, there are better systems out there for that, but there are so many situations in a which an Observium install can and should be able to monitor some devices ping only. I've associated the ticket with the two previous JIRA requests for this feature; which Adam rejected. They were just feature requests as I recall; no code proposed. Totally understandable you guys didn't want to work on the feature when it's not really what you feel Observium is for. Rejecting this now when you have just been gifted working code, that modifies things fairly minimally, and simply leverages the ping information that's already occurring to each and every device (that ping is not disabled for), and uses an existing method to mark devices so that further SNMP based polling does not occur... would make no sense. That said; I totally understand if you want me to fix/add/improve things before considering it further - simply communicate what's necessary and I'll do that. So, for anyone else on the list, who wants the feature; now is your best chance to add your voice and explain why this kind of feature is of use to you. Because you're probably not going to get it otherwise. We've been using this internally since early 2017... I just got around to updating Observium to the latest and greatest (reasons, terrible terrible reasons...), and had to update my patch. But, I'd really prefer not to have to maintain a patch, and I definitely don't want to have to publish it so others can use it and cause Observium Ltd further dramas because of it. How many hosts do we monitor ping only? Six. A grand total of six. Everything else has SNMP in some form; and we want as much info as we can extract from them via it - which is why we love Observium. Nothing else compares for SNMP based monitoring IMHO. Most of those ping only devices are next hop service provider devices just outside our network, for which they won't give us any kind of SNMP access. Ping'ing them offers a small (very small!) degree of insight into link latency/loss from us to them (manual comparison I must admit), and/or potential device problems for the thing that's currently responding for that IP. Hey look... a perfect use case for this feature... The others are devices that we use but which simply don't offer SNMP because vendor X didn't bother to add net-snmp to their standard build for CentOS/Ubuntu/whatever weird and wonderful Linux distro they built them on; when they really should have. Don't care much just need to know they're on the network in some way and want a long track record of response time to them. Hey look... a perfect use case for this feature... Not surprisingly we don't want to have to operate and maintain a separate system for six IP's. Alerting can be flappy because Observium/fping is only sending a single ICMP echo - we're OK with that and don't even really want more pings which would lead to average values anyway - and we've worked around flaps with delays and automatic suppression of duplicate alerts in our alert management and escalation tool. The six are monitored by IP; the DNS hostname test below was simply to double check there was no issue with this and a DNS name. Just a hostname that I've never managed to forget and use a lot to ping test connectivity out. Also, I just realised that the "Skip ping" option is no longer automatically hidden in device settings when "Skip SNMP" is checked; I'll add an updated diff in JIRA shortly after I look at the JS for that again. <Capture.PNG> <observium_ping_only_screens3.PNG> <observium_ping_only_screens4.png> <observium_ping_only_screens5.PNG> On Sun, 3 Mar 2019 at 18:07, Mike Stupalov <mike@observium.org (mailto:mike@observium.org)> wrote: Ohh my dear..
Which really benefit to use Observium for ping_only hosts?
Pls show (screenshots) how this devices displayed on your install.
Colin Stubbs via observium wrote on 03/03/2019 07:43: For the benefit of anyone on the list who doesn't use JIRA... and also so that others who support (want/need) this feature can comment. MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be https://jira.observium.org/browse/OBS-2925 (https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjira.obser...) Attached patch defines OBS_SNMP_SKIP flag and uses snmp_skip device attribute, similar to OBS_PING_SKIP and ping_skip, in order to have hosts that are ping only.
Ping only hosts can still have the Observium Unix Agent installed (tested), and other poller modules such as IPMI enabled (untested).
Tested: * Add/remove/rename ping only hosts via CLI * Add/remove ping only hosts via webUI * View/interact ping only hosts via webUI - SNMP specific features/menus etc are hidden while skip SNMP is enabled * Alerting - device_status equals 0 && device_status_type equals ping - will trigger alerts for host down/recovery events * Shifting a previously SNMP contactable host to ping only by ticking skip SNMP box - old SNMP graphs/etc are maintained and remain available - remove skip SNMP box and SNMP polling begins again Things I know kind of don't work right now: * Location override - poller/discovery doesn't seem to perform geocoding and whatever else is happening there Things that could be improved: * Unix Agent poller module etc is enabled by default for all hosts, for ping only hosts, perhaps it should be disabled by default? Will improve performance by reducing the number of processes that hang while the 10s default connect timeout happens. Totally untested: * Use of autodiscovery SNMP skip - should work in theory, unsure if those parts of the patch should actually be used though. Some people out there may actually want to add anything that does respond to ping and can be found thru adjacency and routing protocol info etc?? Patch generated from recent trunk, touches files as below,
[root@desktop observium]# diff -r -u observium-trunk root | grep -v ^Only > ping_only_hosts.diff [root@desktop observium]# cd root [root@desktop root]# svn status M add_device.php M html/pages/addhost.inc.php M html/pages/device/edit/device.inc.php M html/pages/device/edit.inc.php M html/pages/device/graphs.inc.php M html/pages/device/perf.inc.php M includes/config-variables.inc.php M includes/defaults.inc.php M includes/definitions.inc.php M includes/discovery/functions.inc.php M includes/functions.inc.php M includes/polling/functions.inc.php M poller.php M rename_device.php [root@desktop root]# svn info | grep ^Revision Revision: 9704 [root@desktop root]# [root@desktop observium]# mv ping_only_hosts.diff ping_only_hosts_r9704.diff -Colin
Email: cstubbs @ gmail . com
_______________________________________________
observium mailing list
observium@observium.org (mailto:observium@observium.org)
MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be http://postman.memetic.org/cgi-bin/mailman/listinfo/observium (https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.mem...) -- Mike Stupalov Observium Limited, MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be http://observium.org (https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fobservium.o...) _______________________________________________ observium mailing list observium@observium.org (mailto:observium@observium.org) MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.mem... (https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.mem...)
I'd also be very interested in this.
Like most of us, most of our devices support SNMP and are monitored that way.
Our Edge cases would be VOIP Phones, which only respond to ping, ISP gateways, and a couple of other esoteric devices which either dont have snmp, or the implementation is next to useless.
These devices we only care to know that they are plugged in and responding.
Currently we rely on a real botch of a spreadsheet which allows us to know which switch and port these devices are on, so that when we get a port down notification, we know which device we need to go investigate. Having an alert based purely off the ping would save us a lot of time when these devices go down.
Regards, James Tandy TandyUK Servers Limited
Tel: 01903 247 011 Www: http://www.tandyukservers.co.uk Email: support@tandyukservers.co.uk
TandyUK Servers Limited Registered in England and Wales, Company number 8314911 VAT Registered in the UK, number 182 0661 19 Registered Office: Amelia House, Crescent Road, Worthing, BN11 1QR
On 05/03/2019 09:16, Trik via observium wrote:
Hello all,
Just a little comment to mention that i will be also VERY interested with such feature. My context : arround 1500 devices (mostly routers and switches), with full SNMP stats that are usefull, but i would like to add 500 devices more with only availability check and history of that without consuming unnecessary system ressources for SNMP stats that i don't really care of ...
Tarik
5 mars 2019 10:01 "Mirditor Kuçaj via observium" <observium@observium.org <mailto:observium@observium.org?to=%22Mirditor%20Ku%C3%A7aj%20via%20observium%22%20observium@observium.org>> a écrit:
Hello That would be great because we have too some old devices that can't be added to Observium because of snmp / reply /discovery but will work with custom OID. So they will be added as a device with ping and add them the custom OID-s. More greater would be that in the graphs of custom OID implement formulas. Regards *From:* observium <observium-bounces@observium.org <mailto:observium-bounces@observium.org>> *On Behalf Of *Adam Armstrong via observium *Sent:* Sunday, March 3, 2019 6:09 PM *To:* Ron Marosko via observium <observium@observium.org <mailto:observium@observium.org>> *Cc:* Adam Armstrong <adama@memetic.org <mailto:adama@memetic.org>> *Subject:* Re: [Observium] OBS-2925 - Ping Only Hosts / OBS_SNMP_SKIP flag / snmp_skip attribute My usual answer to this is that this causes too much weirdness in the overall system to have "devices" which have zero information attached to them, so "just ping" should probably be a separate thing, perhaps done by a separate poller process. Just a list of hosts to ping. This probably isn't very difficult to implement, and may be useful for other things (pinging external things to monitor availability a la IP SLA, but without an external device. I'm not sure how good a job we would do of this though. I think fping may help. Just a single fping run should supply the data for all pinged devices, right? One of the problems with donated code is that it invariably gets abandoned by its original creators and then leaves us to maintain it to handle a 0.1% corner case. This has happened to us more than once in the past, and is now why we're resistant to implementing odd things that impact other parts of code and don't seem to have wide utility. adam. On 2019-03-03 16:52:44, Ron Marosko via observium <observium@observium.org <mailto:observium@observium.org>> wrote: I’ll add my $0.02. In my implementation, we monitor 790 devices, and all of them have SNMP on them. There are a grand total of TWO devices that I’d like to add as ping-only because I don’t have an SNMP string on them… our two uplink internet providers. Right now we rely on the BGP idle notification to tell us one of the internet uplinks is down. I’ve also built IP SLA on each of our border routers doing ICMP to the next hop IP on the internet side, and of course OBS will graph the results of the IP SLA… and I’m sure there’s probably a way to do an alert notice if the SLA shows a problem, but not all routers are going to have IP SLA capability. So… to a limited extent, I’d **like** to see the ability to add ping-only devices, but I’m not crippled by not having the capability. …Ron -- *Ron Marosko, Jr.* . . . . . . . . . . . . . . . . . . . . . . . . . . CCIE No. 4526 (R/S) Emeritus, NN5DX, RM1517-ARIN Senior Network Engineer *_DirectBytes, LLC._* 1108A W. Dickinson Blvd. Fort Stockton, TX 79735 USA *o:**+1 432 336 5600 x115* *c:*+1 432 290 6344 *e:*_ron@directbytes.net <mailto:ron@directbytes.net>_ *pgp pubkey:*_0x58AB8B5C <http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x778B3B0258AB8B5C>_ *"To know me is to fly with me."* *From:*observium <observium-bounces@observium.org <mailto:observium-bounces@observium.org>> *On Behalf Of *Ryan Huff via observium *Sent:* Sunday, March 3, 2019 7:44 AM *To:* cstubbs@gmail.com <mailto:cstubbs@gmail.com> *Cc:* Ryan Huff <ryanhuff@outlook.com <mailto:ryanhuff@outlook.com>>; Observium <observium@observium.org <mailto:observium@observium.org>> *Subject:* Re: [Observium] OBS-2925 - Ping Only Hosts / OBS_SNMP_SKIP flag / snmp_skip attribute Naaa, mine is a big use case/implementation; it’s just 3 out of a bagillion devices that this needs done for, its not my personal email, it’s a distro. We could easily do an JSON post to any manner of transports, but “mail();” was so much simpler. All we need to know is when it’s down so someone can go kick the hamster wheel, that’s “ping only”, monitoring to me. If you’re looking for graphs / SLA type metrics too, then I’d argue you’re already looking for more than “ping only” monitoring.... I see what you’re saying though; OBS has such a great altogether GUI experience, it’d be nice! -Ryan On Mar 3, 2019, at 08:15, Colin Stubbs <cstubbs@gmail.com <mailto:cstubbs@gmail.com>> wrote: I know where you're at... but, 0) No refactoring required - this is part of the point of the patch ( I would not have even attempted this if it wasn't so easy... ) and my long writeup for Mike given he hadn't actually looked at JIRA yet. Uses existing code/architecture - bit flag and device attributes - to mark a device as "don't do any SNMP to this device right now" 1) Observium uses fping... 2) Smokeping is horrible - can provide great value sure, but horrible to use, and again, extra tool. I'm a long way from unfamiliar with it and have both used it many years ago, and considered it for our current environments. Not useful to us. 3) Smokeping is not integrated to Observium if the device is not monitored by SNMP - this is fine when it's just you or a small team of people who understand this and are happy to flick between interfaces and how to work with Smokeping. Not great when you have a larger team and you really just need one interface on a monitor in your operations room that tells everyone everything that's up/down/problematic right now. Particularly not fine when your level 1's can't really login on CLI and edit config files etc. 4) IP SLA's - vendor specific thing. I love them in the right environment. Fantastic tool and capability to have in a network. We have zero Cisco in this situation and don't want to add a device specifically for this. 5) Your description sounds like it's just you in this environment... and if email fails, or you're hit by a bus, no one else knows there's a problem let alone how your alerting solution works and how to maintain it. Also, manual investigation/access to history. By having Observium do the ping we get to leverage the alerting capabilities inherent to it, e.g. email/webhook/PagerDuty/OpsGenie/VictorOps etc etc etc., as well as the webUI for visual alert notification and access to graphs/history. It's actually a simpler and cleaner solution; unless you value seeing the min/max/average/jitter values that IP SLA or Smokeping can help you see. In which case it's not what you need. -Colin Email: cstubbs @ gmail . com On Sun, 3 Mar 2019 at 22:44, Ryan Huff <ryanhuff@outlook.com <mailto:ryanhuff@outlook.com>> wrote: I really do understand this Colin, on a deeper level than you might imagine ;). I have no vested interest either way personally, I’m just offering some unsolicited thoughts... I always looked at this specific topic like asking a 10lbs. (4.5kg or 0.7149 stones) sledge hammer to drive a nail. Sure, it can do it but it’ll probably be messy and ugly because a sledge hammer was never meant to drive nails, and when you use tools in ways they weren’t really designed for, things can get messy. Just use a hammer instead. I have done three things to achieve icmp only monitoring and also stick within the base code (as a developer, I hate mucking with base code or re-inventing wheels when I want to do different things). Icmp-only monitoring to me, always just seemed silly (not in purpose) to try and work into an existing and established application that didn’t already do it. It’s just too easy of a problem to solve otherwise, than to go to the time/expense of refactoring an application IMHO. - I use RTT SLAs to monitor next hop devices (which OBS is fabulous at picking up), assuming the device hopped from is SLA capable and OBS monitored. - I use Fping (Smokeping) integrated with OBS. Using the Smokeping web wrapper makes a great RTT only tool, and you also have the OBS integration for when it makes sense to use on OBS devices. - I have a small (very small) bash script that reads a text file of IP address and launches one thread per device to run continuous pings. Not a very scalable script, but like you, I only need it for a very few limited cases. The script uses sendmail to email me if it can’t reach something. Thanks, Ryan On Mar 3, 2019, at 06:35, Colin Stubbs via observium <observium@observium.org <mailto:observium@observium.org>> wrote: Hi Mike, They're in the JIRA ticket, but sure, here's some inserted to this email. I'm aware of Adam/your preference to reject or simply ignore the regular requests for this kind of feature, and the confusion it causes new users considering Observium for use in their organisations when they find out they can't do the most basic of monitoring to something using Observium. A great many of your users would gain a lot of value from being able to add a few ping only hosts. Why build, manage and integrate (to other tools) another ping based monitoring system when all you need is to monitor a handful of devices with ping only; simply because those devices fail to offer SNMP for some reason? Anything that does have a working SNMP service can and should be monitored with SNMP to gain greater insight into what's happening on it... but not everything does. No one in their right mind should attempt to use Observium as a system which monitors massive numbers of DNS names or IP's with ping only, there are better systems out there for that, but there are so many situations in a which an Observium install can and should be able to monitor some devices ping only. I've associated the ticket with the two previous JIRA requests for this feature; which Adam rejected. They were just feature requests as I recall; no code proposed. Totally understandable you guys didn't want to work on the feature when it's not really what you feel Observium is for. Rejecting this now when you have just been gifted working code, that modifies things fairly minimally, and simply leverages the ping information that's already occurring to each and every device (that ping is not disabled for), and uses an existing method to mark devices so that further SNMP based polling does not occur... would make no sense. That said; I totally understand if you want me to fix/add/improve things before considering it further - simply communicate what's necessary and I'll do that. So, for anyone else on the list, who wants the feature; now is your best chance to add your voice and explain why this kind of feature is of use to you. Because you're probably not going to get it otherwise. We've been using this internally since early 2017... I just got around to updating Observium to the latest and greatest (reasons, terrible terrible reasons...), and had to update my patch. But, I'd really prefer not to have to maintain a patch, and I definitely don't want to have to publish it so others can use it and cause Observium Ltd further dramas because of it. How many hosts do we monitor ping only? Six. A grand total of six. Everything else has SNMP in some form; and we want as much info as we can extract from them via it - which is why we love Observium. Nothing else compares for SNMP based monitoring IMHO. Most of those ping only devices are next hop service provider devices just outside our network, for which they won't give us any kind of SNMP access. Ping'ing them offers a small (very small!) degree of insight into link latency/loss from us to them (manual comparison I must admit), and/or potential device problems for the thing that's currently responding for that IP. Hey look... a perfect use case for this feature... The others are devices that we use but which simply don't offer SNMP because vendor X didn't bother to add net-snmp to their standard build for CentOS/Ubuntu/whatever weird and wonderful Linux distro they built them on; when they really should have. Don't care much just need to know they're on the network in some way and want a long track record of response time to them. Hey look... a perfect use case for this feature... Not surprisingly we don't want to have to operate and maintain a separate system for six IP's. Alerting can be flappy because Observium/fping is only sending a single ICMP echo - we're OK with that and don't even really want more pings which would lead to average values anyway - and we've worked around flaps with delays and automatic suppression of duplicate alerts in our alert management and escalation tool. The six are monitored by IP; the DNS hostname test below was simply to double check there was no issue with this and a DNS name. Just a hostname that I've never managed to forget and use a lot to ping test connectivity out. Also, I just realised that the "Skip ping" option is no longer automatically hidden in device settings when "Skip SNMP" is checked; I'll add an updated diff in JIRA shortly after I look at the JS for that again. <Capture.PNG> <observium_ping_only_screens3.PNG> <observium_ping_only_screens4.png> <observium_ping_only_screens5.PNG> On Sun, 3 Mar 2019 at 18:07, Mike Stupalov <mike@observium.org <mailto:mike@observium.org>> wrote: Ohh my dear.. Which really benefit to use Observium for ping_only hosts? Pls show (screenshots) how this devices displayed on your install. Colin Stubbs via observium wrote on 03/03/2019 07:43: For the benefit of anyone on the list who doesn't use JIRA... and also so that others who support (want/need) this feature can comment. *MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* https://jira.observium.org/browse/OBS-2925 <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjira.observium.org%2Fbrowse%2FOBS-2925&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503009597&sdata=J5gbdkhaj1DVqk7F0gvDAk%2B6S25XfYziaa7xh8v5%2Bgc%3D&reserved=0> Attached patch defines OBS_SNMP_SKIP flag and uses snmp_skip device attribute, similar to OBS_PING_SKIP and ping_skip, in order to have hosts that are ping only. Ping only hosts can still have the Observium Unix Agent installed (tested), and other poller modules such as IPMI enabled (untested). Tested: 1. Add/remove/rename ping only hosts via CLI 2. Add/remove ping only hosts via webUI 3. View/interact ping only hosts via webUI - SNMP specific features/menus etc are hidden while skip SNMP is enabled 4. Alerting - device_status equals 0 && device_status_type equals ping - will trigger alerts for host down/recovery events 5. Shifting a previously SNMP contactable host to ping only by ticking skip SNMP box - old SNMP graphs/etc are maintained and remain available - remove skip SNMP box and SNMP polling begins again Things I know kind of don't work right now: 1. Location override - poller/discovery doesn't seem to perform geocoding and whatever else is happening there Things that could be improved: 1. Unix Agent poller module etc is enabled by default for all hosts, for ping only hosts, perhaps it should be disabled by default? Will improve performance by reducing the number of processes that hang while the 10s default connect timeout happens. Totally untested: 1. Use of autodiscovery SNMP skip - should work in theory, unsure if those parts of the patch should actually be used though. Some people out there may actually want to add anything that does respond to ping and can be found thru adjacency and routing protocol info etc?? Patch generated from recent trunk, touches files as below, [root@desktop observium]# diff -r -u observium-trunk root | grep -v ^Only > ping_only_hosts.diff [root@desktop observium]# cd root [root@desktop root]# svn status M add_device.php M html/pages/addhost.inc.php M html/pages/device/edit/device.inc.php M html/pages/device/edit.inc.php M html/pages/device/graphs.inc.php M html/pages/device/perf.inc.php M includes/config-variables.inc.php M includes/defaults.inc.php M includes/definitions.inc.php M includes/discovery/functions.inc.php M includes/functions.inc.php M includes/polling/functions.inc.php M poller.php M rename_device.php [root@desktop root]# svn info | grep ^Revision Revision: 9704 [root@desktop root]# [root@desktop observium]# mv ping_only_hosts.diff ping_only_hosts_r9704.diff -Colin Email: cstubbs @ gmail . com _______________________________________________ observium mailing list observium@observium.org <mailto:observium@observium.org> *MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* http://postman.memetic.org/cgi-bin/mailman/listinfo/observium <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.memetic.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fobservium&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503019596&sdata=E967lJ4buMEKYKMMawVJYnlaC6MYm2evhD8JZDqESXs%3D&reserved=0> -- Mike Stupalov Observium Limited, *MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* http://observium.org <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fobservium.org&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503029613&sdata=bclwJLX8WOiFp4nyfLA2PxXvl40er5OPquD%2FA3XsY3Q%3D&reserved=0> _______________________________________________ observium mailing list observium@observium.org <mailto:observium@observium.org> *MailScanner has detected a possible fraud attempt from "eur03.safelinks.protection.outlook.com" claiming to be* https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.memetic.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fobservium&data=02%7C01%7C%7C80ff8602df2d450722c308d69fcc50ed%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872097241129331&sdata=06cAZMXbynH1msQQJ1ONKDhNoJ3dag%2FbeJArAMph1Jg%3D&reserved=0 <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpostman.memetic.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fobservium&data=02%7C01%7C%7C2802454d64664ef4a80008d69fda5ae1%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636872157503039618&sdata=DH9UrWMzx08fgOTm%2FH4FgeOtYBt4P2K%2BzNIiqtqxIPs%3D&reserved=0>
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
participants (9)
-
Adam Armstrong -
Colin Stubbs -
James Tandy -
Markus Klock -
Mirditor Kuçaj -
Philippe Laurent -
Ron Marosko -
Ryan Huff -
todevnull＠free.fr