- observium - lists.observium.org

Debuging slow SNMP-queries
by Markus Klock 27 Jun '16

27 Jun '16

Hey guys, I noticed I have some devices in my network that are responding very slow on just some specific SNMP-queries so I hacked together a small script that will parse the debug-output of Observium poller or discovery and then show you a list of the snmp-commands that took the longest to complete. I wrote a blogpost on how to use it here: http://blog.best-practice.se/2016/06/observium-performance-tuning.html and you can download the script from github: https://github.com/ZerxXxes/observium-logparser [image: Infogad bild 1] It was very helpful in finding what devices needed a software update ;) I hope it can be useful for others too /Markus

2 2

Re: [Observium] observium Digest, Vol 71, Issue 24
by Dudu Handelman 27 Jun '16

27 Jun '16

You should monitor your Dell server using snmp on the idrac itself like its a different device. The most important thing is the global status It will display in alert in the statuses front page. The global status is the color of the led when it's amber you have an issue. Now you will be able to see that in observium as well. Good luck. David Sent from my Samsung device -------- Original message -------- From: observium-request(a)observium.org Date: 27/06/2016 14:00 (GMT+02:00) To: observium(a)observium.org Subject: observium Digest, Vol 71, Issue 24 Send observium mailing list submissions to observium(a)observium.org To subscribe or unsubscribe via the World Wide Web, visit http://postman.memetic.org/cgi-bin/mailman/listinfo/observium or, via email, send a message with subject or body 'help' to observium-request(a)observium.org You can reach the person managing the list at observium-owner(a)observium.org When replying, please edit your Subject line so it is more specific than "Re: Contents of observium digest..." Today's Topics: 1. Re: Doc about IPMI (for Dell Server) (prost pierrick) 2. Re: Delay the alert trigger in case of loss only one ping (prost pierrick) 3. Re: Doc about IPMI (for Dell Server) (Adam Armstrong) 4. Re: Delay the alert trigger in case of loss only one ping (Adam Armstrong) 5. Re: Doc about IPMI (for Dell Server) (prost pierrick) 6. Re: Delay the alert trigger in case of loss only one ping (Moerman, Maarten) ---------------------------------------------------------------------- Message: 1 Date: Mon, 27 Jun 2016 09:03:18 +0000 From: prost pierrick <pierrick.prost(a)cnrs.fr> To: "observium(a)observium.org" <observium(a)observium.org> Subject: Re: [Observium] Doc about IPMI (for Dell Server) Message-ID: <AA561185F8EC764C88F0BA9DB4F2948E22680FD5(a)CNMB02WVP.core-res.rootcore.local> Content-Type: text/plain; charset="iso-8859-1" Hi, Nobody use IMPI ? regards De : prost pierrick Envoyé : vendredi 24 juin 2016 15:04 À : observium(a)observium.org Objet : Doc about IPMI (for Dell Server) Importance : Haute Hi everyone, There is documentation about IPMI on observium ? what the advantage ? benefit ? I have several DELL server with IDRAC and IPMI and only monitored my server with SNMP. Have a good day. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://postman.memetic.org/pipermail/observium/attachments/20160627/0b636d6…> ------------------------------ Message: 2 Date: Mon, 27 Jun 2016 09:07:38 +0000 From: prost pierrick <pierrick.prost(a)cnrs.fr> To: Observium Network Observation System <observium(a)observium.org> Subject: Re: [Observium] Delay the alert trigger in case of loss only one ping Message-ID: <AA561185F8EC764C88F0BA9DB4F2948E22680FE6(a)CNMB02WVP.core-res.rootcore.local> Content-Type: text/plain; charset="utf-8" Hello, I have make some tries but I’m still spammed with fake alerting. Delay value is in minute ? How can I debug poller to check if I really lost ping ? (I’m on a 10 gbit backbone network.. I’m really surprise). Regards pierrick De : prost pierrick Envoyé : jeudi 23 juin 2016 12:36 À : Observium Network Observation System Objet : RE: [Observium] Delay the alert trigger in case of loss only one ping Delay ..yes i see it ! Thanks have a good day. De : observium [mailto:observium-bounces@observium.org] De la part de Adam Armstrong Envoyé : jeudi 23 juin 2016 12:21 À : Observium Network Observation System Cc : observium(a)observium.org<mailto:observium@observium.org> Objet : Re: [Observium] Delay the alert trigger in case of loss only one ping it isn't just one ping, it tries multiple before it marks a device as down. You can set the delay field, which will cause the alert to wait for x number of down periods before it triggers. Adam. Sent from BlueMail<http://www.bluemail.me/r> On 23 Jun 2016, at 10:39, prost pierrick <pierrick.prost(a)cnrs.fr<mailto:pierrick.prost@cnrs.fr>> wrote: Hi everyone , Observium lost 1 ping sometime (I don’t know why ) after each ping lost, alerts are triggered. Actually I have only two Alerts Checks : Device Down device_status equals 0 * * Device UP device_status equals 1 * * It’s possible to add a delay before triggering ? Like waiting 5 ping lost before triggering ? Regards. Pierrick Prost ________________________________ observium mailing list observium(a)observium.org<mailto:observium@observium.org> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://postman.memetic.org/pipermail/observium/attachments/20160627/748d78f…> ------------------------------ Message: 3 Date: Mon, 27 Jun 2016 10:15:10 +0100 From: Adam Armstrong <adama(a)memetic.org> To: Observium Network Observation System <observium(a)observium.org> Cc: observium(a)observium.org Subject: Re: [Observium] Doc about IPMI (for Dell Server) Message-ID: <84cdd0b2-803e-4b92-8e21-9630f7e2a1e4(a)typeapp.com> Content-Type: text/plain; charset="utf-8" Did you miss the conversation people had about ipmi last week? OK Adam. Sent from BlueMail On 27 Jun 2016, 10:03, at 10:03, prost pierrick <pierrick.prost(a)cnrs.fr> wrote: >Hi, > >Nobody use IMPI ? > >regards > >De : prost pierrick >Envoyé : vendredi 24 juin 2016 15:04 >À : observium(a)observium.org >Objet : Doc about IPMI (for Dell Server) >Importance : Haute > >Hi everyone, > >There is documentation about IPMI on observium ? what the advantage ? >benefit ? >I have several DELL server with IDRAC and IPMI and only monitored my >server with SNMP. > > > > >Have a good day. > > > > > >------------------------------------------------------------------------ > >_______________________________________________ >observium mailing list >observium(a)observium.org >http://postman.memetic.org/cgi-bin/mailman/listinfo/observium -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://postman.memetic.org/pipermail/observium/attachments/20160627/d1d9a27…> ------------------------------ Message: 4 Date: Mon, 27 Jun 2016 10:16:32 +0100 From: Adam Armstrong <adama(a)memetic.org> To: Observium Network Observation System <observium(a)observium.org> Subject: Re: [Observium] Delay the alert trigger in case of loss only one ping Message-ID: <945c9cc7-fa6a-4735-80ba-a74cfa699056(a)typeapp.com> Content-Type: text/plain; charset="utf-8" Delay is in poller intervals. If you're getting notifications that your devices are down, you have some pretty serious network problems, as it takes quite a lot to actually mark something as down. Adam. Sent from BlueMail On 27 Jun 2016, 10:07, at 10:07, prost pierrick <pierrick.prost(a)cnrs.fr> wrote: >Hello, > >I have make some tries but I’m still spammed with fake alerting. > >Delay value is in minute ? How can I debug poller to check if I really >lost ping ? (I’m on a 10 gbit backbone network.. I’m really surprise). > > >Regards > >pierrick > > >De : prost pierrick >Envoyé : jeudi 23 juin 2016 12:36 >À : Observium Network Observation System >Objet : RE: [Observium] Delay the alert trigger in case of loss only >one ping > >Delay ..yes i see it ! > >Thanks have a good day. > > >De : observium [mailto:observium-bounces@observium.org] De la part de >Adam Armstrong >Envoyé : jeudi 23 juin 2016 12:21 >À : Observium Network Observation System >Cc : observium(a)observium.org<mailto:observium@observium.org> >Objet : Re: [Observium] Delay the alert trigger in case of loss only >one ping > > >it isn't just one ping, it tries multiple before it marks a device as >down. > >You can set the delay field, which will cause the alert to wait for x >number of down periods before it triggers. > >Adam. > >Sent from BlueMail<http://www.bluemail.me/r> >On 23 Jun 2016, at 10:39, prost pierrick ><pierrick.prost(a)cnrs.fr<mailto:pierrick.prost@cnrs.fr>> wrote: >Hi everyone , >Observium lost 1 ping sometime (I don’t know why ) after each ping >lost, alerts are triggered. > > >Actually I have only two Alerts Checks : > > >Device Down device_status equals 0 * * >Device UP device_status equals 1 * >* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >It’s possible to add a delay before triggering ? Like waiting 5 ping >lost before triggering ? > > > > >Regards. > > > > >Pierrick Prost > > > > > > > > > > > > > > > > > > > > > > > > >________________________________ > >observium mailing list >observium(a)observium.org<mailto:observium@observium.org> >http://postman.memetic.org/cgi-bin/mailman/listinfo/observium > > >------------------------------------------------------------------------ > >_______________________________________________ >observium mailing list >observium(a)observium.org >http://postman.memetic.org/cgi-bin/mailman/listinfo/observium -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://postman.memetic.org/pipermail/observium/attachments/20160627/78abc42…> ------------------------------ Message: 5 Date: Mon, 27 Jun 2016 09:33:12 +0000 From: prost pierrick <pierrick.prost(a)cnrs.fr> To: Observium Network Observation System <observium(a)observium.org> Subject: Re: [Observium] Doc about IPMI (for Dell Server) Message-ID: <AA561185F8EC764C88F0BA9DB4F2948E22681012(a)CNMB02WVP.core-res.rootcore.local> Content-Type: text/plain; charset="utf-8" Ho …it’s Monday ..sorry ☺ Regards. Pierrick De : observium [mailto:observium-bounces@observium.org] De la part de Adam Armstrong Envoyé : lundi 27 juin 2016 11:15 À : Observium Network Observation System Cc : observium(a)observium.org Objet : Re: [Observium] Doc about IPMI (for Dell Server) Did you miss the conversation people had about ipmi last week? OK Adam. Sent from BlueMail<http://www.bluemail.me/r> On 27 Jun 2016, at 10:03, prost pierrick <pierrick.prost(a)cnrs.fr<mailto:pierrick.prost@cnrs.fr>> wrote: Hi, Nobody use IMPI ? regards De : prost pierrick Envoyé : vendredi 24 juin 2016 15:04 À : observium(a)observium.org<mailto:observium@observium.org> Objet : Doc about IPMI (for Dell Server) Importance : Haute Hi everyone, There is documentation about IPMI on observium ? what the advantage ? benefit ? I have several DELL server with IDRAC and IPMI and only monitored my server with SNMP. Have a good day. ________________________________ observium mailing list observium(a)observium.org<mailto:observium@observium.org> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://postman.memetic.org/pipermail/observium/attachments/20160627/4304824…> ------------------------------ Message: 6 Date: Mon, 27 Jun 2016 09:39:34 +0000 From: "Moerman, Maarten" <mmoerman(a)ebay.com> To: Observium Network Observation System <observium(a)observium.org> Subject: Re: [Observium] Delay the alert trigger in case of loss only one ping Message-ID: <3DA5C4D1-F878-4D4A-AADB-27E10BC44F86(a)ebay.com> Content-Type: text/plain; charset="utf-8" Hi, my 2 cents: I have a delay of 1 on my device_down alert, just to prevent false positives. SNMP just sometimes has a hickup. Although usually I don’t have these, but in order to prevent panic/heart attacks, I did introduce that 1 time delay on my device down alerts. Haven you applied alert checkers on port-errors all around your network as well? CRC/port errors, and things like these you might want to fix first before you actually get in to device down alerting. Just a simple "ifInErrors_rate gt 0 & ifOutErrors_rate gt 0 on ifType equals ethernetCsmacd” alert-check will give you all these ports. you’d be surprised how much stuff is wrong in an average network. Also certainly check the port to which your observium machine is connected to, and make sure to monitor your observium machine as well, make sure your machine is well specced out. In addition to this, I’ve configured IP SLA on all our core routers to see/verify that latency on our links is within the SLA parameters of our suppliers. Do this before you check on devices down. Maarten -- Maarten Moerman | Mgr, Network Engineering | eBay Classifieds | +31-655122247 | mmoerman(a)ebay.com<mailto:mmoerman@ebay.com> From: observium <observium-bounces(a)observium.org<mailto:observium-bounces@observium.org>> on behalf of Adam Armstrong <adama(a)memetic.org<mailto:adama@memetic.org>> Reply-To: Observium Network Observation System <observium(a)observium.org<mailto:observium@observium.org>> Date: Monday, June 27, 2016 at 11:16 AM To: Observium Network Observation System <observium(a)observium.org<mailto:observium@observium.org>> Subject: Re: [Observium] Delay the alert trigger in case of loss only one ping Delay is in poller intervals. If you're getting notifications that your devices are down, you have some pretty serious network problems, as it takes quite a lot to actually mark something as down. Adam. Sent from BlueMail<http://www.bluemail.me/r> On 27 Jun 2016, at 10:07, prost pierrick <pierrick.prost(a)cnrs.fr<mailto:pierrick.prost@cnrs.fr>> wrote: Hello, I have make some tries but I’m still spammed with fake alerting. Delay value is in minute ? How can I debug poller to check if I really lost ping ? (I’m on a 10 gbit backbone network.. I’m really surprise). Regards pierrick De : prost pierrick Envoyé : jeudi 23 juin 2016 12:36 À : Observium Network Observation System Objet : RE: [Observium] Delay the alert trigger in case of loss only one ping Delay ..yes i see it ! Thanks have a good day. De : observium [mailto:observium-bounces@observium.org] De la part de Adam Armstrong Envoyé : jeudi 23 juin 2016 12:21 À : Observium Network Observation System Cc : observium(a)observium.org<mailto:observium@observium.org> Objet : Re: [Observium] Delay the alert trigger in case of loss only one ping it isn't just one ping, it tries multiple before it marks a device as down. You can set the delay field, which will cause the alert to wait for x number of down periods before it triggers. Adam. Sent from BlueMail<http://www.bluemail.me/r> On 23 Jun 2016, at 10:39, prost pierrick <pierrick.prost(a)cnrs.fr<mailto:pierrick.prost@cnrs.fr>> wrote: Hi everyone , Observium lost 1 ping sometime (I don’t know why ) after each ping lost, alerts are triggered. Actually I have only two Alerts Checks : Device Down device_status equals 0 * * Device UP device_status equals 1 * * It’s possible to add a delay before triggering ? Like waiting 5 ping lost before triggering ? Regards. Pierrick Prost ________________________________ observium mailing list observium(a)observium.org<mailto:observium@observium.org> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium ________________________________ observium mailing list observium(a)observium.org<mailto:observium@observium.org> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://postman.memetic.org/pipermail/observium/attachments/20160627/3b658a9…> ------------------------------ Subject: Digest Footer _______________________________________________ observium mailing list observium(a)observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium ------------------------------ End of observium Digest, Vol 71, Issue 24 *****************************************

1 0

Re: [Observium] Delay the alert trigger in case of loss only one ping
by ron＠rjr-services.com 27 Jun '16

27 Jun '16

I'll echo this sentiment. We were seeing a large number of false positives because of lack of SNMP response from some devices even though they were responding to pings without problem. Setting a delay of 1 reduced the number of alerts we were seeing substantially. We do still get a couple devices that will fail both checks, usually right after the discovery run, and we'll get an alarm on that, but it clears itself next poll. Damn Cat6509s and Nexus 1000Vs. ;-) Sent from my Verizon 4G LTE smartphone -------- Original message -------- From: "Moerman, Maarten" <mmoerman(a)ebay.com> Date: 6/27/16 4:39 AM (GMT-06:00) To: Observium Network Observation System <observium(a)observium.org> Subject: Re: [Observium] Delay the alert trigger in case of loss only one ping -- This message has been scanned by E.F.A. Project and is believed to be clean.

1 0

Re: [Observium] Doc about IPMI (for Dell Server)
by prost pierrick 27 Jun '16

27 Jun '16

Hi, Nobody use IMPI ? regards De : prost pierrick Envoyé : vendredi 24 juin 2016 15:04 À : observium(a)observium.org Objet : Doc about IPMI (for Dell Server) Importance : Haute Hi everyone, There is documentation about IPMI on observium ? what the advantage ? benefit ? I have several DELL server with IDRAC and IPMI and only monitored my server with SNMP. Have a good day.

2 2

Doc about IPMI (for Dell Server)
by prost pierrick 24 Jun '16

24 Jun '16

Hi everyone, There is documentation about IPMI on observium ? what the advantage ? benefit ? I have several DELL server with IDRAC and IPMI and only monitored my server with SNMP. Have a good day.

3 2

Re: [Observium] One of my Apache servers do not show [SOLVED]
by Bernard Savonet (M) 24 Jun '16

24 Jun '16

Hi Peter, Thx for your answer and its check-list... which helped me to find my mistake and to solve the problem when you run the agent manually on the affected server, does it return an output /usr/lib/observium_agent/local/apache <<<apache>>> 56775 No. If not, do you use vhosts ? Yes B-) but I had not enabled the default site. As soon as I did, things worked perfectly Date: Fri, 24 Jun 2016 08:50:20 +1000 From: Peter.Hine(a)familycourt.gov.au To: Observium Network Observation System <observium(a)observium.org> Subject: Re: [Observium] One of my Apache servers do not show [SEC=UNCLASSIFIED] when you run the agent manually on the affected server, does it return an output /usr/lib/observium_agent/local/apache <<<apache>>> 56775 [...] >>> If not, do you use vhosts ? does mod_security.conf need updating to allow access from localhost. do you get a perl error on LWP:Simple ? thanks

1 0

Graphs with Color Coded Percent Utilization
by Gumilar Siegfried 24 Jun '16

24 Jun '16

Hello, I am currently trying to completely replace our cacti installation with Observium. One of the last things that make me hold on to cacti are these kind of graphs: [cid:image001.gif@01D1C580.2CFF4E30] The graphs are described here: http://forums.cacti.net/viewtopic.php?f=6&t=8520 We have put the most important interfaces on our NOC screen and can immediately recognize if some interfaces are under unusual high load. Is there a way I can achieve something similar with Observium? Thanks, Regards Siegfried

4 6

One of my Apache servers do not show
by Bernard Savonet (M) 24 Jun '16

24 Jun '16

Hi, I've recently added some hosts to my monitored services. - All are running Apache 2.4 so I applied the Apache configuration as described at http://www.observium.org/docs/apps/#apache - All except one show the Apache application info as expected. - Except for the missing Apache info, this site is correctly reported (so general snmp / xinet config are fine) - I have carefully reviewed the different points for this host and Apache config, all is nominal. Which are the points I might have done wrong in the checklist? Which other points not in the checklist should I look at? Thx for your help Bernard

2 1

Alert Checker for memory?
by Dano, Leo 23 Jun '16

23 Jun '16

I created an alert to let me know when a host uses over 90% of its memory. The problem is that linux uses over that in "cached" memory. Is there a way to exclude cached memory? Physical and Swap are fine. I am using mempool_perc greater 90 in Check Conditions. Thanks, ----------------------- Leo Dano

3 6

Alert Checkers and Syslog Rules?
by Dano, Leo 23 Jun '16

23 Jun '16

Hello all, I am able to edit/delete my Alert Checkers but cannot edit or delete my Syslog Rules that I made. Any suggestions? Thanks, ----------------------- Leo Dano

2 8