I hope the graphs show up below, thanks for your help:
[cid:image001.png@01D5C244.174B6780]
[cid:image002.png@01D5C244.CC3CD0C0]
[cid:image003.png@01D5C244.CC3CD0C0]
[cid:image004.png@01D5C244.CC3CD0C0]
Please provide a screenshot of the "Polling information"-page.
And if you monitor the Observium-VM itself with Observium then go to that device, Graphs>Processor>Extended Processor Utilisation and screenshot that graph too please
/Markus
-----Original Message----- From: observium observium-bounces@observium.org On Behalf Of observium-request@observium.org Sent: Friday, January 3, 2020 4:00 AM To: observium@observium.org Subject: observium Digest, Vol 114, Issue 2
Send observium mailing list submissions to
observium@observium.orgmailto:observium@observium.org
To subscribe or unsubscribe via the World Wide Web, visit
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
or, via email, send a message with subject or body 'help' to
observium-request@observium.orgmailto:observium-request@observium.org
You can reach the person managing the list at
observium-owner@observium.orgmailto:observium-owner@observium.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of observium digest..."
Today's Topics:
1. Gaps in Graphs (Justin Radke)
2. Re: Gaps in Graphs (Michael)
3. Re: Gaps in Graphs (Justin Radke)
4. Re: Gaps in Graphs (Markus Klock)
----------------------------------------------------------------------
Message: 1
Date: Thu, 2 Jan 2020 20:29:29 +0000
From: Justin Radke <justin.radke@directlink.coopmailto:justin.radke@directlink.coop>
To: "observium@observium.orgmailto:observium@observium.org" <observium@observium.orgmailto:observium@observium.org>
Subject: [Observium] Gaps in Graphs
Message-ID:
<MWHPR06MB3374668112EE9110EE59D63782200@MWHPR06MB3374.namprd06.prod.outlook.commailto:MWHPR06MB3374668112EE9110EE59D63782200@MWHPR06MB3374.namprd06.prod.outlook.com>
Content-Type: text/plain; charset="utf-8"
Hello, periodically I have "gaps" in my graphs which appears to be when Observium was unable to reach a particular host. This problem occurs on multiple hosts at the same time on different network segments and I have been unable to find any commonality except for the Observium server itself. I noticed that the gaps in the graph also exist at the same times for Observium's checks of localhost. Checking localhost of course demonstrates this is not a lack of network connectivity outside of the server.
At these times I also found these entries but I'm not sure if they are related since they are merely syslog messages but thought I'd mention it:
Jan 1 21:39:24 voldemort rsyslogd-2177: imuxsock begins to drop messages from pid 27161 due to rate-limiting Jan 1 21:45:50 voldemort rsyslogd-2177: imuxsock lost 22 messages from pid 27161 due to rate-limiting
root@observium(~)
:ps auxw | grep 27161
root 27161 0.0 0.0 200848 6520 ? S 2019 15:32 /usr/sbin/snmpd -LS0-6d -Lf /dev/null -p /var/run/snmpd.pid
What would cause this and how can I solve this matter?
-=JGR
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://postman.memetic.org/pipermail/observium/attachments/20200102/d19edce8/attachment-0001.html
------------------------------
Message: 2
Date: Fri, 03 Jan 2020 08:17:38 +1100
From: Michael <obslist@smarsz.commailto:obslist@smarsz.com>
To: Observium <observium@observium.orgmailto:observium@observium.org>, Justin Radke via observium
<observium@observium.orgmailto:observium@observium.org>, "observium@observium.orgmailto:observium@observium.org"
<observium@observium.orgmailto:observium@observium.org>
Cc: Justin Radke <justin.radke@directlink.coopmailto:justin.radke@directlink.coop>
Subject: Re: [Observium] Gaps in Graphs
Message-ID: <930E4B10-85FA-4268-925D-B70E38AF1E50@smarsz.commailto:930E4B10-85FA-4268-925D-B70E38AF1E50@smarsz.com>
Content-Type: text/plain; charset=utf-8
It sounds like you are getting a large amount of udp traffic at one time.
We run our observium server as a vmware guest. We had to do a fair bit of tuning to get udp and rx buffers to a point where we weren't losing responses (& in your case, syslog messages).
Might be worth looking into.
Michael
On 3 January 2020 7:29:29 AM AEDT, Justin Radke via observium <observium@observium.orgmailto:observium@observium.org> wrote:
Hello, periodically I have "gaps" in my graphs which appears to be when
Observium was unable to reach a particular host. This problem occurs on
multiple hosts at the same time on different network segments and I
have been unable to find any commonality except for the Observium
server itself. I noticed that the gaps in the graph also exist at the
same times for Observium's checks of localhost. Checking localhost of
course demonstrates this is not a lack of network connectivity outside
of the server.
At these times I also found these entries but I'm not sure if they are
related since they are merely syslog messages but thought I'd mention
it:
Jan 1 21:39:24 voldemort rsyslogd-2177: imuxsock begins to drop
messages from pid 27161 due to rate-limiting Jan 1 21:45:50 voldemort
rsyslogd-2177: imuxsock lost 22 messages from pid 27161 due to
rate-limiting
root@observium(~)
:ps auxw | grep 27161
root 27161 0.0 0.0 200848 6520 ? S 2019 15:32
/usr/sbin/snmpd -LS0-6d -Lf /dev/null -p /var/run/snmpd.pid
What would cause this and how can I solve this matter?
-=JGR
------------------------------
Message: 3
Date: Thu, 2 Jan 2020 22:05:22 +0000
From: Justin Radke <justin.radke@directlink.coopmailto:justin.radke@directlink.coop>
To: Michael <obslist@smarsz.commailto:obslist@smarsz.com>, Observium <observium@observium.orgmailto:observium@observium.org>,
Justin Radke via observium <observium@observium.orgmailto:observium@observium.org>,
"observium@observium.orgmailto:observium@observium.org" <observium@observium.orgmailto:observium@observium.org>
Subject: Re: [Observium] Gaps in Graphs
Message-ID:
<MWHPR06MB33746498C7B0EAB0DCDCADD382200@MWHPR06MB3374.namprd06.prod.outlook.commailto:MWHPR06MB33746498C7B0EAB0DCDCADD382200@MWHPR06MB3374.namprd06.prod.outlook.com>
Content-Type: text/plain; charset="utf-8"
Thank you for your suggestion! Our Observium installation is virtualized using KVM on CentOS. Can you share what changes you needed to make? Did you have to make your changes on the Linux host or guest? Or was it a VMware nerd knob?
-=JGR
-----Original Message-----
From: Michael <obslist@smarsz.commailto:obslist@smarsz.com>
Sent: Thursday, January 2, 2020 1:18 PM
To: Observium <observium@observium.orgmailto:observium@observium.org>; Justin Radke via observium <observium@observium.orgmailto:observium@observium.org>; observium@observium.orgmailto:observium@observium.org
Cc: Justin Radke <justin.radke@directlink.coopmailto:justin.radke@directlink.coop>
Subject: Re: [Observium] Gaps in Graphs
It sounds like you are getting a large amount of udp traffic at one time.
We run our observium server as a vmware guest. We had to do a fair bit of tuning to get udp and rx buffers to a point where we weren't losing responses (& in your case, syslog messages).
Might be worth looking into.
Michael
On 3 January 2020 7:29:29 AM AEDT, Justin Radke via observium <observium@observium.orgmailto:observium@observium.org> wrote:
Hello, periodically I have "gaps" in my graphs which appears to be when
Observium was unable to reach a particular host. This problem occurs on
multiple hosts at the same time on different network segments and I
have been unable to find any commonality except for the Observium
server itself. I noticed that the gaps in the graph also exist at the
same times for Observium's checks of localhost. Checking localhost of
course demonstrates this is not a lack of network connectivity outside
of the server.
At these times I also found these entries but I'm not sure if they are
related since they are merely syslog messages but thought I'd mention
it:
Jan 1 21:39:24 voldemort rsyslogd-2177: imuxsock begins to drop
messages from pid 27161 due to rate-limiting Jan 1 21:45:50 voldemort
rsyslogd-2177: imuxsock lost 22 messages from pid 27161 due to
rate-limiting
root@observium(~)
:ps auxw | grep 27161
root 27161 0.0 0.0 200848 6520 ? S 2019 15:32
/usr/sbin/snmpd -LS0-6d -Lf /dev/null -p /var/run/snmpd.pid
What would cause this and how can I solve this matter?
-=JGR
------------------------------
Message: 4
Date: Fri, 3 Jan 2020 08:07:53 +0100
From: Markus Klock <markus@best-practice.semailto:markus@best-practice.se>
To: Observium <observium@observium.orgmailto:observium@observium.org>
Subject: Re: [Observium] Gaps in Graphs
Message-ID:
<CAF6uZm+fVgrJq8YxVnN1Bh=kE6UtQWkUKrB4mi3iH_PSvXS7mQ@mail.gmail.commailto:CAF6uZm+fVgrJq8YxVnN1Bh=kE6UtQWkUKrB4mi3iH_PSvXS7mQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Please provide a screenshot of the "Polling information"-page.
And if you monitor the Observium-VM itself with Observium then go to that device, Graphs>Processor>Extended Processor Utilisation and screenshot that graph too please
/Markus
Den tors 2 jan. 2020 kl 23:05 skrev Justin Radke via observium <
observium@observium.orgmailto:observium@observium.org>:
Thank you for your suggestion! Our Observium installation is
virtualized using KVM on CentOS. Can you share what changes you needed
to make? Did you have to make your changes on the Linux host or guest?
Or was it a VMware nerd knob?
-=JGR
-----Original Message-----
From: Michael <obslist@smarsz.commailto:obslist@smarsz.com>
Sent: Thursday, January 2, 2020 1:18 PM
To: Observium <observium@observium.orgmailto:observium@observium.org>; Justin Radke via observium <
observium@observium.orgmailto:observium@observium.org>; observium@observium.orgmailto:observium@observium.org
Cc: Justin Radke <justin.radke@directlink.coopmailto:justin.radke@directlink.coop>
Subject: Re: [Observium] Gaps in Graphs
It sounds like you are getting a large amount of udp traffic at one time.
We run our observium server as a vmware guest. We had to do a fair bit
of tuning to get udp and rx buffers to a point where we weren't losing
responses (& in your case, syslog messages).
Might be worth looking into.
Michael
On 3 January 2020 7:29:29 AM AEDT, Justin Radke via observium <
observium@observium.orgmailto:observium@observium.org> wrote:
Hello, periodically I have "gaps" in my graphs which appears to be
when Observium was unable to reach a particular host. This problem
occurs on multiple hosts at the same time on different network
segments and I have been unable to find any commonality except for
the Observium server itself. I noticed that the gaps in the graph
also exist at the same times for Observium's checks of localhost.
Checking localhost of course demonstrates this is not a lack of
network connectivity outside of the server.
At these times I also found these entries but I'm not sure if they
are related since they are merely syslog messages but thought I'd
mention
it:
Jan 1 21:39:24 voldemort rsyslogd-2177: imuxsock begins to drop
messages from pid 27161 due to rate-limiting Jan 1 21:45:50
voldemort
rsyslogd-2177: imuxsock lost 22 messages from pid 27161 due to
rate-limiting
root@observium(~)
:ps auxw | grep 27161
root 27161 0.0 0.0 200848 6520 ? S 2019 15:32
/usr/sbin/snmpd -LS0-6d -Lf /dev/null -p /var/run/snmpd.pid
What would cause this and how can I solve this matter?
-=JGR
observium mailing list
observium@observium.orgmailto:observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://postman.memetic.org/pipermail/observium/attachments/20200103/3f220eb4/attachment-0001.html
------------------------------
Subject: Digest Footer
_______________________________________________
observium mailing list
observium@observium.orgmailto:observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
------------------------------
End of observium Digest, Vol 114, Issue 2
*****************************************
Hi Justin, The poller needs to finish polling all your devices in 5min(300s) and your system takes 1600s to finish one pollinground. This is because you only run the poller-wrapper with 1 thread, you should run as many threads as twice the amount of CPU-cores on this VM. Try increasing them and wait 10min and then check the poller-wrapper Total time-graph again.
/Markus
Den fre 3 jan. 2020 kl 23:48 skrev Justin Radke via observium < observium@observium.org>:
I hope the graphs show up below, thanks for your help:
Please provide a screenshot of the "Polling information"-page.
And if you monitor the Observium-VM itself with Observium then go to that device, Graphs>Processor>Extended Processor Utilisation and screenshot that graph too please
/Markus
-----Original Message----- From: observium observium-bounces@observium.org On Behalf Of observium-request@observium.org Sent: Friday, January 3, 2020 4:00 AM To: observium@observium.org Subject: observium Digest, Vol 114, Issue 2
Send observium mailing list submissions to
observium@observium.orgTo subscribe or unsubscribe via the World Wide Web, visit
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
or, via email, send a message with subject or body 'help' to
observium-request@observium.orgYou can reach the person managing the list at
observium-owner@observium.orgWhen replying, please edit your Subject line so it is more specific than "Re: Contents of observium digest..."
Today's Topics:
Gaps in Graphs (Justin Radke)
Re: Gaps in Graphs (Michael)
Re: Gaps in Graphs (Justin Radke)
Re: Gaps in Graphs (Markus Klock)
Message: 1
Date: Thu, 2 Jan 2020 20:29:29 +0000
From: Justin Radke justin.radke@directlink.coop
To: "observium@observium.org" observium@observium.org
Subject: [Observium] Gaps in Graphs
Message-ID:
<MWHPR06MB3374668112EE9110EE59D63782200@MWHPR06MB3374.namprd06.prod.outlook.com
Content-Type: text/plain; charset="utf-8"
Hello, periodically I have "gaps" in my graphs which appears to be when Observium was unable to reach a particular host. This problem occurs on multiple hosts at the same time on different network segments and I have been unable to find any commonality except for the Observium server itself. I noticed that the gaps in the graph also exist at the same times for Observium's checks of localhost. Checking localhost of course demonstrates this is not a lack of network connectivity outside of the server.
At these times I also found these entries but I'm not sure if they are related since they are merely syslog messages but thought I'd mention it:
Jan 1 21:39:24 voldemort rsyslogd-2177: imuxsock begins to drop messages from pid 27161 due to rate-limiting Jan 1 21:45:50 voldemort rsyslogd-2177: imuxsock lost 22 messages from pid 27161 due to rate-limiting
root@observium(~)
:ps auxw | grep 27161
root 27161 0.0 0.0 200848 6520 ? S 2019 15:32 /usr/sbin/snmpd -LS0-6d -Lf /dev/null -p /var/run/snmpd.pid
What would cause this and how can I solve this matter?
-=JGR
participants (2)
-
Justin Radke -
Markus Klock