Yeah, I usually even out the server load by splitting the number of devices in 5 and start the polling of them 1 minute apart with cron like this:

0-59/5 * * * * observium /opt/observium/observium-wrapper polling -i 5 -n 0 >> /dev/null 2>&1

1-59/5 * * * * observium /opt/observium/observium-wrapper polling -i 5 -n 1 >> /dev/null 2>&1

2-59/5 * * * * observium /opt/observium/observium-wrapper polling -i 5 -n 2 >> /dev/null 2>&1

3-59/5 * * * * observium /opt/observium/observium-wrapper polling -i 5 -n 3 >> /dev/null 2>&1

4-59/5 * * * * observium /opt/observium/observium-wrapper polling -i 5 -n 4 >> /dev/null 2>&1

This makes it work with a lot fewer threads and the load on disk and database is much lower as only 1/5 of the data needs to be progressed at the same time.

/Markus

2018-07-12 7:09 GMT+02:00 Adam Armstrong <adama@memetic.org>:

You should probably decide threads to even out the load. The goal is to be as close to 300 seconds as you can manage, else you'll get spiky io and myself load.

Adam.

Sent from BlueMail
On 11 Jul 2018, at 07:18, "Balistreri, Thomas C - DOC" <thomas.balistreri@wisconsin.gov> wrote:
Wow! What hardware are you using? Where are your devices located? I’m running about 90s for around 900 devices (21,000+ ports), but I have to transverse WAN links for 95% of it.

Tommy

From: observium [mailto:observium-bounces@observium.org] On Behalf Of Markus Klock
Sent: Wednesday, July 11, 2018 8:57 AM
To: Observium Network Observation System <observium@observium.org>
Subject: [Observium] New record?

Hi guys,

just wanted to show the new Observium I deployed for a customer this week :)

Everything polled from one single server without problems (pollingtime is about 140s for all devices)

Big thanks to Mike who helped me identify and fix a scaling issue when running a huge amount of threads. (if you run an Observium-install that runs poller-wrapper with 100+ threads you should check out r9333)

/Markus

CONFIDENTIALITY NOTICE: This electronic mail transmission and any accompanying documents contain information belonging to the sender which may be confidential and legally privileged. This information is only for the use of the individual or entity to whom this electronic mail transmission was intended. If you are not the intended recipient, any disclosure, copying, distribution, or action taken in reliance on the contents of the information contained in this transmission is strictly prohibited. If you have received this transmission in error, please immediately contact the sender and delete the message. Thank you.
 

observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium