Hi all,
Are you not sleeping well because there are gaps in your graphs? Do you feel like your observium poller isn't putting enough effort into its sole task of polling every device and completing that task in less than 5 minutes?
Fear not, a dirty script called "poller-wrapper.py" is here!
I wrote a small wrapper that has a slightly more modern approach to getting things done. An issue with the multiple poller instances approach[1] is that when a single instance has to poll a few slow devices, that single instance cannot finish its work in time, while the instances that already finished their portion of the work will not 'help' the instance with the slow devices.
This wrapper script will create a Queue with all the work that has to be done and launch a number of threads which will take work from the Queue until the queue is empty. This distributes the workload over the instances move evenly, thus improving performance and thus making it more likely to finish all the polling work within 5 minutes.
You can easily test what number of threads works well with your environment by running the script a few times manually, here I run the script with 32 threads:
root@observium:/opt/observium# python poller-wrapper.py 32 worker Thread-23 finished device 75 in 9 seconds worker Thread-25 finished device 84 in 2 seconds worker Thread-21 finished device 81 in 5 seconds <SNIP> worker Thread-23 finished device 98 in 81 seconds worker Thread-3 finished device 94 in 116 seconds worker Thread-32 finished device 93 in 126 seconds poller-wrapper polled 130 devices in 179 seconds with 32 workers root@observium:/opt/observium#
Source: -------
URL: http://noc.as5580.net/~job/observium-poller-wrapper.py.txt
Installation: -------------
cd /opt/observium wget http://noc.as5580.net/~job/observium-poller-wrapper.py.txt -O poller-wrapper.py
Now open poller-wrapper.py with a text editor to change the database username & password.
open /etc/cron.d/observium and replace this line: */5 * * * * root /opt/observium/poller.php -h all >> /dev/null 2>&1 with something like this: */5 * * * * root python /opt/observium/poller-wrapper.py 16 >> /dev/null 2>&1
Kind regards,
Job Snijders Atrato IP Networks
[1] http://www.observium.org/wiki/Performance_tuning#Multiple_poller_instances