Fwiw, we have a similar issue in that existing RRD files can be updated from both our pollers but only one poller can create NEW RRD files.
For now we workaround this by polling every device from the ‘working’ poller periodically to create any new RRDs.
[cid:image001.jpg@01CE9215.D39244B0]https://www.it.ox.ac.uk/
Richard Wigzell Technical Lead, Storage Platforms IT Services | University of Oxford 13 Banbury Road, Oxford OX2 6NN
T: 01865 616282 www.it.ox.ac.ukhttps://www.it.ox.ac.uk/
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Christian Eriksson Sent: 02 August 2018 14:24 To: Mike Stupalov mike@observium.org; Observium observium@observium.org Subject: Re: [Observium] rrdcached
The file does not exist on the rrdcached server
Vänliga hälsningar / Best regards,
[Advania_Trappa_CMYK]
Christian Eriksson System Consultant
[cid:image017.png@01D42A6D.799BEA10]mailto:christian.eriksson@advania.se[Advania-Bikupa Facebook]https://www.facebook.com/advania.se/[Advania-Bikupa LinkedIn]https://www.linkedin.com/company/advania-ab/[Advania-Bikupa Twitter]https://twitter.com/advania_se
+4641090488tel:+4641090488 | +46761400400tel:+46761400400 www.advania.sehttp://www.advania.se/ | Stockholmhttps://www.google.se/maps/place/Advania+Sverige+AB/@59.3062176,18.0278762,16.58z/data=!4m5!3m4!1s0x0:0x3e74e9d066786de7!8m2!3d59.3059982!4d18.0291626
Från: Mike Stupalov <mike@stupalov.rumailto:mike@stupalov.ru> För Mike Stupalov Skickat: den 2 augusti 2018 15:21 Till: Observium <observium@observium.orgmailto:observium@observium.org>; Christian Eriksson <christian.eriksson@advania.semailto:christian.eriksson@advania.se> Ämne: Re: [Observium] rrdcached
Show file permissions:
ls -l /opt/observium/rrd/device.net.ad.customer.se/ping.rrd Christian Erikssonmailto:christian.eriksson@advania.se 2 August 2018 at 14:48 Attached poller.php -d for one of the devices RRD /opt/observium/rrd/device.net.ad.customer.se/ping.rrd already exists - no need to create. ERROR: rrdcached: RRD Error: opening '/opt/observium/rrd/device.net.ad.customer.se/ping.rrd': No such file or directory
”rrdtool info” gives me: root@server:/opt/observium# rrdtool info --daemon device/ping.rrd ERROR: rrdcached: No such file: /opt/observium/rrd/device/ping.rrd.
The same command for uptime.rrd works
Vänliga hälsningar / Best regards,
[Advania_Trappa_CMYK]
Christian Eriksson System Consultant
[cid:image023.png@01D42A6D.799BEA10]mailto:christian.eriksson@advania.se[Advania-Bikupa Facebook]https://www.facebook.com/advania.se/[Advania-Bikupa LinkedIn]https://www.linkedin.com/company/advania-ab/[Advania-Bikupa Twitter]https://twitter.com/advania_se
+4641090488tel:+4641090488 | +46761400400tel:+46761400400 www.advania.sehttp://www.advania.se/ | Stockholmhttps://www.google.se/maps/place/Advania+Sverige+AB/@59.3062176,18.0278762,16.58z/data=%214m5%213m4%211s0x0:0x3e74e9d066786de7%218m2%213d59.3059982%214d18.0291626
Från: Christian Eriksson Skickat: den 2 augusti 2018 13:11 Till: Observium observium@observium.orgmailto:observium@observium.org Ämne: rrdcached
Hi guys,
Hope someone here can shed some light on my issue with rrdcached and observium. I am using a distributed setup with web/sql/poller on separate machines. Followed the guide here http://blog.best-practice.se/2017/10/scaling-observium-horizontally.html and currently using two pollers with great success overall
What I can’t seem to figure out is why some rrd files won’t get created. It doesn’t matter what poller I am using or what devices, it is always the following rrd-files ping.rrd ping_snmp.rrd status.rrd
[cid:image028.png@01D42A6D.799BEA10][cid:image029.png@01D42A6D.799BEA10]
Some information regarding my system
Observium 18.7.9354 (30th July 2018) OS Linux 4.4.0-130-generic [amd64] (Ubuntu 16.04) Apache 2.4.18 (Ubuntu) PHP 7.0.30-0ubuntu0.16.04.1 (OPcache: ENABLED) Python 2.7.12 MySQL 5.7.23-0ubuntu0.16.04.1 (extension: mysqli 5.0.12-dev) SNMP NET-SNMP 5.7.3 RRDtool 1.5.5 (rrdcached 1.5.5: unix:/var/run/rrdcached.sock) Fping 3.13 (IPv4 and IPv6)
rrdcached settings $config['rrdcached'] = 'ip address'; $config['rrd']['no_local'] = TRUE;
root@server:/home/user# ps aux | grep rrdcached observi+ 13596 1.6 0.5 1727784 45980 ? Ssl Aug01 15:07 /usr/bin/rrdcached -B, -F, -O, -R -L -w 1800 -z 1800 -t 8 -b /opt/observium/rrd/ -j /var/lib/rrdcached/journal/ -G www-data -U observium -p /var/run/rrdcached.pid -s www-data -m 0660 -l unix:/var/run/rrdcached.sock
rrdcached journal file show no trace of the files
Syslog errors on rrdcached server Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device1.net.ad.handels.se/status.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device1.net.ad.handels.se/ping.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device2/status.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device1.net.ad.handels.se/ping_snmp.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device2/ping.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device2/ping_snmp.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device3/status.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device3/ping.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device3/ping_snmp.rrd) failed.
I can manually create a rrd file from the poller with success rrdtool create device/status.rrd --daemon ip-address RRA:AVERAGE:0.5:1:2016 DS:status:GAUGE:600:-20000:U
Best regards Christian Eriksson
_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium Christian Erikssonmailto:christian.eriksson@advania.se 2 August 2018 at 14:10 Hi guys,
Hope someone here can shed some light on my issue with rrdcached and observium. I am using a distributed setup with web/sql/poller on separate machines. Followed the guide here http://blog.best-practice.se/2017/10/scaling-observium-horizontally.html and currently using two pollers with great success overall
What I can’t seem to figure out is why some rrd files won’t get created. It doesn’t matter what poller I am using or what devices, it is always the following rrd-files ping.rrd ping_snmp.rrd status.rrd
[cid:image028.png@01D42A6D.799BEA10][cid:image029.png@01D42A6D.799BEA10]
Some information regarding my system
Observium 18.7.9354 (30th July 2018) OS Linux 4.4.0-130-generic [amd64] (Ubuntu 16.04) Apache 2.4.18 (Ubuntu) PHP 7.0.30-0ubuntu0.16.04.1 (OPcache: ENABLED) Python 2.7.12 MySQL 5.7.23-0ubuntu0.16.04.1 (extension: mysqli 5.0.12-dev) SNMP NET-SNMP 5.7.3 RRDtool 1.5.5 (rrdcached 1.5.5: unix:/var/run/rrdcached.sock) Fping 3.13 (IPv4 and IPv6)
rrdcached settings $config['rrdcached'] = 'ip address'; $config['rrd']['no_local'] = TRUE;
root@server:/home/user# ps aux | grep rrdcached observi+ 13596 1.6 0.5 1727784 45980 ? Ssl Aug01 15:07 /usr/bin/rrdcached -B, -F, -O, -R -L -w 1800 -z 1800 -t 8 -b /opt/observium/rrd/ -j /var/lib/rrdcached/journal/ -G www-data -U observium -p /var/run/rrdcached.pid -s www-data -m 0660 -l unix:/var/run/rrdcached.sock
rrdcached journal file show no trace of the files
Syslog errors on rrdcached server Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device1.net.ad.handels.se/status.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device1.net.ad.handels.se/ping.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device2/status.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device1.net.ad.handels.se/ping_snmp.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device2/ping.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device2/ping_snmp.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device3/status.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device3/ping.rrd) failed. Aug 2 13:01:01 server rrdcached[13596]: handle_request_update: stat (/opt/observium/rrd/device3/ping_snmp.rrd) failed.
I can manually create a rrd file from the poller with success rrdtool create device/status.rrd --daemon ip-address RRA:AVERAGE:0.5:1:2016 DS:status:GAUGE:600:-20000:U
Best regards Christian Eriksson
_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
-- Mike Stupalov Observium Limited, http://observium.org
