Robert,
Can you send a full debug log?
Thanks, Tom
On 08/31/2016 10:09 AM, Robert Williams wrote:
Hi,
Just noticed this stayed broken, I had forgotten about it!
So, done some further digging and found a related error message revealed by *journalctl*:
Aug 30 16:29:22 OBSERVIUS rrdcached[3392]: queue_thread_main: rrd_update_r (/opt/observium/rrd/xxx.yyy.zzz/app-bind-31-cache-default.rrd) failed with status -1. (/opt/observium/rrd/xxx.yyy.zzz/app-bind-31-cache-default.rrd) failed with status -1. (/opt/observium/rrd/xxx.yyy.zzz/app-bind-31-cache-default.rrd: Function update_pdp_prep, *case DST_GAUGE - Cannot convert '' to float*)
It appears that there is something in the parsing of the data to the rrd file which IS acceptable when you do it with RRDTOOL direclty, but the same thing is NOT acceptable when you do it via RRDCACHED. Looks like a blank value that can’t be written to a float. Maybe it wants a ‘0’ not a null or something?
Is it feasible that this can be resolved with a tweak on the Observium side? There is little I can do to modify the way RRDCACHED decides if it is valid or not :(
Would be nice to have all those stats back again - Cheers!
Robert Williams Custodian Data Centre Email: Robert@CustodianDC.com http://www.CustodianDC.com
*From:*observium [mailto:observium-bounces@observium.org] *On Behalf Of *Robert Williams *Sent:* 08 July 2016 18:12 *To:* Observium Network Observation System (observium@observium.org) observium@observium.org *Subject:* [Observium] bind_cache failing to write via RRDCached
In summary, all servers running BIND (they are still on 14.04) have now got zero ‘cache content’ stats since the Observium server was upgraded on Tuesday.
However, all the other BIND stats are working OK:
Looking at the debug output, it is getting the data fine, and it’s writing out the values to RRDCacheD OK.
I tried disabling RRDCacheD – it started working straight away. So, it looks like RRDCacheD (version 1.5.5 under 16.04) is unhappy with writing the ‘Cache’ elements content output for BIND.
The write attempt is:
mupdate /opt/observium/rrd/x.y.net/app-bind-23-cache-default.rrd N:6:3:::392500:799:13060:27296:825744:29:17815:555:67263:160:8::9660:2238:49:382:23:14:::::::259181:22:2::45094::26347::::::6715:::::::
My old RRDCacheD config was:
DISABLE=0
OPTS="-w 1800 -z 1800 -f 3600 -s www-data -l unix:/var/run/rrdcached.sock -j /var/lib/rrdcached/journal/ -F -b /opt/observium/rrd -B"
MAXWAIT=30
ENABLE_COREFILES=0
The new config has different parameters, so I’ve setup as follows:
DAEMON=/usr/bin/rrdcached
WRITE_TIMEOUT=1800
WRITE_JITTER=1800
BASE_PATH=/opt/observium/rrd/
JOURNAL_PATH=/var/lib/rrdcached/journal/
PIDFILE=/var/run/rrdcached.pid
SOCKFILE=/var/run/rrdcached.sock
SOCKGROUP=www-data
BASE_OPTIONS="-F -B"
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
