Re: [Observium] Stacked and combined line graphs breaks after upgrading to 8889 stable

31 Oct 2017

      Hmm.
It seems that our code is deciding the generation failed and is sending the error image.
Adam.
⁣Sent from BlueMail 
On 31 Oct 2017, 19:52, at 19:52, Jesper Frank Nemholt jfn@dassic.com wrote:
...
OK, so the command run with no errors and deliver a png image that
actually
looks correct...so I guess that the issue may be elsewhere....but I
can't
find any errors in access_log.
[image: image.png]
On Tue, Oct 31, 2017 at 7:33 PM Adam Armstrong adama@observium.org
wrote:
...
Copy the rrdtool command and run it on the console. I'm not sure why
it's
...
not giving the correct error, bleh!
adam.
On 2017-10-31 19:29:40, Jesper Frank Nemholt jfn@dassic.com wrote:
Too big to capture all in one screenshot, so split in 3 + last
picture
...
when not stacked (where it works) :[image: image.png]
[image: image.png]
[image: image.png]
[image: image.png]
On Tue, Oct 31, 2017 at 7:14 PM Adam Armstrong adama@memetic.org
wrote:
...
...
Screenshot the entire page, including the rrd output and the image.
Sent from BlueMail http://www.bluemail.me/r?b=11012
On 31 Oct 2017, at 19:10, Jesper Frank Nemholt jfn@dassic.com
wrote:
...
...
...
Hmm, strange. I get this...technically it's an image, but the data
is
...
...
...
missing :
[image: image.png]
If I look only in the overview, so each device individually, all
works,
...
...
...
but when stacking or combining, it breaks.
/Jesper
On Tue, Oct 31, 2017 at 6:00 PM Adam Armstrong
adama@observium.org
...
...
...
wrote:
...
RRDTool Output: 1233x607
This is actually generating an image. It also works ok for me.
adam.
On 2017-10-31 17:47:55, Jesper Frank Nemholt jfn@dassic.com
wrote:
...
...
...
...
Hi!
Correct it's not a numeric OID, but it always worked, with with >
100
...
...
...
...
custom OIDs, I prefer the more human readable version, at least
for some of
...
...
...
...
it.
All worked before the upgrade to 8889 stable.
The RRD data is the following :
Performance & Output
RRDTool Output: 1233x607
RRDtool Runtime: 0.103s | Total time: 0.104s
RRDTool Command
/usr/bin/rrdtool graph /tmp/mfu0vmGXvSF9OEla.png -Y --lower-limit
0
...
...
...
...
--alt-autoscale-max --start 1509385321 --end 1509471721 --width
1152
...
...
...
...
--height 300 -c BACK#EEEEEE00 -c SHADEA#EEEEEE00 -c
SHADEB#EEEEEE00 -c
...
...
...
...
FONT#000000 -c CANVAS#FFFFFF00 -c GRID#a5a5a5 -c MGRID#FF9999 -c
FRAME#5e5e5e -c ARROW#5e5e5e -R normal --font
LEGEND:8:'DroidSansMono,DejaVuSansMono' --font
AXIS:7:'DroidSansMono,DejaVuSansMono' --font-render-mode normal -E
COMMENT:'Bits per second Now Min Max Avg' COMMENT:'\l'
DEF:value1=/home/observium/rrd/ssf-nas-is7.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value1min=/home/observium/rrd/ssf-nas-is7.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value1max=/home/observium/rrd/ssf-nas-is7.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value2=/home/observium/rrd/nas-rwc-is3.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value2min=/home/observium/rrd/nas-rwc-is3.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value2max=/home/observium/rrd/nas-rwc-is3.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value3=/home/observium/rrd/nas-rwc-is2.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value3min=/home/observium/rrd/nas-rwc-is2.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value3max=/home/observium/rrd/nas-rwc-is2.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value4=/home/observium/rrd/nas-rwc-is4.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value4min=/home/observium/rrd/nas-rwc-is4.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value4max=/home/observium/rrd/nas-rwc-is4.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value5=/home/observium/rrd/ssf-arch-is1.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value5min=/home/observium/rrd/ssf-arch-is1.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value5max=/home/observium/rrd/ssf-arch-is1.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value6=/home/observium/rrd/rnunsis01.tot.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value6min=/home/observium/rrd/rnunsis01.tot.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value6max=/home/observium/rrd/rnunsis01.tot.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value7=/home/observium/rrd/nas-rwc-is5.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value7min=/home/observium/rrd/nas-rwc-is5.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value7max=/home/observium/rrd/nas-rwc-is5.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value8=/home/observium/rrd/nas-rwc-is6.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value8min=/home/observium/rrd/nas-rwc-is6.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value8max=/home/observium/rrd/nas-rwc-is6.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value9=/home/observium/rrd/ridnsis02x.ind.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value9min=/home/observium/rrd/ridnsis02x.ind.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value9max=/home/observium/rrd/ridnsis02x.ind.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value10=/home/observium/rrd/nas-vv-is3.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value10min=/home/observium/rrd/nas-vv-is3.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value10max=/home/observium/rrd/nas-vv-is3.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value11=/home/observium/rrd/nas-vv-is2.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value11min=/home/observium/rrd/nas-vv-is2.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value11max=/home/observium/rrd/nas-vv-is2.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value12=/home/observium/rrd/ssfnsispredrep01.sc1.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value12min=/home/observium/rrd/ssfnsispredrep01.sc1.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value12max=/home/observium/rrd/ssfnsispredrep01.sc1.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value13=/home/observium/rrd/ssf-m01-is01.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value13min=/home/observium/rrd/ssf-m01-is01.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value13max=/home/observium/rrd/ssf-m01-is01.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value14=/home/observium/rrd/ssf-m02-is02.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value14min=/home/observium/rrd/ssf-m02-is02.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value14max=/home/observium/rrd/ssf-m02-is02.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value15=/home/observium/rrd/rlfnsis01.lif.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value15min=/home/observium/rrd/rlfnsis01.lif.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value15max=/home/observium/rrd/rlfnsis01.lif.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value16=/home/observium/rrd/ridnsis05x.ind.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value16min=/home/observium/rrd/ridnsis05x.ind.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value16max=/home/observium/rrd/ridnsis05x.ind.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
DEF:value17=/home/observium/rrd/nas-b41-is1.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:AVERAGE
...
...
...
...
DEF:value17min=/home/observium/rrd/nas-b41-is1.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MIN
...
...
...
...
DEF:value17max=/home/observium/rrd/nas-b41-is1.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd:value:MAX
...
...
...
...
AREA:value1#365C81:'ssf-nas-is7.foo.bar '
GPRINT:value1:LAST:%5.1lf%s
...
...
...
...
GPRINT:value1min:MIN:%5.1lf%s GPRINT:value1max:MAX:%5.1lf%s
GPRINT:value1:AVERAGE:%5.1lf%s'\n' COMMENT:'\n'
AREA:value2#D8929F:'nas-rwc-is3.foo.bar ':STACK
GPRINT:value2:LAST:%5.1lf%s
...
...
...
...
GPRINT:value2min:MIN:%5.1lf%s GPRINT:value2max:MAX:%5.1lf%s
GPRINT:value2:AVERAGE:%5.1lf%s'\n' COMMENT:'\n'
AREA:value3#A99DCB:'nas-rwc-is2.foo.bar ':STACK
GPRINT:value3:LAST:%5.1lf%s
...
...
...
...
GPRINT:value3min:MIN:%5.1lf%s GPRINT:value3max:MAX:%5.1lf%s
GPRINT:value3:AVERAGE:%5.1lf%s'\n' COMMENT:'\n'
AREA:value4#6CA4D5:'nas-rwc-is4.foo.bar ':STACK
GPRINT:value4:LAST:%5.1lf%s
...
...
...
...
GPRINT:value4min:MIN:%5.1lf%s GPRINT:value4max:MAX:%5.1lf%s
GPRINT:value4:AVERAGE:%5.1lf%s'\n' COMMENT:'\n'
AREA:value5#8BA15F:'ssf-arch-is1.foo.bar ':STACK
GPRINT:value5:LAST:%5.1lf%s GPRINT:value5min:MIN:%5.1lf%s
GPRINT:value5max:MAX:%5.1lf%s GPRINT:value5:AVERAGE:%5.1lf%s'\n'
COMMENT:'\n' AREA:value6#E9CA5D:'rnunsis01.tot.foo.bar ':STACK
GPRINT:value6:LAST:%5.1lf%s GPRINT:value6min:MIN:%5.1lf%s
GPRINT:value6max:MAX:%5.1lf%s GPRINT:value6:AVERAGE:%5.1lf%s'\n'
COMMENT:'\n' AREA:value7#DF9933:'nas-rwc-is5.foo.bar ':STACK
GPRINT:value7:LAST:%5.1lf%s GPRINT:value7min:MIN:%5.1lf%s
GPRINT:value7max:MAX:%5.1lf%s GPRINT:value7:AVERAGE:%5.1lf%s'\n'
COMMENT:'\n' AREA:value8#D33627:'nas-rwc-is6.foo.bar ':STACK
GPRINT:value8:LAST:%5.1lf%s GPRINT:value8min:MIN:%5.1lf%s
GPRINT:value8max:MAX:%5.1lf%s GPRINT:value8:AVERAGE:%5.1lf%s'\n'
COMMENT:'\n' AREA:value9#881C45:'ridnsis02x.ind.foo.bar ':STACK
GPRINT:value9:LAST:%5.1lf%s GPRINT:value9min:MIN:%5.1lf%s
GPRINT:value9max:MAX:%5.1lf%s GPRINT:value9:AVERAGE:%5.1lf%s'\n'
COMMENT:'\n' AREA:value10#C74379:'nas-vv-is3.foo.bar ':STACK
GPRINT:value10:LAST:%5.1lf%s GPRINT:value10min:MIN:%5.1lf%s
GPRINT:value10max:MAX:%5.1lf%s GPRINT:value10:AVERAGE:%5.1lf%s'\n'
COMMENT:'\n' AREA:value11#9B2F82:'nas-vv-is2.foo.bar ':STACK
GPRINT:value11:LAST:%5.1lf%s GPRINT:value11min:MIN:%5.1lf%s
GPRINT:value11max:MAX:%5.1lf%s GPRINT:value11:AVERAGE:%5.1lf%s'\n'
COMMENT:'\n' AREA:value12#345AA8:'ssfnsispredrep01.sc1.foo.bar
':STACK
...
...
...
...
GPRINT:value12:LAST:%5.1lf%s GPRINT:value12min:MIN:%5.1lf%s
GPRINT:value12max:MAX:%5.1lf%s GPRINT:value12:AVERAGE:%5.1lf%s'\n'
COMMENT:'\n' AREA:value13#88C3C9:'ssf-m01-is01.foo.bar ':STACK
GPRINT:value13:LAST:%5.1lf%s GPRINT:value13min:MIN:%5.1lf%s
GPRINT:value13max:MAX:%5.1lf%s GPRINT:value13:AVERAGE:%5.1lf%s'\n'
COMMENT:'\n' AREA:value14#519466:'ssf-m02-is02.foo.bar ':STACK
GPRINT:value14:LAST:%5.1lf%s GPRINT:value14min:MIN:%5.1lf%s
GPRINT:value14max:MAX:%5.1lf%s GPRINT:value14:AVERAGE:%5.1lf%s'\n'
COMMENT:'\n' AREA:value15#A7C662:'rlfnsis01.lif.foo.bar ':STACK
GPRINT:value15:LAST:%5.1lf%s GPRINT:value15min:MIN:%5.1lf%s
GPRINT:value15max:MAX:%5.1lf%s GPRINT:value15:AVERAGE:%5.1lf%s'\n'
COMMENT:'\n' AREA:value16#BA723D:'ridnsis05x.ind.foo.bar ':STACK
GPRINT:value16:LAST:%5.1lf%s GPRINT:value16min:MIN:%5.1lf%s
GPRINT:value16max:MAX:%5.1lf%s GPRINT:value16:AVERAGE:%5.1lf%s'\n'
COMMENT:'\n' AREA:value17#864A23:'nas-b41-is1.foo.bar ':STACK
GPRINT:value17:LAST:%5.1lf%s GPRINT:value17min:MIN:%5.1lf%s
GPRINT:value17max:MAX:%5.1lf%s GPRINT:value17:AVERAGE:%5.1lf%s'\n'
COMMENT:'\n'
CDEF:aggregate=value1,UN,0,value1,IF,value2,UN,0,value2,IF,value3,UN,0,value3,IF,value4,UN,0,value4,IF,value5,UN,0,value5,IF,value6,UN,0,value6,IF,value7,UN,0,value7,IF,value8,UN,0,value8,IF,value9,UN,0,value9,IF,value10,UN,0,value10,IF,value11,UN,0,value11,IF,value12,UN,0,value12,IF,value13,UN,0,value13,IF,value14,UN,0,value14,IF,value15,UN,0,value15,IF,value16,UN,0,value16,IF,value17,UN,0,value17,IF,+,+,+,+,+,+,+,+,+,+,+,+,+,+,+,+
...
...
...
...
LINE1.5:aggregate#000000:'Aggregate '
GPRINT:aggregate:LAST:%5.1lf%s
...
...
...
...
GPRINT:aggregate:MIN:%5.1lf%s GPRINT:aggregate:MAX:%5.1lf%s
GPRINT:aggregate:AVERAGE:%5.1lf%s'\n' COMMENT:'\n' HRULE:0#555555
RRDTool Files Used
/home/observium/rrd/ssf-nas-is7.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/nas-rwc-is3.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/nas-rwc-is2.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/nas-rwc-is4.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/ssf-arch-is1.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/rnunsis01.tot.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/nas-rwc-is5.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/nas-rwc-is6.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/ridnsis02x.ind.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/nas-vv-is3.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/nas-vv-is2.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/ssfnsispredrep01.sc1.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/ssf-m01-is01.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/ssf-m02-is02.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/rlfnsis01.lif.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/ridnsis05x.ind.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
/home/observium/rrd/nas-b41-is1.foo.bar/oid-enterprises.12124.1.2.2.4.0-GAUGE.rrd
...
...
...
...
On Fri, Oct 27, 2017 at 11:20 AM Adam Armstrong
adama@observium.org
...
...
...
...
wrote:
...
It works fine for me, both graphtypes on both oid types.
btw, that isn't a numeric OID, the OID name is wrong (it's
missing the
...
...
...
...
...
index) and that description is much longer than intended :P
If you click on the graph and then get the graph browser to give
you
...
...
...
...
...
the rrd output, you might get some useful information.
adam.
On 2017-10-26 05:40:36, Jesper Frank Nemholt jfn@dassic.com
wrote:
...
...
...
...
...
Hi!
Upgraded and now it breaks the stacked and combined line graphs.
The
...
...
...
...
...
individual graphs themselves are OK.
It seems to be across the board on all combined graphs and not
specific to the example attached (custom OIDs for EMC Isilon)
/Jesper
_______________________________________________ observium mailing
list
...
...
...
...
...
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________ observium mailing
list
...
...
...
...
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________ observium mailing
list
...
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium