Aggregate vs individual port graph maximum value & 95th % on bits graph
I have two graph questions that I haven't been able to get an answer on via IRC, so I thought this might be easier since I can include more information up front:
1st Question: A 2-port aggregate graph (agg_bits.png) shows a maximum value based on :AVERAGE while individual port graphs (port1_bits.png, port2_bits.png) for the ports that make up the aggregate show a maximum value based on :MAX. Is this intentional? If not, would it be possible to remove or toggle graphing average vs. maximum data on individual port graphs? I think both data points are useful, but in general when providing graphs the 5-minute average the only data we'd want to show them. More data below.
2nd Question: The "bits_old" graph includes 95th % both graphed and in the legend. Would it be possible to get the same for the other bits graphs, or at minimum, just bits? I did some digging and didn't find a means to use options via graphs.php to generate it myself.
Question 1 data:
An aggregate graph (bits) takes the :AVERAGE inbound and outbound of the two ports, using that data to create the graph and populate the legend (min/max/avg, etc.):
DEF:in0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:INOCTETS:AVERAGE DEF:out0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:OUTOCTETS:AVERAGE
DEF:in1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:INOCTETS:AVERAGE DEF:out1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
The individual ports use both :AVERAGE and :MAX.
DEF:outoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:AVERAG E
DEF:inoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:AVERAGE
DEF:outoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:MA X
DEF:inoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:MAX
(I didn't include all the CDEFs, AREAS, HRULES, etc. for brevity, but I have copies of each RRD command for reference, if needed.)
The "Max" value in the legend reflects the :AVERAGE-based maximum on the aggregate graph and the :MAX-based maximum on the individual graph. Because of this, when compared side-by-side, the values do not match.
Aggregate Max:
p1_in: 118.25Mb
p1_out: 31.95Mb
p2_in: 40.15Mb
p2_out: 2.61Mb
Port 1 Max:
p1_in: 304.39Mb
p1_out: 98.77Mb
Port 2 Max:
p2_in: 90.94Mb
p2_out: 14.05Mb
--
Thanks,
skpio
Wanted to bump this – if anyone has time to take a quick look and let me know way or another, it would be really helpful. Trying to replace several aging Cacti servers and billing scripts with Observium and this is all that’s holding me back. ☹
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Tuesday, November 28, 2017 7:20 AM To: observium@observium.org Subject: Aggregate vs individual port graph maximum value & 95th % on bits graph
I have two graph questions that I haven't been able to get an answer on via IRC, so I thought this might be easier since I can include more information up front:
1st Question: A 2-port aggregate graph (agg_bits.png) shows a maximum value based on :AVERAGE while individual port graphs (port1_bits.png, port2_bits.png) for the ports that make up the aggregate show a maximum value based on :MAX. Is this intentional? If not, would it be possible to remove or toggle graphing average vs. maximum data on individual port graphs? I think both data points are useful, but in general when providing graphs the 5-minute average the only data we’d want to show them. More data below.
2nd Question: The “bits_old” graph includes 95th % both graphed and in the legend. Would it be possible to get the same for the other bits graphs, or at minimum, just bits? I did some digging and didn’t find a means to use options via graphs.php to generate it myself.
Question 1 data:
An aggregate graph (bits) takes the :AVERAGE inbound and outbound of the two ports, using that data to create the graph and populate the legend (min/max/avg, etc.):
DEF:in0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:INOCTETS:AVERAGE DEF:out0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:OUTOCTETS:AVERAGE
DEF:in1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:INOCTETS:AVERAGE DEF:out1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
The individual ports use both :AVERAGE and :MAX.
DEF:outoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
DEF:inoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:AVERAGE
DEF:outoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:MAX
DEF:inoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:MAX
(I didn't include all the CDEFs, AREAS, HRULES, etc. for brevity, but I have copies of each RRD command for reference, if needed.)
The "Max" value in the legend reflects the :AVERAGE-based maximum on the aggregate graph and the :MAX-based maximum on the individual graph. Because of this, when compared side-by-side, the values do not match.
Aggregate Max:
p1_in: 118.25Mb
p1_out: 31.95Mb
p2_in: 40.15Mb
p2_out: 2.61Mb
Port 1 Max:
p1_in: 304.39Mb
p1_out: 98.77Mb
Port 2 Max:
p2_in: 90.94Mb
p2_out: 14.05Mb
--
Thanks,
skpio
For question 2, I added the following to /opt/observium/html/includes/graphs/generic_multi_bits_separated.inc.php after line 146:
$rrd_options .= " LINE1:95thin#aa0000";
$rrd_options .= " LINE1:d95thout#aa0000";
$rrd_options .= " 'COMMENT: \n'";
$rrd_options .= " 'COMMENT:95th %\n'";
$rrd_options .= " GPRINT:95thin:%6.2lf%s\n";
$rrd_options .= " GPRINT:95thout:%6.2lf%s\n";
This gets me what I need in a round-about fashion, but would obviously be overwritten the next time an update comes through for that file. I could change the output around a little bit to move the numbers up to the top of the legend with the rest of the stats, but didn’t want to make extensive changes to the file just for a proof of concept.
Unfortunately, the 95th % value is off by 6Mbps, so there’s probably some adjustments that still need to be made. Either that or it’s a byproduct of how the billing and aggregate ports graphs are generated – deltas vs. RRD info.
Alternatively, if there was an easy means to export the graphs that traffic accounting generates into a regular image with 95th % included, that would be even better.
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Thursday, November 30, 2017 11:01 AM To: observium@observium.org Subject: RE: Aggregate vs individual port graph maximum value & 95th % on bits graph
Wanted to bump this – if anyone has time to take a quick look and let me know way or another, it would be really helpful. Trying to replace several aging Cacti servers and billing scripts with Observium and this is all that’s holding me back. ☹
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Tuesday, November 28, 2017 7:20 AM To: observium@observium.org mailto:observium@observium.org Subject: Aggregate vs individual port graph maximum value & 95th % on bits graph
I have two graph questions that I haven't been able to get an answer on via IRC, so I thought this might be easier since I can include more information up front:
1st Question: A 2-port aggregate graph (agg_bits.png) shows a maximum value based on :AVERAGE while individual port graphs (port1_bits.png, port2_bits.png) for the ports that make up the aggregate show a maximum value based on :MAX. Is this intentional? If not, would it be possible to remove or toggle graphing average vs. maximum data on individual port graphs? I think both data points are useful, but in general when providing graphs the 5-minute average the only data we’d want to show them. More data below.
2nd Question: The “bits_old” graph includes 95th % both graphed and in the legend. Would it be possible to get the same for the other bits graphs, or at minimum, just bits? I did some digging and didn’t find a means to use options via graphs.php to generate it myself.
Question 1 data:
An aggregate graph (bits) takes the :AVERAGE inbound and outbound of the two ports, using that data to create the graph and populate the legend (min/max/avg, etc.):
DEF:in0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:INOCTETS:AVERAGE DEF:out0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:OUTOCTETS:AVERAGE
DEF:in1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:INOCTETS:AVERAGE DEF:out1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
The individual ports use both :AVERAGE and :MAX.
DEF:outoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
DEF:inoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:AVERAGE
DEF:outoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:MAX
DEF:inoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:MAX
(I didn't include all the CDEFs, AREAS, HRULES, etc. for brevity, but I have copies of each RRD command for reference, if needed.)
The "Max" value in the legend reflects the :AVERAGE-based maximum on the aggregate graph and the :MAX-based maximum on the individual graph. Because of this, when compared side-by-side, the values do not match.
Aggregate Max:
p1_in: 118.25Mb
p1_out: 31.95Mb
p2_in: 40.15Mb
p2_out: 2.61Mb
Port 1 Max:
p1_in: 304.39Mb
p1_out: 98.77Mb
Port 2 Max:
p2_in: 90.94Mb
p2_out: 14.05Mb
--
Thanks,
skpio
The 95th value from rrd is useless for billing purposes.
Adam.
Sent from BlueMail
On 4 Dec 2017, 16:52, at 16:52, Sean Pedersen spedersen.lists@gmail.com wrote:
For question 2, I added the following to /opt/observium/html/includes/graphs/generic_multi_bits_separated.inc.php after line 146:
$rrd_options .= " LINE1:95thin#aa0000";
$rrd_options .= " LINE1:d95thout#aa0000";
$rrd_options .= " 'COMMENT: \n'";
$rrd_options .= " 'COMMENT:95th %\n'";
$rrd_options .= " GPRINT:95thin:%6.2lf%s\n";
$rrd_options .= " GPRINT:95thout:%6.2lf%s\n";
This gets me what I need in a round-about fashion, but would obviously be overwritten the next time an update comes through for that file. I could change the output around a little bit to move the numbers up to the top of the legend with the rest of the stats, but didn’t want to make extensive changes to the file just for a proof of concept.
Unfortunately, the 95th % value is off by 6Mbps, so there’s probably some adjustments that still need to be made. Either that or it’s a byproduct of how the billing and aggregate ports graphs are generated – deltas vs. RRD info.
Alternatively, if there was an easy means to export the graphs that traffic accounting generates into a regular image with 95th % included, that would be even better.
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Thursday, November 30, 2017 11:01 AM To: observium@observium.org Subject: RE: Aggregate vs individual port graph maximum value & 95th % on bits graph
Wanted to bump this – if anyone has time to take a quick look and let me know way or another, it would be really helpful. Trying to replace several aging Cacti servers and billing scripts with Observium and this is all that’s holding me back. ☹
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Tuesday, November 28, 2017 7:20 AM To: observium@observium.org mailto:observium@observium.org Subject: Aggregate vs individual port graph maximum value & 95th % on bits graph
I have two graph questions that I haven't been able to get an answer on via IRC, so I thought this might be easier since I can include more information up front:
1st Question: A 2-port aggregate graph (agg_bits.png) shows a maximum value based on :AVERAGE while individual port graphs (port1_bits.png, port2_bits.png) for the ports that make up the aggregate show a maximum value based on :MAX. Is this intentional? If not, would it be possible to remove or toggle graphing average vs. maximum data on individual port graphs? I think both data points are useful, but in general when providing graphs the 5-minute average the only data we’d want to show them. More data below.
2nd Question: The “bits_old” graph includes 95th % both graphed and in the legend. Would it be possible to get the same for the other bits graphs, or at minimum, just bits? I did some digging and didn’t find a means to use options via graphs.php to generate it myself.
Question 1 data:
An aggregate graph (bits) takes the :AVERAGE inbound and outbound of the two ports, using that data to create the graph and populate the legend (min/max/avg, etc.):
DEF:in0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:INOCTETS:AVERAGE DEF:out0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:OUTOCTETS:AVERAGE
DEF:in1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:INOCTETS:AVERAGE DEF:out1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
The individual ports use both :AVERAGE and :MAX.
DEF:outoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
DEF:inoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:AVERAGE
DEF:outoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:MAX
DEF:inoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:MAX
(I didn't include all the CDEFs, AREAS, HRULES, etc. for brevity, but I have copies of each RRD command for reference, if needed.)
The "Max" value in the legend reflects the :AVERAGE-based maximum on the aggregate graph and the :MAX-based maximum on the individual graph. Because of this, when compared side-by-side, the values do not match.
Aggregate Max:
p1_in: 118.25Mb
p1_out: 31.95Mb
p2_in: 40.15Mb
p2_out: 2.61Mb
Port 1 Max:
p1_in: 304.39Mb
p1_out: 98.77Mb
Port 2 Max:
p2_in: 90.94Mb
p2_out: 14.05Mb
--
Thanks,
skpio
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
I figured as much, and assumed that’s why you built the better, more dynamic graphing solution. We’ve unfortunately been using RRD 95th for quite a while. We can easily move away from it since no one would know the difference, but the challenge is the “proof” (either automatic or on-demand) we’re required to provide our customers.
Is there a way to generate the billing graph with 95th % in a .png format for export?
If I didn’t have to provide the graph I’d be set with what’s available already. In fact, the billing table in the DB makes it even easier for me to pull the info we used to pull from Cacti.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 10:37 AM To: 'Observium' observium@observium.org Cc: 'Observium' observium@observium.org Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
The 95th value from rrd is useless for billing purposes.
Adam.
Sent from BlueMail http://www.bluemail.me/r?b=11327
On 4 Dec 2017, at 16:52, Sean Pedersen <spedersen.lists@gmail.com mailto:spedersen.lists@gmail.com > wrote:
For question 2, I added the following to /opt/observium/html/includes/graphs/generic_multi_bits_separated.inc.php after line 146:
$rrd_options .= " LINE1:95thin#aa0000";
$rrd_options .= " LINE1:d95thout#aa0000";
$rrd_options .= " 'COMMENT: \n file://n '";
$rrd_options .= " 'COMMENT:95th %\n'";
$rrd_options .= " GPRINT:95thin:%6.2lf%s\n";
$rrd_options .= " GPRINT:95thout:%6.2lf%s\n";
This gets me what I need in a round-about fashion, but would obviously be overwritten the next time an update comes through for that file. I could change the output around a little bit to move the numbers up to the top of the legend with the rest of the stats, but didn’t want to make extensive changes to the file just for a proof of concept.
Unfortunately, the 95th % value is off by 6Mbps, so there’s probably some adjustments that still need to be made. Either that or it’s a byproduct of how the billing and aggregate ports graphs are generated – deltas vs. RRD info.
Alternatively, if there was an easy means to export the graphs that traffic accounting generates into a regular image with 95th % included, that would be even better.
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Thursday, November 30, 2017 11:01 AM To: observium@observium.org mailto:observium@observium.org Subject: RE: Aggregate vs individual port graph maximum value & 95th % on bits graph
Wanted to bump this – if anyone has time to take a quick look and let me know way or another, it would be really helpful. Trying to replace several aging Cacti servers and billing scripts with Observium and this is all that’s holding me back. ☹
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Tuesday, November 28, 2017 7:20 AM To: observium@observium.org mailto:observium@observium.org Subject: Aggregate vs individual port graph maximum value & 95th % on bits graph
I have two graph questions that I haven't been able to get an answer on via IRC, so I thought this might be easier since I can include more information up front:
1st Question: A 2-port aggregate graph (agg_bits.png) shows a maximum value based on :AVERAGE while individual port graphs (port1_bits.png, port2_bits.png) for the ports that make up the aggregate show a maximum value based on :MAX. Is this intentional? If not, would it be possible to remove or toggle graphing average vs. maximum data on individual port graphs? I think both data points are useful, but in general when providing graphs the 5-minute average the only data we’d want to show them. More data below.
2nd Question: The “bits_old” graph includes 95th % both graphed and in the legend. Would it be possible to get the same for the other bits graphs, or at minimum, just bits? I did some digging and didn’t find a means to use options via graphs.php to generate it myself.
Question 1 data:
An aggregate graph (bits) takes the :AVERAGE inbound and outbound of the two ports, using that data to create the graph and populate the legend (min/max/avg, etc.):
DEF:in0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:INOCTETS:AVERAGE DEF:out0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:OUTOCTETS:AVERAGE
DEF:in1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:INOCTETS:AVERAGE DEF:out1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
The individual ports use both :AVERAGE and :MAX.
DEF:outoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
DEF:inoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:AVERAGE
DEF:outoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:MAX
DEF:inoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:MAX
(I didn't include all the CDEFs, AREAS, HRULES, etc. for brevity, but I have copies of each RRD command for reference, if needed.)
The "Max" value in the legend reflects the :AVERAGE-based maximum on the aggregate graph and the :MAX-based maximum on the individual graph. Because of this, when compared side-by-side, the values do not match.
Aggregate Max:
p1_in: 118.25Mb
p1_out: 31.95Mb
p2_in: 40.15Mb
p2_out: 2.61Mb
Port 1 Max:
p1_in: 304.39Mb
p1_out: 98.77Mb
Port 2 Max:
p2_in: 90.94Mb
p2_out: 14.05Mb
--
Thanks,
skpio
_____
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Haha. We just moved away from generating .png billing graphs to generating them in the browser with JS because the graph-drawing library was trash.
Perhaps I can find another solutions for drawing a png from the billing data.
adam.
Adam Armstrong Managing Director & Lead Architect Observium Limited http://www.observium.org http://docs.observium.org http://jira.observium.org On 2017-12-04 18:57:29, Sean Pedersen spedersen.lists@gmail.com wrote: I figured as much, and assumed that’s why you built the better, more dynamic graphing solution. We’ve unfortunately been using RRD 95th for quite a while. We can easily move away from it since no one would know the difference, but the challenge is the “proof” (either automatic or on-demand) we’re required to provide our customers. Is there a way to generate the billing graph with 95th % in a .png format for export? If I didn’t have to provide the graph I’d be set with what’s available already. In fact, the billing table in the DB makes it even easier for me to pull the info we used to pull from Cacti. From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 10:37 AM To: 'Observium' observium@observium.org Cc: 'Observium' observium@observium.org Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph The 95th value from rrd is useless for billing purposes. Adam. Sent from BlueMail [http://www.bluemail.me/r?b=11327] On 4 Dec 2017, at 16:52, Sean Pedersen <spedersen.lists@gmail.com [mailto:spedersen.lists@gmail.com]> wrote: For question 2, I added the following to /opt/observium/html/includes/graphs/generic_multi_bits_separated.inc.php after line 146: $rrd_options .= " LINE1:95thin#aa0000"; $rrd_options .= " LINE1:d95thout#aa0000"; $rrd_options .= " 'COMMENT: \n [file://n]'"; $rrd_options .= " 'COMMENT:95th %\n'"; $rrd_options .= " GPRINT:95thin:%6.2lf%s\n"; $rrd_options .= " GPRINT:95thout:%6.2lf%s\n"; This gets me what I need in a round-about fashion, but would obviously be overwritten the next time an update comes through for that file. I could change the output around a little bit to move the numbers up to the top of the legend with the rest of the stats, but didn’t want to make extensive changes to the file just for a proof of concept. Unfortunately, the 95th % value is off by 6Mbps, so there’s probably some adjustments that still need to be made. Either that or it’s a byproduct of how the billing and aggregate ports graphs are generated – deltas vs. RRD info. Alternatively, if there was an easy means to export the graphs that traffic accounting generates into a regular image with 95th % included, that would be even better. From: Sean Pedersen [mailto:spedersen.lists@gmail.com [mailto:spedersen.lists@gmail.com]] Sent: Thursday, November 30, 2017 11:01 AM To: observium@observium.org [mailto:observium@observium.org] Subject: RE: Aggregate vs individual port graph maximum value & 95th % on bits graph Wanted to bump this – if anyone has time to take a quick look and let me know way or another, it would be really helpful. Trying to replace several aging Cacti servers and billing scripts with Observium and this is all that’s holding me back. ☹ From: Sean Pedersen [mailto:spedersen.lists@gmail.com [mailto:spedersen.lists@gmail.com]] Sent: Tuesday, November 28, 2017 7:20 AM To: observium@observium.org [mailto:observium@observium.org] Subject: Aggregate vs individual port graph maximum value & 95th % on bits graph I have two graph questions that I haven't been able to get an answer on via IRC, so I thought this might be easier since I can include more information up front: 1st Question: A 2-port aggregate graph (agg_bits.png) shows a maximum value based on :AVERAGE while individual port graphs (port1_bits.png, port2_bits.png) for the ports that make up the aggregate show a maximum value based on :MAX. Is this intentional? If not, would it be possible to remove or toggle graphing average vs. maximum data on individual port graphs? I think both data points are useful, but in general when providing graphs the 5-minute average the only data we’d want to show them. More data below. 2nd Question: The “bits_old” graph includes 95th % both graphed and in the legend. Would it be possible to get the same for the other bits graphs, or at minimum, just bits? I did some digging and didn’t find a means to use options via graphs.php to generate it myself. Question 1 data: An aggregate graph (bits) takes the :AVERAGE inbound and outbound of the two ports, using that data to create the graph and populate the legend (min/max/avg, etc.): DEF:in0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:INOCTETS:AVERAGE DEF:out0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:OUTOCTETS:AVERAGE DEF:in1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:INOCTETS:AVERAGE DEF:out1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE The individual ports use both :AVERAGE and :MAX. DEF:outoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE DEF:inoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:AVERAGE DEF:outoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:MAX DEF:inoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:MAX (I didn't include all the CDEFs, AREAS, HRULES, etc. for brevity, but I have copies of each RRD command for reference, if needed.) The "Max" value in the legend reflects the :AVERAGE-based maximum on the aggregate graph and the :MAX-based maximum on the individual graph. Because of this, when compared side-by-side, the values do not match. Aggregate Max: p1_in: 118.25Mb p1_out: 31.95Mb p2_in: 40.15Mb p2_out: 2.61Mb Port 1 Max: p1_in: 304.39Mb p1_out: 98.77Mb Port 2 Max: p2_in: 90.94Mb p2_out: 14.05Mb -- Thanks, skpio
observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
I like the JS-driven graphs, I just can’t export them since they’re not a flattened picture. It’s kind of funny, if you right-click and save under the right circumstances you can end up with just a green or purple squiggly line and nothing else.
In a sense we dug ourselves into a hole. They attach the 30 day graphs to billed customers as “proof” of their overage due to some previous accuracy issues, which established a precedent. That means I can’t take it away. It’s really frustrating that drawing a stupid picture is what is keeping me on Cacti and outdated plug-ins/scripts.
Can the image that JS generates be flattened somehow and stored in a temporary cache? Maybe only called via a specific mechanism. Something similar to graphs.php? Throw it a bill ID, to/from, and it takes the same data and draws a graph that can be scraped.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 12:00 PM To: observium@observium.org Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
Haha. We just moved away from generating .png billing graphs to generating them in the browser with JS because the graph-drawing library was trash.
Perhaps I can find another solutions for drawing a png from the billing data.
adam.
Adam Armstrong
Managing Director & Lead Architect
Observium Limited
On 2017-12-04 18:57:29, Sean Pedersen <spedersen.lists@gmail.com mailto:spedersen.lists@gmail.com > wrote:
I figured as much, and assumed that’s why you built the better, more dynamic graphing solution. We’ve unfortunately been using RRD 95th for quite a while. We can easily move away from it since no one would know the difference, but the challenge is the “proof” (either automatic or on-demand) we’re required to provide our customers.
Is there a way to generate the billing graph with 95th % in a .png format for export?
If I didn’t have to provide the graph I’d be set with what’s available already. In fact, the billing table in the DB makes it even easier for me to pull the info we used to pull from Cacti.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 10:37 AM To: 'Observium' <observium@observium.org mailto:observium@observium.org > Cc: 'Observium' <observium@observium.org mailto:observium@observium.org > Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
The 95th value from rrd is useless for billing purposes.
Adam.
Sent from BlueMail http://www.bluemail.me/r?b=11327
On 4 Dec 2017, at 16:52, Sean Pedersen <spedersen.lists@gmail.com mailto:spedersen.lists@gmail.com > wrote:
For question 2, I added the following to /opt/observium/html/includes/graphs/generic_multi_bits_separated.inc.php after line 146:
$rrd_options .= " LINE1:95thin#aa0000";
$rrd_options .= " LINE1:d95thout#aa0000";
$rrd_options .= " 'COMMENT: \n file://n '";
$rrd_options .= " 'COMMENT:95th %\n'";
$rrd_options .= " GPRINT:95thin:%6.2lf%s\n";
$rrd_options .= " GPRINT:95thout:%6.2lf%s\n";
This gets me what I need in a round-about fashion, but would obviously be overwritten the next time an update comes through for that file. I could change the output around a little bit to move the numbers up to the top of the legend with the rest of the stats, but didn’t want to make extensive changes to the file just for a proof of concept.
Unfortunately, the 95th % value is off by 6Mbps, so there’s probably some adjustments that still need to be made. Either that or it’s a byproduct of how the billing and aggregate ports graphs are generated – deltas vs. RRD info.
Alternatively, if there was an easy means to export the graphs that traffic accounting generates into a regular image with 95th % included, that would be even better.
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Thursday, November 30, 2017 11:01 AM To: observium@observium.org mailto:observium@observium.org Subject: RE: Aggregate vs individual port graph maximum value & 95th % on bits graph
Wanted to bump this – if anyone has time to take a quick look and let me know way or another, it would be really helpful. Trying to replace several aging Cacti servers and billing scripts with Observium and this is all that’s holding me back. ☹
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Tuesday, November 28, 2017 7:20 AM To: observium@observium.org mailto:observium@observium.org Subject: Aggregate vs individual port graph maximum value & 95th % on bits graph
I have two graph questions that I haven't been able to get an answer on via IRC, so I thought this might be easier since I can include more information up front:
1st Question: A 2-port aggregate graph (agg_bits.png) shows a maximum value based on :AVERAGE while individual port graphs (port1_bits.png, port2_bits.png) for the ports that make up the aggregate show a maximum value based on :MAX. Is this intentional? If not, would it be possible to remove or toggle graphing average vs. maximum data on individual port graphs? I think both data points are useful, but in general when providing graphs the 5-minute average the only data we’d want to show them. More data below.
2nd Question: The “bits_old” graph includes 95th % both graphed and in the legend. Would it be possible to get the same for the other bits graphs, or at minimum, just bits? I did some digging and didn’t find a means to use options via graphs.php to generate it myself.
Question 1 data:
An aggregate graph (bits) takes the :AVERAGE inbound and outbound of the two ports, using that data to create the graph and populate the legend (min/max/avg, etc.):
DEF:in0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:INOCTETS:AVERAGE DEF:out0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:OUTOCTETS:AVERAGE
DEF:in1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:INOCTETS:AVERAGE DEF:out1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
The individual ports use both :AVERAGE and :MAX.
DEF:outoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
DEF:inoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:AVERAGE
DEF:outoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:MAX
DEF:inoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:MAX
(I didn't include all the CDEFs, AREAS, HRULES, etc. for brevity, but I have copies of each RRD command for reference, if needed.)
The "Max" value in the legend reflects the :AVERAGE-based maximum on the aggregate graph and the :MAX-based maximum on the individual graph. Because of this, when compared side-by-side, the values do not match.
Aggregate Max:
p1_in: 118.25Mb
p1_out: 31.95Mb
p2_in: 40.15Mb
p2_out: 2.61Mb
Port 1 Max:
p1_in: 304.39Mb
p1_out: 98.77Mb
Port 2 Max:
p2_in: 90.94Mb
p2_out: 14.05Mb
--
Thanks,
skpio
_____
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
If you just want to save the image, install a screen cap tool like sharex. It'll let you snip the graph on keybind.
The JS isn't actually generating an image, it'll be building a SVG representation of the graph in the DOM that the browser is rendering. The only way to save this is via screenshotting.
adam.
Adam Armstrong Managing Director & Lead Architect Observium Limited http://www.observium.org http://docs.observium.org http://jira.observium.org On 2017-12-04 19:37:12, Sean Pedersen spedersen.lists@gmail.com wrote: I like the JS-driven graphs, I just can’t export them since they’re not a flattened picture. It’s kind of funny, if you right-click and save under the right circumstances you can end up with just a green or purple squiggly line and nothing else. In a sense we dug ourselves into a hole. They attach the 30 day graphs to billed customers as “proof” of their overage due to some previous accuracy issues, which established a precedent. That means I can’t take it away. It’s really frustrating that drawing a stupid picture is what is keeping me on Cacti and outdated plug-ins/scripts. Can the image that JS generates be flattened somehow and stored in a temporary cache? Maybe only called via a specific mechanism. Something similar to graphs.php? Throw it a bill ID, to/from, and it takes the same data and draws a graph that can be scraped. From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 12:00 PM To: observium@observium.org Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph Haha. We just moved away from generating .png billing graphs to generating them in the browser with JS because the graph-drawing library was trash. Perhaps I can find another solutions for drawing a png from the billing data. adam. Adam Armstrong Managing Director & Lead Architect Observium Limited http://www.observium.org [http://www.observium.org] http://docs.observium.org [http://docs.observium.org] http://jira.observium.org [http://jira.observium.org] On 2017-12-04 18:57:29, Sean Pedersen <spedersen.lists@gmail.com [mailto:spedersen.lists@gmail.com]> wrote: I figured as much, and assumed that’s why you built the better, more dynamic graphing solution. We’ve unfortunately been using RRD 95th for quite a while. We can easily move away from it since no one would know the difference, but the challenge is the “proof” (either automatic or on-demand) we’re required to provide our customers. Is there a way to generate the billing graph with 95th % in a .png format for export? If I didn’t have to provide the graph I’d be set with what’s available already. In fact, the billing table in the DB makes it even easier for me to pull the info we used to pull from Cacti. From: observium [mailto:observium-bounces@observium.org [mailto:observium-bounces@observium.org]] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 10:37 AM To: 'Observium' <observium@observium.org [mailto:observium@observium.org]> Cc: 'Observium' <observium@observium.org [mailto:observium@observium.org]> Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph The 95th value from rrd is useless for billing purposes. Adam. Sent from BlueMail [http://www.bluemail.me/r?b=11327] On 4 Dec 2017, at 16:52, Sean Pedersen <spedersen.lists@gmail.com [mailto:spedersen.lists@gmail.com]> wrote: For question 2, I added the following to /opt/observium/html/includes/graphs/generic_multi_bits_separated.inc.php after line 146: $rrd_options .= " LINE1:95thin#aa0000"; $rrd_options .= " LINE1:d95thout#aa0000"; $rrd_options .= " 'COMMENT: \n [file://n]'"; $rrd_options .= " 'COMMENT:95th %\n'"; $rrd_options .= " GPRINT:95thin:%6.2lf%s\n"; $rrd_options .= " GPRINT:95thout:%6.2lf%s\n"; This gets me what I need in a round-about fashion, but would obviously be overwritten the next time an update comes through for that file. I could change the output around a little bit to move the numbers up to the top of the legend with the rest of the stats, but didn’t want to make extensive changes to the file just for a proof of concept. Unfortunately, the 95th % value is off by 6Mbps, so there’s probably some adjustments that still need to be made. Either that or it’s a byproduct of how the billing and aggregate ports graphs are generated – deltas vs. RRD info. Alternatively, if there was an easy means to export the graphs that traffic accounting generates into a regular image with 95th % included, that would be even better. From: Sean Pedersen [mailto:spedersen.lists@gmail.com [mailto:spedersen.lists@gmail.com]] Sent: Thursday, November 30, 2017 11:01 AM To: observium@observium.org [mailto:observium@observium.org] Subject: RE: Aggregate vs individual port graph maximum value & 95th % on bits graph Wanted to bump this – if anyone has time to take a quick look and let me know way or another, it would be really helpful. Trying to replace several aging Cacti servers and billing scripts with Observium and this is all that’s holding me back. ☹ From: Sean Pedersen [mailto:spedersen.lists@gmail.com [mailto:spedersen.lists@gmail.com]] Sent: Tuesday, November 28, 2017 7:20 AM To: observium@observium.org [mailto:observium@observium.org] Subject: Aggregate vs individual port graph maximum value & 95th % on bits graph I have two graph questions that I haven't been able to get an answer on via IRC, so I thought this might be easier since I can include more information up front: 1st Question: A 2-port aggregate graph (agg_bits.png) shows a maximum value based on :AVERAGE while individual port graphs (port1_bits.png, port2_bits.png) for the ports that make up the aggregate show a maximum value based on :MAX. Is this intentional? If not, would it be possible to remove or toggle graphing average vs. maximum data on individual port graphs? I think both data points are useful, but in general when providing graphs the 5-minute average the only data we’d want to show them. More data below. 2nd Question: The “bits_old” graph includes 95th % both graphed and in the legend. Would it be possible to get the same for the other bits graphs, or at minimum, just bits? I did some digging and didn’t find a means to use options via graphs.php to generate it myself. Question 1 data: An aggregate graph (bits) takes the :AVERAGE inbound and outbound of the two ports, using that data to create the graph and populate the legend (min/max/avg, etc.): DEF:in0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:INOCTETS:AVERAGE DEF:out0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:OUTOCTETS:AVERAGE DEF:in1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:INOCTETS:AVERAGE DEF:out1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE The individual ports use both :AVERAGE and :MAX. DEF:outoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE DEF:inoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:AVERAGE DEF:outoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:MAX DEF:inoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:MAX (I didn't include all the CDEFs, AREAS, HRULES, etc. for brevity, but I have copies of each RRD command for reference, if needed.) The "Max" value in the legend reflects the :AVERAGE-based maximum on the aggregate graph and the :MAX-based maximum on the individual graph. Because of this, when compared side-by-side, the values do not match. Aggregate Max: p1_in: 118.25Mb p1_out: 31.95Mb p2_in: 40.15Mb p2_out: 2.61Mb Port 1 Max: p1_in: 304.39Mb p1_out: 98.77Mb Port 2 Max: p2_in: 90.94Mb p2_out: 14.05Mb -- Thanks, skpio
observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
Unfortunately, I can’t automate that in a script to generate our monthly billing reports, nor could I expect anyone on my team or from the billing department to screenshot graphs to give them to customers. It definitely works in a pinch, but not long-term. I don’t expect magic to be worked. I just have to figure out a way around the limitation so it can be scripted/automated into a monthly billing report that can be emailed or exported.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 12:50 PM To: observium@observium.org Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
If you just want to save the image, install a screen cap tool like sharex. It'll let you snip the graph on keybind.
The JS isn't actually generating an image, it'll be building a SVG representation of the graph in the DOM that the browser is rendering. The only way to save this is via screenshotting.
adam.
Adam Armstrong
Managing Director & Lead Architect
Observium Limited
On 2017-12-04 19:37:12, Sean Pedersen <spedersen.lists@gmail.com mailto:spedersen.lists@gmail.com > wrote:
I like the JS-driven graphs, I just can’t export them since they’re not a flattened picture. It’s kind of funny, if you right-click and save under the right circumstances you can end up with just a green or purple squiggly line and nothing else.
In a sense we dug ourselves into a hole. They attach the 30 day graphs to billed customers as “proof” of their overage due to some previous accuracy issues, which established a precedent. That means I can’t take it away. It’s really frustrating that drawing a stupid picture is what is keeping me on Cacti and outdated plug-ins/scripts.
Can the image that JS generates be flattened somehow and stored in a temporary cache? Maybe only called via a specific mechanism. Something similar to graphs.php? Throw it a bill ID, to/from, and it takes the same data and draws a graph that can be scraped.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 12:00 PM To: observium@observium.org mailto:observium@observium.org Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
Haha. We just moved away from generating .png billing graphs to generating them in the browser with JS because the graph-drawing library was trash.
Perhaps I can find another solutions for drawing a png from the billing data.
adam.
Adam Armstrong
Managing Director & Lead Architect
Observium Limited
On 2017-12-04 18:57:29, Sean Pedersen <spedersen.lists@gmail.com mailto:spedersen.lists@gmail.com > wrote:
I figured as much, and assumed that’s why you built the better, more dynamic graphing solution. We’ve unfortunately been using RRD 95th for quite a while. We can easily move away from it since no one would know the difference, but the challenge is the “proof” (either automatic or on-demand) we’re required to provide our customers.
Is there a way to generate the billing graph with 95th % in a .png format for export?
If I didn’t have to provide the graph I’d be set with what’s available already. In fact, the billing table in the DB makes it even easier for me to pull the info we used to pull from Cacti.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 10:37 AM To: 'Observium' <observium@observium.org mailto:observium@observium.org > Cc: 'Observium' <observium@observium.org mailto:observium@observium.org > Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
The 95th value from rrd is useless for billing purposes.
Adam.
Sent from BlueMail http://www.bluemail.me/r?b=11327
On 4 Dec 2017, at 16:52, Sean Pedersen <spedersen.lists@gmail.com mailto:spedersen.lists@gmail.com > wrote:
For question 2, I added the following to /opt/observium/html/includes/graphs/generic_multi_bits_separated.inc.php after line 146:
$rrd_options .= " LINE1:95thin#aa0000";
$rrd_options .= " LINE1:d95thout#aa0000";
$rrd_options .= " 'COMMENT: \n file://n '";
$rrd_options .= " 'COMMENT:95th %\n'";
$rrd_options .= " GPRINT:95thin:%6.2lf%s\n";
$rrd_options .= " GPRINT:95thout:%6.2lf%s\n";
This gets me what I need in a round-about fashion, but would obviously be overwritten the next time an update comes through for that file. I could change the output around a little bit to move the numbers up to the top of the legend with the rest of the stats, but didn’t want to make extensive changes to the file just for a proof of concept.
Unfortunately, the 95th % value is off by 6Mbps, so there’s probably some adjustments that still need to be made. Either that or it’s a byproduct of how the billing and aggregate ports graphs are generated – deltas vs. RRD info.
Alternatively, if there was an easy means to export the graphs that traffic accounting generates into a regular image with 95th % included, that would be even better.
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Thursday, November 30, 2017 11:01 AM To: observium@observium.org mailto:observium@observium.org Subject: RE: Aggregate vs individual port graph maximum value & 95th % on bits graph
Wanted to bump this – if anyone has time to take a quick look and let me know way or another, it would be really helpful. Trying to replace several aging Cacti servers and billing scripts with Observium and this is all that’s holding me back. ☹
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Tuesday, November 28, 2017 7:20 AM To: observium@observium.org mailto:observium@observium.org Subject: Aggregate vs individual port graph maximum value & 95th % on bits graph
I have two graph questions that I haven't been able to get an answer on via IRC, so I thought this might be easier since I can include more information up front:
1st Question: A 2-port aggregate graph (agg_bits.png) shows a maximum value based on :AVERAGE while individual port graphs (port1_bits.png, port2_bits.png) for the ports that make up the aggregate show a maximum value based on :MAX. Is this intentional? If not, would it be possible to remove or toggle graphing average vs. maximum data on individual port graphs? I think both data points are useful, but in general when providing graphs the 5-minute average the only data we’d want to show them. More data below.
2nd Question: The “bits_old” graph includes 95th % both graphed and in the legend. Would it be possible to get the same for the other bits graphs, or at minimum, just bits? I did some digging and didn’t find a means to use options via graphs.php to generate it myself.
Question 1 data:
An aggregate graph (bits) takes the :AVERAGE inbound and outbound of the two ports, using that data to create the graph and populate the legend (min/max/avg, etc.):
DEF:in0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:INOCTETS:AVERAGE DEF:out0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:OUTOCTETS:AVERAGE
DEF:in1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:INOCTETS:AVERAGE DEF:out1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
The individual ports use both :AVERAGE and :MAX.
DEF:outoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
DEF:inoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:AVERAGE
DEF:outoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:MAX
DEF:inoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:MAX
(I didn't include all the CDEFs, AREAS, HRULES, etc. for brevity, but I have copies of each RRD command for reference, if needed.)
The "Max" value in the legend reflects the :AVERAGE-based maximum on the aggregate graph and the :MAX-based maximum on the individual graph. Because of this, when compared side-by-side, the values do not match.
Aggregate Max:
p1_in: 118.25Mb
p1_out: 31.95Mb
p2_in: 40.15Mb
p2_out: 2.61Mb
Port 1 Max:
p1_in: 304.39Mb
p1_out: 98.77Mb
Port 2 Max:
p2_in: 90.94Mb
p2_out: 14.05Mb
--
Thanks,
skpio
_____
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Side-thought: if I can use billing-data.php to generate the CSV data for a timeframe, I wonder if there is a local Linux package that would convert that to a graphic? Something like gnuplot?
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Monday, December 4, 2017 12:54 PM To: 'Observium' observium@observium.org Subject: RE: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
Unfortunately, I can’t automate that in a script to generate our monthly billing reports, nor could I expect anyone on my team or from the billing department to screenshot graphs to give them to customers. It definitely works in a pinch, but not long-term. I don’t expect magic to be worked. I just have to figure out a way around the limitation so it can be scripted/automated into a monthly billing report that can be emailed or exported.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 12:50 PM To: observium@observium.org mailto:observium@observium.org Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
If you just want to save the image, install a screen cap tool like sharex. It'll let you snip the graph on keybind.
The JS isn't actually generating an image, it'll be building a SVG representation of the graph in the DOM that the browser is rendering. The only way to save this is via screenshotting.
adam.
Adam Armstrong
Managing Director & Lead Architect
Observium Limited
On 2017-12-04 19:37:12, Sean Pedersen <spedersen.lists@gmail.com mailto:spedersen.lists@gmail.com > wrote:
I like the JS-driven graphs, I just can’t export them since they’re not a flattened picture. It’s kind of funny, if you right-click and save under the right circumstances you can end up with just a green or purple squiggly line and nothing else.
In a sense we dug ourselves into a hole. They attach the 30 day graphs to billed customers as “proof” of their overage due to some previous accuracy issues, which established a precedent. That means I can’t take it away. It’s really frustrating that drawing a stupid picture is what is keeping me on Cacti and outdated plug-ins/scripts.
Can the image that JS generates be flattened somehow and stored in a temporary cache? Maybe only called via a specific mechanism. Something similar to graphs.php? Throw it a bill ID, to/from, and it takes the same data and draws a graph that can be scraped.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 12:00 PM To: observium@observium.org mailto:observium@observium.org Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
Haha. We just moved away from generating .png billing graphs to generating them in the browser with JS because the graph-drawing library was trash.
Perhaps I can find another solutions for drawing a png from the billing data.
adam.
Adam Armstrong
Managing Director & Lead Architect
Observium Limited
On 2017-12-04 18:57:29, Sean Pedersen <spedersen.lists@gmail.com mailto:spedersen.lists@gmail.com > wrote:
I figured as much, and assumed that’s why you built the better, more dynamic graphing solution. We’ve unfortunately been using RRD 95th for quite a while. We can easily move away from it since no one would know the difference, but the challenge is the “proof” (either automatic or on-demand) we’re required to provide our customers.
Is there a way to generate the billing graph with 95th % in a .png format for export?
If I didn’t have to provide the graph I’d be set with what’s available already. In fact, the billing table in the DB makes it even easier for me to pull the info we used to pull from Cacti.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 10:37 AM To: 'Observium' <observium@observium.org mailto:observium@observium.org > Cc: 'Observium' <observium@observium.org mailto:observium@observium.org > Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
The 95th value from rrd is useless for billing purposes.
Adam.
Sent from BlueMail http://www.bluemail.me/r?b=11327
On 4 Dec 2017, at 16:52, Sean Pedersen <spedersen.lists@gmail.com mailto:spedersen.lists@gmail.com > wrote:
For question 2, I added the following to /opt/observium/html/includes/graphs/generic_multi_bits_separated.inc.php after line 146:
$rrd_options .= " LINE1:95thin#aa0000";
$rrd_options .= " LINE1:d95thout#aa0000";
$rrd_options .= " 'COMMENT: \n file://n '";
$rrd_options .= " 'COMMENT:95th %\n'";
$rrd_options .= " GPRINT:95thin:%6.2lf%s\n";
$rrd_options .= " GPRINT:95thout:%6.2lf%s\n";
This gets me what I need in a round-about fashion, but would obviously be overwritten the next time an update comes through for that file. I could change the output around a little bit to move the numbers up to the top of the legend with the rest of the stats, but didn’t want to make extensive changes to the file just for a proof of concept.
Unfortunately, the 95th % value is off by 6Mbps, so there’s probably some adjustments that still need to be made. Either that or it’s a byproduct of how the billing and aggregate ports graphs are generated – deltas vs. RRD info.
Alternatively, if there was an easy means to export the graphs that traffic accounting generates into a regular image with 95th % included, that would be even better.
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Thursday, November 30, 2017 11:01 AM To: observium@observium.org mailto:observium@observium.org Subject: RE: Aggregate vs individual port graph maximum value & 95th % on bits graph
Wanted to bump this – if anyone has time to take a quick look and let me know way or another, it would be really helpful. Trying to replace several aging Cacti servers and billing scripts with Observium and this is all that’s holding me back. ☹
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Tuesday, November 28, 2017 7:20 AM To: observium@observium.org mailto:observium@observium.org Subject: Aggregate vs individual port graph maximum value & 95th % on bits graph
I have two graph questions that I haven't been able to get an answer on via IRC, so I thought this might be easier since I can include more information up front:
1st Question: A 2-port aggregate graph (agg_bits.png) shows a maximum value based on :AVERAGE while individual port graphs (port1_bits.png, port2_bits.png) for the ports that make up the aggregate show a maximum value based on :MAX. Is this intentional? If not, would it be possible to remove or toggle graphing average vs. maximum data on individual port graphs? I think both data points are useful, but in general when providing graphs the 5-minute average the only data we’d want to show them. More data below.
2nd Question: The “bits_old” graph includes 95th % both graphed and in the legend. Would it be possible to get the same for the other bits graphs, or at minimum, just bits? I did some digging and didn’t find a means to use options via graphs.php to generate it myself.
Question 1 data:
An aggregate graph (bits) takes the :AVERAGE inbound and outbound of the two ports, using that data to create the graph and populate the legend (min/max/avg, etc.):
DEF:in0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:INOCTETS:AVERAGE DEF:out0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:OUTOCTETS:AVERAGE
DEF:in1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:INOCTETS:AVERAGE DEF:out1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
The individual ports use both :AVERAGE and :MAX.
DEF:outoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
DEF:inoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:AVERAGE
DEF:outoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:MAX
DEF:inoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:MAX
(I didn't include all the CDEFs, AREAS, HRULES, etc. for brevity, but I have copies of each RRD command for reference, if needed.)
The "Max" value in the legend reflects the :AVERAGE-based maximum on the aggregate graph and the :MAX-based maximum on the individual graph. Because of this, when compared side-by-side, the values do not match.
Aggregate Max:
p1_in: 118.25Mb
p1_out: 31.95Mb
p2_in: 40.15Mb
p2_out: 2.61Mb
Port 1 Max:
p1_in: 304.39Mb
p1_out: 98.77Mb
Port 2 Max:
p2_in: 90.94Mb
p2_out: 14.05Mb
--
Thanks,
skpio
_____
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Cobbled this together with gnuplot via the CSV data generated by billing-data.php – appears to work. No 95%, yet … ? I suspect a method exists to plot it, just haven’t figured it out. Messy config that got me there:
set title 'test'
set autoscale
set datafile separator ','
set style line 1 lc rgb '#0060ad' lt 1 lw 2
set style line 2 lc rgb '#dd181f' lt 1 lw 2
set style data filledcurves
set ylabel 'Bits/Second'
set format y '%.0fM'
set ytic auto
set xdata time
set timefmt '%Y-%m-%dT%H:%M:%S'
set format x '%m/%d'
set xlabel 'Date (Month/Day)'
set style data lines
set key below
set grid
set term png
set output 'test.png'
plot 'test.csv' using 1:($2/1000000) title 'download' ls 1, 'test.csv' using 1:($3/1000000) title 'upload' ls 2
Ignore download/upload, need to confirm the axis.
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Monday, December 4, 2017 1:18 PM To: 'Observium' observium@observium.org Subject: RE: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
Side-thought: if I can use billing-data.php to generate the CSV data for a timeframe, I wonder if there is a local Linux package that would convert that to a graphic? Something like gnuplot?
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Monday, December 4, 2017 12:54 PM To: 'Observium' <observium@observium.org mailto:observium@observium.org > Subject: RE: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
Unfortunately, I can’t automate that in a script to generate our monthly billing reports, nor could I expect anyone on my team or from the billing department to screenshot graphs to give them to customers. It definitely works in a pinch, but not long-term. I don’t expect magic to be worked. I just have to figure out a way around the limitation so it can be scripted/automated into a monthly billing report that can be emailed or exported.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 12:50 PM To: observium@observium.org mailto:observium@observium.org Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
If you just want to save the image, install a screen cap tool like sharex. It'll let you snip the graph on keybind.
The JS isn't actually generating an image, it'll be building a SVG representation of the graph in the DOM that the browser is rendering. The only way to save this is via screenshotting.
adam.
Adam Armstrong
Managing Director & Lead Architect
Observium Limited
On 2017-12-04 19:37:12, Sean Pedersen <spedersen.lists@gmail.com mailto:spedersen.lists@gmail.com > wrote:
I like the JS-driven graphs, I just can’t export them since they’re not a flattened picture. It’s kind of funny, if you right-click and save under the right circumstances you can end up with just a green or purple squiggly line and nothing else.
In a sense we dug ourselves into a hole. They attach the 30 day graphs to billed customers as “proof” of their overage due to some previous accuracy issues, which established a precedent. That means I can’t take it away. It’s really frustrating that drawing a stupid picture is what is keeping me on Cacti and outdated plug-ins/scripts.
Can the image that JS generates be flattened somehow and stored in a temporary cache? Maybe only called via a specific mechanism. Something similar to graphs.php? Throw it a bill ID, to/from, and it takes the same data and draws a graph that can be scraped.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 12:00 PM To: observium@observium.org mailto:observium@observium.org Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
Haha. We just moved away from generating .png billing graphs to generating them in the browser with JS because the graph-drawing library was trash.
Perhaps I can find another solutions for drawing a png from the billing data.
adam.
Adam Armstrong
Managing Director & Lead Architect
Observium Limited
On 2017-12-04 18:57:29, Sean Pedersen <spedersen.lists@gmail.com mailto:spedersen.lists@gmail.com > wrote:
I figured as much, and assumed that’s why you built the better, more dynamic graphing solution. We’ve unfortunately been using RRD 95th for quite a while. We can easily move away from it since no one would know the difference, but the challenge is the “proof” (either automatic or on-demand) we’re required to provide our customers.
Is there a way to generate the billing graph with 95th % in a .png format for export?
If I didn’t have to provide the graph I’d be set with what’s available already. In fact, the billing table in the DB makes it even easier for me to pull the info we used to pull from Cacti.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Monday, December 4, 2017 10:37 AM To: 'Observium' <observium@observium.org mailto:observium@observium.org > Cc: 'Observium' <observium@observium.org mailto:observium@observium.org > Subject: Re: [Observium] Aggregate vs individual port graph maximum value & 95th % on bits graph
The 95th value from rrd is useless for billing purposes.
Adam.
Sent from BlueMail http://www.bluemail.me/r?b=11327
On 4 Dec 2017, at 16:52, Sean Pedersen <spedersen.lists@gmail.com mailto:spedersen.lists@gmail.com > wrote:
For question 2, I added the following to /opt/observium/html/includes/graphs/generic_multi_bits_separated.inc.php after line 146:
$rrd_options .= " LINE1:95thin#aa0000";
$rrd_options .= " LINE1:d95thout#aa0000";
$rrd_options .= " 'COMMENT: \n file://n '";
$rrd_options .= " 'COMMENT:95th %\n'";
$rrd_options .= " GPRINT:95thin:%6.2lf%s\n";
$rrd_options .= " GPRINT:95thout:%6.2lf%s\n";
This gets me what I need in a round-about fashion, but would obviously be overwritten the next time an update comes through for that file. I could change the output around a little bit to move the numbers up to the top of the legend with the rest of the stats, but didn’t want to make extensive changes to the file just for a proof of concept.
Unfortunately, the 95th % value is off by 6Mbps, so there’s probably some adjustments that still need to be made. Either that or it’s a byproduct of how the billing and aggregate ports graphs are generated – deltas vs. RRD info.
Alternatively, if there was an easy means to export the graphs that traffic accounting generates into a regular image with 95th % included, that would be even better.
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Thursday, November 30, 2017 11:01 AM To: observium@observium.org mailto:observium@observium.org Subject: RE: Aggregate vs individual port graph maximum value & 95th % on bits graph
Wanted to bump this – if anyone has time to take a quick look and let me know way or another, it would be really helpful. Trying to replace several aging Cacti servers and billing scripts with Observium and this is all that’s holding me back. ☹
From: Sean Pedersen [mailto:spedersen.lists@gmail.com] Sent: Tuesday, November 28, 2017 7:20 AM To: observium@observium.org mailto:observium@observium.org Subject: Aggregate vs individual port graph maximum value & 95th % on bits graph
I have two graph questions that I haven't been able to get an answer on via IRC, so I thought this might be easier since I can include more information up front:
1st Question: A 2-port aggregate graph (agg_bits.png) shows a maximum value based on :AVERAGE while individual port graphs (port1_bits.png, port2_bits.png) for the ports that make up the aggregate show a maximum value based on :MAX. Is this intentional? If not, would it be possible to remove or toggle graphing average vs. maximum data on individual port graphs? I think both data points are useful, but in general when providing graphs the 5-minute average the only data we’d want to show them. More data below.
2nd Question: The “bits_old” graph includes 95th % both graphed and in the legend. Would it be possible to get the same for the other bits graphs, or at minimum, just bits? I did some digging and didn’t find a means to use options via graphs.php to generate it myself.
Question 1 data:
An aggregate graph (bits) takes the :AVERAGE inbound and outbound of the two ports, using that data to create the graph and populate the legend (min/max/avg, etc.):
DEF:in0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:INOCTETS:AVERAGE DEF:out0=/opt/observium/rrd/dce01.01.mmr01/port-10.rrd:OUTOCTETS:AVERAGE
DEF:in1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:INOCTETS:AVERAGE DEF:out1=/opt/observium/rrd/dce02.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
The individual ports use both :AVERAGE and :MAX.
DEF:outoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:AVERAGE
DEF:inoctets=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:AVERAGE
DEF:outoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:OUTOCTETS:MAX
DEF:inoctets_max=/opt/observium/rrd/dce01.01.mmr02/port-10.rrd:INOCTETS:MAX
(I didn't include all the CDEFs, AREAS, HRULES, etc. for brevity, but I have copies of each RRD command for reference, if needed.)
The "Max" value in the legend reflects the :AVERAGE-based maximum on the aggregate graph and the :MAX-based maximum on the individual graph. Because of this, when compared side-by-side, the values do not match.
Aggregate Max:
p1_in: 118.25Mb
p1_out: 31.95Mb
p2_in: 40.15Mb
p2_out: 2.61Mb
Port 1 Max:
p1_in: 304.39Mb
p1_out: 98.77Mb
Port 2 Max:
p2_in: 90.94Mb
p2_out: 14.05Mb
--
Thanks,
skpio
_____
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
participants (3)
-
Adam Armstrong -
Adam Armstrong -
Sean Pedersen