Renamed interface - how to keep unified history?
I made a change on one of our routers that caused a new ifIndex & name to be assigned to a customer circuit (interface). Observium is correctly reporting the data on the old virtual i/f (which still technically exists) and – naturally – separately on the new virtual i/f. Is there any way to (retroactively???) tell Observium that these are really the same interface, and to display only one graph for the two ifIndexes?
Or, if I correct the situation so the old ifIndex & name is being showing traffic again, is there any way to move, or inject, the collected data on the ‘new’ port into the ‘old’ port’s RRDs? (i.e. avoid a multi-day gap in a multi-year dataset)
We are reasonably up-to-date, running 19.2.9712 – not sure what other info will be helpful here.
Thanks, -Adam
Adam Thompson Consultant, Infrastructure Services [merlin-email-logo] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athompson@merlin.mb.camailto:athompson@merlin.mb.ca www.merlin.mb.cahttp://www.merlin.mb.ca/
Port statistics are just held in a .rrd file on the server. You can copy one file over another file. This is usually not a good thing to do, because you're copying statistics for errors and other things too, but if it's legitimately the same interface, this isn't a problem.
You can see what the files are by clicking on a graph and seeing what RRDs are used (or showing the RRD command for graphs where the RRDs aren't shown in the UI)
adam. On 2019-02-21 15:07:58, Adam Thompson via observium observium@observium.org wrote: I made a change on one of our routers that caused a new ifIndex & name to be assigned to a customer circuit (interface). Observium is correctly reporting the data on the old virtual i/f (which still technically exists) and – naturally – separately on the new virtual i/f. Is there any way to (retroactively???) tell Observium that these are really the same interface, and to display only one graph for the two ifIndexes? Or, if I correct the situation so the old ifIndex & name is being showing traffic again, is there any way to move, or inject, the collected data on the ‘new’ port into the ‘old’ port’s RRDs? (i.e. avoid a multi-day gap in a multi-year dataset) We are reasonably up-to-date, running 19.2.9712 – not sure what other info will be helpful here. Thanks, -Adam Adam Thompson Consultant, Infrastructure Services [merlin-email-logo] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athompson@merlin.mb.ca [mailto:athompson@merlin.mb.ca] www.merlin.mb.ca [http://www.merlin.mb.ca/]
There is a number of ways to join the stats from two rrds together into a single new rrd.
Search for "rrdtool merge".
I've done this a number of times during different device migrations/upgrades in order to preserve long term trend data. But i can't recall the exact commands of the top of my head and I'm AFK.
Hope this helps,
Michael
On February 22, 2019 6:28:17 AM GMT+11:00, Adam Armstrong via observium observium@observium.org wrote:
Port statistics are just held in a .rrd file on the server. You can copy one file over another file. This is usually not a good thing to do, because you're copying statistics for errors and other things too, but if it's legitimately the same interface, this isn't a problem.
You can see what the files are by clicking on a graph and seeing what RRDs are used (or showing the RRD command for graphs where the RRDs aren't shown in the UI)
adam. On 2019-02-21 15:07:58, Adam Thompson via observium observium@observium.org wrote: I made a change on one of our routers that caused a new ifIndex & name to be assigned to a customer circuit (interface). Observium is correctly reporting the data on the old virtual i/f (which still technically exists) and – naturally – separately on the new virtual i/f. Is there any way to (retroactively???) tell Observium that these are really the same interface, and to display only one graph for the two ifIndexes? Or, if I correct the situation so the old ifIndex & name is being showing traffic again, is there any way to move, or inject, the collected data on the ‘new’ port into the ‘old’ port’s RRDs? (i.e. avoid a multi-day gap in a multi-year dataset) We are reasonably up-to-date, running 19.2.9712 – not sure what other info will be helpful here. Thanks, -Adam Adam Thompson Consultant, Infrastructure Services [merlin-email-logo] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athompson@merlin.mb.ca [mailto:athompson@merlin.mb.ca] www.merlin.mb.ca [http://www.merlin.mb.ca/]
https://github.com/mcdarren/rrdtool-merge
Looks like it might do the trick.
adam. On 2019-02-21 20:10:24, Michael via observium observium@observium.org wrote: There is a number of ways to join the stats from two rrds together into a single new rrd.
Search for "rrdtool merge".
I've done this a number of times during different device migrations/upgrades in order to preserve long term trend data. But i can't recall the exact commands of the top of my head and I'm AFK.
Hope this helps,
Michael
On February 22, 2019 6:28:17 AM GMT+11:00, Adam Armstrong via observium wrote:
Port statistics are just held in a .rrd file on the server. You can copy one file over another file. This is usually not a good thing to do, because you're copying statistics for errors and other things too, but if it's legitimately the same interface, this isn't a problem.
You can see what the files are by clicking on a graph and seeing what RRDs are used (or showing the RRD command for graphs where the RRDs aren't shown in the UI)
adam. On 2019-02-21 15:07:58, Adam Thompson via observium wrote: I made a change on one of our routers that caused a new ifIndex & name to be assigned to a customer circuit (interface). Observium is correctly reporting the data on the old virtual i/f (which still technically exists) and – naturally – separately on the new virtual i/f. Is there any way to (retroactively???) tell Observium that these are really the same interface, and to display only one graph for the two ifIndexes?
Or, if I correct the situation so the old ifIndex & name is being showing traffic again, is there any way to move, or inject, the collected data on the ‘new’ port into the ‘old’ port’s RRDs? (i.e. avoid a multi-day gap in a multi-year dataset)
We are reasonably up-to-date, running 19.2.9712 – not sure what other info will be helpful here.
Thanks, -Adam
Adam Thompson Consultant, Infrastructure Services [merlin-email-logo] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athompson@merlin.mb.ca [mailto:athompson@merlin.mb.ca] www.merlin.mb.ca [http://www.merlin.mb.ca/]
_______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Right, I knew that. What I want to do is the metaphorical equivalent of stop observium cat old-port-rrd new-port-rrd > temp mv temp old-port-rrd rm new-port-rrd # now go fix the port names and numbers on the router restart observium
I know RRD files aren’t ASCII to be manipulated this way, so is there a way to merge two RRD files that overlap in time, taking the data from file A up to timestamp X, then the data from file B up to timestamp Y, then switch back to file A? If this was SQL, it would be something like:
SELECT * FROM old_port WHERE timestamp < x UNION SELECT * FROM new_port WHERE timestamp BETWEEN x AND y UNION SELECT * FROM old_port WHERE timestamp > y;
…or am I just, well, screwed?
Adam Thompson Consultant, Infrastructure Services [merlin-email-logo] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athompson@merlin.mb.camailto:athompson@merlin.mb.ca www.merlin.mb.cahttp://www.merlin.mb.ca/
From: observium observium-bounces@observium.org On Behalf Of Adam Armstrong via observium Sent: Thursday, February 21, 2019 1:28 PM To: Adam Thompson via observium observium@observium.org Cc: Adam Armstrong adama@memetic.org Subject: Re: [Observium] Renamed interface - how to keep unified history?
Port statistics are just held in a .rrd file on the server. You can copy one file over another file. This is usually not a good thing to do, because you're copying statistics for errors and other things too, but if it's legitimately the same interface, this isn't a problem.
You can see what the files are by clicking on a graph and seeing what RRDs are used (or showing the RRD command for graphs where the RRDs aren't shown in the UI)
adam.
On 2019-02-21 15:07:58, Adam Thompson via observium <observium@observium.orgmailto:observium@observium.org> wrote: I made a change on one of our routers that caused a new ifIndex & name to be assigned to a customer circuit (interface). Observium is correctly reporting the data on the old virtual i/f (which still technically exists) and – naturally – separately on the new virtual i/f. Is there any way to (retroactively???) tell Observium that these are really the same interface, and to display only one graph for the two ifIndexes?
Or, if I correct the situation so the old ifIndex & name is being showing traffic again, is there any way to move, or inject, the collected data on the ‘new’ port into the ‘old’ port’s RRDs? (i.e. avoid a multi-day gap in a multi-year dataset)
We are reasonably up-to-date, running 19.2.9712 – not sure what other info will be helpful here.
Thanks, -Adam
Adam Thompson Consultant, Infrastructure Services [merlin-email-logo] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athompson@merlin.mb.camailto:athompson@merlin.mb.ca www.merlin.mb.cahttp://www.merlin.mb.ca/
You could probably export them to XML, modify them and then reimport them, but I don't know of any tools which automate that.
adam. On 2019-02-21 20:02:03, Adam Thompson via observium observium@observium.org wrote: Right, I knew that. What I want to do is the metaphorical equivalent of stop observium cat old-port-rrd new-port-rrd > temp mv temp old-port-rrd rm new-port-rrd # now go fix the port names and numbers on the router restart observium I know RRD files aren’t ASCII to be manipulated this way, so is there a way to merge two RRD files that overlap in time, taking the data from file A up to timestamp X, then the data from file B up to timestamp Y, then switch back to file A? If this was SQL, it would be something like: SELECT * FROM old_port WHERE timestamp < x UNION SELECT * FROM new_port WHERE timestamp BETWEEN x AND y UNION SELECT * FROM old_port WHERE timestamp > y; …or am I just, well, screwed? Adam Thompson Consultant, Infrastructure Services [merlin-email-logo] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athompson@merlin.mb.ca [mailto:athompson@merlin.mb.ca] www.merlin.mb.ca [http://www.merlin.mb.ca/] From: observium observium-bounces@observium.org On Behalf Of Adam Armstrong via observium Sent: Thursday, February 21, 2019 1:28 PM To: Adam Thompson via observium observium@observium.org Cc: Adam Armstrong adama@memetic.org Subject: Re: [Observium] Renamed interface - how to keep unified history? Port statistics are just held in a .rrd file on the server. You can copy one file over another file. This is usually not a good thing to do, because you're copying statistics for errors and other things too, but if it's legitimately the same interface, this isn't a problem. You can see what the files are by clicking on a graph and seeing what RRDs are used (or showing the RRD command for graphs where the RRDs aren't shown in the UI) adam. On 2019-02-21 15:07:58, Adam Thompson via observium <observium@observium.org [mailto:observium@observium.org]> wrote: I made a change on one of our routers that caused a new ifIndex & name to be assigned to a customer circuit (interface). Observium is correctly reporting the data on the old virtual i/f (which still technically exists) and – naturally – separately on the new virtual i/f. Is there any way to (retroactively???) tell Observium that these are really the same interface, and to display only one graph for the two ifIndexes? Or, if I correct the situation so the old ifIndex & name is being showing traffic again, is there any way to move, or inject, the collected data on the ‘new’ port into the ‘old’ port’s RRDs? (i.e. avoid a multi-day gap in a multi-year dataset) We are reasonably up-to-date, running 19.2.9712 – not sure what other info will be helpful here. Thanks, -Adam Adam Thompson Consultant, Infrastructure Services [merlin-email-logo] 100 - 135 Innovation Drive Winnipeg, MB, R3T 6A8 (204) 977-6824 or 1-800-430-6404 (MB only) athompson@merlin.mb.ca [mailto:athompson@merlin.mb.ca] www.merlin.mb.ca [http://www.merlin.mb.ca/]
participants (3)
-
Adam Armstrong -
Adam Thompson -
Michael