Linux server RRDs-port info after server reboot gets messy
Hello,
We started to use Observium for Linux server. So far everything was good, but after reboot of Linux server some ports swaped the data and show incorrect values.
Is there any prevention to this ?
on the server we're using net-snmp-utils
Hi Edvinas,
This is an issue with Linux not having persistent port ifIndexes. It shouldn't happen frequently, but may happen when the kernel changes the order that it enumerates ports.
There's not really a good way to mitigate it at the moment.
adam. On 2018-06-12 15:07:31, Edvinas K edvinas.email@gmail.com wrote: Hello,
We started to use Observium for Linux server. So far everything was good, but after reboot of Linux server some ports swaped the data and show incorrect values.
Is there any prevention to this ?
on the server we're using net-snmp-utils _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hi Adam!
Wouldn't the concept outlined and implemented in LibreNMS solve the problem? Having a portID for every port and creating a setting for each device (with some sane default) whether this portID should get associated with an ifIndex or with an ifName. See Inconsistent port/interface graphs for unix systems #386 https://github.com/librenms/librenms/issues/386
It seems like a plausible fix to this problem, not to mention it has a side effect fixing some other possible problems like dynamically allocated VM interfaces etc.
Regards, Tylla
On 2018-06-12 16:18, Adam Armstrong wrote:
Hi Edvinas,
This is an issue with Linux not having persistent port ifIndexes. It shouldn't happen frequently, but may happen when the kernel changes the order that it enumerates ports.
There's not really a good way to mitigate it at the moment.
adam.
On 2018-06-12 15:07:31, Edvinas K edvinas.email@gmail.com wrote:
Hello,
We started to use Observium for Linux server. So far everything was good, but after reboot of Linux server some ports swaped the data and show incorrect values.
Is there any prevention to this ?
on the server we're using net-snmp-utils _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Moreover, now that I think about, maybe a third option for MAC address (ifPhysAddress if I'm not completely wrong) would be usable as well.
On 2018-06-12 16:56, Attila Nagy wrote:
Hi Adam!
Wouldn't the concept outlined and implemented in LibreNMS solve the problem? Having a portID for every port and creating a setting for each device (with some sane default) whether this portID should get associated with an ifIndex or with an ifName. See Inconsistent port/interface graphs for unix systems #386 https://github.com/librenms/librenms/issues/386
It seems like a plausible fix to this problem, not to mention it has a side effect fixing some other possible problems like dynamically allocated VM interfaces etc.
Regards, Tylla
On 2018-06-12 16:18, Adam Armstrong wrote:
Hi Edvinas,
This is an issue with Linux not having persistent port ifIndexes. It shouldn't happen frequently, but may happen when the kernel changes the order that it enumerates ports.
There's not really a good way to mitigate it at the moment.
adam.
On 2018-06-12 15:07:31, Edvinas K edvinas.email@gmail.com wrote:
Hello,
We started to use Observium for Linux server. So far everything was good, but after reboot of Linux server some ports swaped the data and show incorrect values.
Is there any prevention to this ?
on the server we're using net-snmp-utils _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Our intended fix (it's been discussed about a million times but it never annoys anyone enough to actually do the work to fix it) was to have the code that generates the rrd file name switch between ifIndex and ifDescr depending upon the OS.
I think using ifPhysAddress might be troublesome when the problem you're trying to solve is caused by virtual interfaces, because the physical addresses might differ, too.
Probably it's best to stick to switching between ifIndex and ifAlias (or ifName) based on the OS (or other rules, i think it would actually be better to have a more complex ruleset, so we switch to this method also when dealing with temporary interfaces on other platforms).
If you are lucky our resident russian fixer might be interested in making this work (I'm working on other parts of the code at the moment and don't want to start messing with this) :)
adam. On 2018-06-12 16:20:10, Attila Nagy tylla_at_memetic.org@tylla.hu wrote: Moreover, now that I think about, maybe a third option for MAC address (ifPhysAddress if I'm not completely wrong) would be usable as well.
On 2018-06-12 16:56, Attila Nagy wrote:
Hi Adam!
Wouldn't the concept outlined and implemented in solve the problem? Having a portID for every port and creating a setting for each device (with some sane default) whether this portID should get associated with an ifIndex or with an ifName.
It seems like a plausible fix to this problem, not to mention it has a side effect fixing some other possible problems like dynamically allocated VM interfaces etc.
Regards, Tylla
On 2018-06-12 16:18, Adam Armstrong wrote:
Hi Edvinas,
This is an issue with Linux not having persistent port ifIndexes. It shouldn't happen frequently, but may happen when the kernel changes the order that it enumerates ports.
There's not really a good way to mitigate it at the moment.
adam. On 2018-06-12 15:07:31, Edvinas K edvinas.email@gmail.com [mailto:edvinas.email@gmail.com] wrote: Hello,
We started to use Observium for Linux server. So far everything was good, but after reboot of Linux server some ports swaped the data and show incorrect values.
Is there any prevention to this ?
on the server we're using net-snmp-utils _______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
_______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
_______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
Well I might have washed together two distinct problems... sorry for that.
ifPhysAddress is not really the solution for virtual interfaces, it would be more a solution for any OS where changing the physical order of interface cards changes lots of things. E.g. Windows, where the only fixed thing is the MAC address, everything else can change (interface number, connection name etc.) The same goes for a machine where you have to override the MAC address to a fixed one (albeit I have to admit this is a really small segment of the possible use-cases :))))
Regards, Tylla
On 2018-06-12 17:26, Adam Armstrong wrote:
Our intended fix (it's been discussed about a million times but it never annoys anyone enough to actually do the work to fix it) was to have the code that generates the rrd file name switch between ifIndex and ifDescr depending upon the OS.
I think using ifPhysAddress might be troublesome when the problem you're trying to solve is caused by virtual interfaces, because the physical addresses might differ, too.
Probably it's best to stick to switching between ifIndex and ifAlias (or ifName) based on the OS (or other rules, i think it would actually be better to have a more complex ruleset, so we switch to this method also when dealing with temporary interfaces on other platforms).
If you are lucky our resident russian fixer might be interested in making this work (I'm working on other parts of the code at the moment and don't want to start messing with this) :)
adam.
On 2018-06-12 16:20:10, Attila Nagy tylla_at_memetic.org@tylla.hu wrote:
Moreover, now that I think about, maybe a third option for MAC address (ifPhysAddress if I'm not completely wrong) would be usable as well.
On 2018-06-12 16:56, Attila Nagy wrote:
Hi Adam!
Wouldn't the concept outlined and implemented in solve the problem? Having a portID for every port and creating a setting for each device (with some sane default) whether this portID should get associated with an ifIndex or with an ifName.
It seems like a plausible fix to this problem, not to mention it has a side effect fixing some other possible problems like dynamically allocated VM interfaces etc.
Regards, Tylla
On 2018-06-12 16:18, Adam Armstrong wrote:
Hi Edvinas,
This is an issue with Linux not having persistent port ifIndexes. It shouldn't happen frequently, but may happen when the kernel changes the order that it enumerates ports.
There's not really a good way to mitigate it at the moment.
adam.
On 2018-06-12 15:07:31, Edvinas K edvinas.email@gmail.com wrote:
Hello,
We started to use Observium for Linux server. So far everything was good, but after reboot of Linux server some ports swaped the data and show incorrect values.
Is there any prevention to this ?
on the server we're using net-snmp-utils _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
participants (3)
-
Adam Armstrong -
Attila Nagy -
Edvinas K