Possible pseudowire bug - Backup PSW's classed as DOWN/ALERT, results in broken PW alerting
Hi List,
Just a quick one to check that I do not need to configure something extra for this to work?
We have lots of Pseudowire's configured using IOS-XR, IOS-XE and IOS, and these all show in Observium with the pwID, and the Local and Remote details (so Observium is correctly matching the stanzas across the two endpoints as being the same service) which is awesome :)
However when configuring 'Backup' Pseudowire, the standby/backup Pseudowire is shown as 'Down' and 'Alert' in Observium? This is really confusing for support staff who do not understand if down/alert means something is wrong or not, and as such we are currently unable to monitor any pseudowire and have all alerts disabled for them :(
The attached image shows an example, of how all the PWs to the secondary device are reported as down and in an alert state! They are all down but in the normal state as standby/backup pseudowires.
Below shows the output for the PW states, and it can be seen that a status of 'sec' and 'SB' is provided for Standby/Backup (the Pseudowire is explicitly configured with a 'Backup' path etc) pseudowires.
BYF-CGW-2#show xconnect all Legend: XC ST=Xconnect State S1=Segment1 State S2=Segment2 State UP=Up DN=Down AD=Admin Down IA=Inactive SB=Standby HS=Hot Standby RV=Recovering NH=No Hardware
XC ST Segment 1 S1 Segment 2 S2 ------+---------------------------------+--+---------------------------------+-- UP pri ac Po20.339:339(Eth VLAN) UP mpls <remote-rt-1-ip>:339 UP IA sec ac Po20.339:339(Eth VLAN) UP mpls <remote-rt-2-ip>:339 SB UP pri ac Po20.341:341(Eth VLAN) UP mpls <remote-rt-1-ip>:341 UP IA sec ac Po20.341:341(Eth VLAN) UP mpls <remote-rt-2-ip>:341 SB UP pri ac Po20.348:348(Eth VLAN) UP mpls <remote-rt-1-ip>:348 UP IA sec ac Po20.348:348(Eth VLAN) UP mpls <remote-rt-2-ip>:348 SB
So the question I guess is, do the OIDs show the third status of 'SB' (or the 'XC' and 'ST' states of, 'IA' & 'sec' = Standby/Backup etc) so we can have something like; 'UP' = Green (Up), 'SB' = Amber (Up), 'DN' = Red (Down/Alert)? Without this pseudowire alerting and state monitoring is broken :(
As always, Thanks for all your time and effort with Observium Kind regards, Andy.
Hi Andy,
I had a ticket (#684907603) with Cisco TAC for IOS-XR about this "issue". Looks similar to our VPLS. Final answer: will not be fixed.
Original quote: " Hello Denis Below is the feedback I received from the BU Unfortunately, there are just a few available values in the MIB... up/local-up maps to cpwVcOperStatus "up" anything else maps to "down", including standby. The MIB defines the "dormant" value but it's description does not match the state of a PW in standby. You can also check this further on the below URL http://snmp.cloudapps.cisco.com/Support/SNMP/do/BrowseOID.do?objectInput=1.3... "
Mit freundlichem Gruß Stadtwerke Norderstedt
Denis Klimek
Professional Network Engineer IP-Systemtechnik
Tel: 040 / 521 04 - 1049 Mobil: 0151 / 652 219 06
dklimek@stadtwerke-norderstedt.demailto:dklimek@stadtwerke-norderstedt.de www.stadtwerke-norderstedt.dehttp://www.stadtwerke-norderstedt.de/
Von: observium [mailto:observium-bounces@observium.org] Im Auftrag von Andrew Lemin via observium Gesendet: Dienstag, 11. Dezember 2018 14:27 An: 'observium@observium.org' Cc: Andrew Lemin Betreff: [Observium] Possible pseudowire bug - Backup PSW's classed as DOWN/ALERT, results in broken PW alerting
Hi List,
Just a quick one to check that I do not need to configure something extra for this to work?
We have lots of Pseudowire's configured using IOS-XR, IOS-XE and IOS, and these all show in Observium with the pwID, and the Local and Remote details (so Observium is correctly matching the stanzas across the two endpoints as being the same service) which is awesome :)
However when configuring 'Backup' Pseudowire, the standby/backup Pseudowire is shown as 'Down' and 'Alert' in Observium? This is really confusing for support staff who do not understand if down/alert means something is wrong or not, and as such we are currently unable to monitor any pseudowire and have all alerts disabled for them :(
The attached image shows an example, of how all the PWs to the secondary device are reported as down and in an alert state! They are all down but in the normal state as standby/backup pseudowires.
Below shows the output for the PW states, and it can be seen that a status of 'sec' and 'SB' is provided for Standby/Backup (the Pseudowire is explicitly configured with a 'Backup' path etc) pseudowires.
BYF-CGW-2#show xconnect all Legend: XC ST=Xconnect State S1=Segment1 State S2=Segment2 State UP=Up DN=Down AD=Admin Down IA=Inactive SB=Standby HS=Hot Standby RV=Recovering NH=No Hardware
XC ST Segment 1 S1 Segment 2 S2 ------+---------------------------------+--+---------------------------------+-- UP pri ac Po20.339:339(Eth VLAN) UP mpls <remote-rt-1-ip>:339 UP IA sec ac Po20.339:339(Eth VLAN) UP mpls <remote-rt-2-ip>:339 SB UP pri ac Po20.341:341(Eth VLAN) UP mpls <remote-rt-1-ip>:341 UP IA sec ac Po20.341:341(Eth VLAN) UP mpls <remote-rt-2-ip>:341 SB UP pri ac Po20.348:348(Eth VLAN) UP mpls <remote-rt-1-ip>:348 UP IA sec ac Po20.348:348(Eth VLAN) UP mpls <remote-rt-2-ip>:348 SB
So the question I guess is, do the OIDs show the third status of 'SB' (or the 'XC' and 'ST' states of, 'IA' & 'sec' = Standby/Backup etc) so we can have something like; 'UP' = Green (Up), 'SB' = Amber (Up), 'DN' = Red (Down/Alert)? Without this pseudowire alerting and state monitoring is broken :(
As always, Thanks for all your time and effort with Observium Kind regards, Andy.
Hrm, if cpwVcOperStatus report down, than this is definitely is DOWN event, not possible to fix it. But if standby will report "dormant" state, than possible to fix.
Klimek, Denis via observium wrote on 11/12/2018 16:53:
Hi Andy,
I had a ticket (#684907603) with Cisco TAC for IOS-XR about this “issue”. Looks similar to our VPLS. Final answer: will not be fixed.
Original quote:
“
Hello Denis Below is the feedback I received from the BU Unfortunately, there are just a few available values in the MIB... up/local-up maps to cpwVcOperStatus "up" anything else maps to "down", including standby. The MIB defines the "dormant" value but it's description does not match the state of a PW in standby. You can also check this further on the below URL http://snmp.cloudapps.cisco.com/Support/SNMP/do/BrowseOID.do?objectInput=1.3...
“
Mit freundlichem Gruß
Stadtwerke Norderstedt
*Denis Klimek*
Professional Network Engineer
IP-Systemtechnik
Tel: 040 / 521 04 – 1049
Mobil: 0151 / 652 219 06
dklimek@stadtwerke-norderstedt.de mailto:dklimek@stadtwerke-norderstedt.de
www.stadtwerke-norderstedt.de http://www.stadtwerke-norderstedt.de/
*Von:*observium [mailto:observium-bounces@observium.org] *Im Auftrag von *Andrew Lemin via observium *Gesendet:* Dienstag, 11. Dezember 2018 14:27 *An:* 'observium@observium.org' *Cc:* Andrew Lemin *Betreff:* [Observium] Possible pseudowire bug - Backup PSW's classed as DOWN/ALERT, results in broken PW alerting
Hi List,
Just a quick one to check that I do not need to configure something extra for this to work?
We have lots of Pseudowire’s configured using IOS-XR, IOS-XE and IOS, and these all show in Observium with the pwID, and the Local and Remote details (so Observium is correctly matching the stanzas across the two endpoints as being the same service) which is awesome J
However when configuring ‘Backup’ Pseudowire, the standby/backup Pseudowire is shown as ‘Down’ and ‘Alert’ in Observium?
This is really confusing for support staff who do not understand if down/alert means something is wrong or not, and as such we are currently unable to monitor any pseudowire and have all alerts disabled for them L
The attached image shows an example, of how all the PWs to the secondary device are reported as down and in an alert state! They are all down but in the normal state as standby/backup pseudowires.
Below shows the output for the PW states, and it can be seen that a status of ‘sec’ and ‘SB’ is provided for Standby/Backup (the Pseudowire is explicitly configured with a ‘Backup’ path etc) pseudowires.
BYF-CGW-2#show xconnect all
Legend: XC ST=Xconnect State S1=Segment1 State S2=Segment2 State
UP=Up DN=Down AD=Admin Down IA=Inactive
SB=Standby HS=Hot Standby RV=Recovering NH=No Hardware
XC ST Segment 1 S1 Segment 2 S2
------+---------------------------------+--+---------------------------------+--
UP pri ac Po20.339:339(Eth VLAN) UP mpls <remote-rt-1-ip>:339 UP
IA sec ac Po20.339:339(Eth VLAN) UP mpls <remote-rt-2-ip>:339 SB
UP pri ac Po20.341:341(Eth VLAN) UP mpls <remote-rt-1-ip>:341 UP
IA sec ac Po20.341:341(Eth VLAN) UP mpls <remote-rt-2-ip>:341 SB
UP pri ac Po20.348:348(Eth VLAN) UP mpls <remote-rt-1-ip>:348 UP
IA sec ac Po20.348:348(Eth VLAN) UP mpls <remote-rt-2-ip>:348 SB
So the question I guess is, do the OIDs show the third status of ‘SB’ (or the ‘XC’ and ‘ST’ states of, ‘IA’ & ‘sec’ = Standby/Backup etc) so we can have something like; ‘UP’ = Green (Up), ‘SB’ = Amber (Up), ‘DN’ = Red (Down/Alert)?
Without this pseudowire alerting and state monitoring is broken L
As always, Thanks for all your time and effort with Observium
Kind regards, Andy.
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hi Klimek
Ahh, So Mr.Cisco how are we supposed to actually monitor the stuff you support..! And don’t say screen-scraping (we are not stuck in the 90’s).
Another reason to move away from Cisco then.. ☹
For others, I can confirm by walking the OID “1.3.6.1.4.1.9.10.106.1.2.1.26” I only see Up or Down in IOS-XE, IOS-XR and IOS! At least they have failed consistently.. 😉
This is also an issue for our VPLS PW’s and not just the P2P PW’s..
Thanks for your help 😊
From: "Klimek, Denis" DKlimek@Stadtwerke-Norderstedt.de Date: Tuesday, 11 December 2018 at 13:53 To: 'Observium' observium@observium.org Cc: Andrew Lemin AndrewL@4d-dc.com Subject: AW: Possible pseudowire bug - Backup PSW's classed as DOWN/ALERT, results in broken PW alerting
Hi Andy,
I had a ticket (#684907603) with Cisco TAC for IOS-XR about this “issue”. Looks similar to our VPLS. Final answer: will not be fixed.
Original quote: “ Hello Denis Below is the feedback I received from the BU Unfortunately, there are just a few available values in the MIB... up/local-up maps to cpwVcOperStatus "up" anything else maps to "down", including standby. The MIB defines the "dormant" value but it's description does not match the state of a PW in standby. You can also check this further on the below URL http://snmp.cloudapps.cisco.com/Support/SNMP/do/BrowseOID.do?objectInput=1.3... “
Mit freundlichem Gruß Stadtwerke Norderstedt
Denis Klimek
Professional Network Engineer IP-Systemtechnik
Tel: 040 / 521 04 – 1049 Mobil: 0151 / 652 219 06
dklimek@stadtwerke-norderstedt.demailto:dklimek@stadtwerke-norderstedt.de www.stadtwerke-norderstedt.dehttp://www.stadtwerke-norderstedt.de/
Von: observium [mailto:observium-bounces@observium.org] Im Auftrag von Andrew Lemin via observium Gesendet: Dienstag, 11. Dezember 2018 14:27 An: 'observium@observium.org' Cc: Andrew Lemin Betreff: [Observium] Possible pseudowire bug - Backup PSW's classed as DOWN/ALERT, results in broken PW alerting
Hi List,
Just a quick one to check that I do not need to configure something extra for this to work?
We have lots of Pseudowire’s configured using IOS-XR, IOS-XE and IOS, and these all show in Observium with the pwID, and the Local and Remote details (so Observium is correctly matching the stanzas across the two endpoints as being the same service) which is awesome ☺
However when configuring ‘Backup’ Pseudowire, the standby/backup Pseudowire is shown as ‘Down’ and ‘Alert’ in Observium? This is really confusing for support staff who do not understand if down/alert means something is wrong or not, and as such we are currently unable to monitor any pseudowire and have all alerts disabled for them ☹
The attached image shows an example, of how all the PWs to the secondary device are reported as down and in an alert state! They are all down but in the normal state as standby/backup pseudowires.
Below shows the output for the PW states, and it can be seen that a status of ‘sec’ and ‘SB’ is provided for Standby/Backup (the Pseudowire is explicitly configured with a ‘Backup’ path etc) pseudowires.
BYF-CGW-2#show xconnect all Legend: XC ST=Xconnect State S1=Segment1 State S2=Segment2 State UP=Up DN=Down AD=Admin Down IA=Inactive SB=Standby HS=Hot Standby RV=Recovering NH=No Hardware
XC ST Segment 1 S1 Segment 2 S2 ------+---------------------------------+--+---------------------------------+-- UP pri ac Po20.339:339(Eth VLAN) UP mpls <remote-rt-1-ip>:339 UP IA sec ac Po20.339:339(Eth VLAN) UP mpls <remote-rt-2-ip>:339 SB UP pri ac Po20.341:341(Eth VLAN) UP mpls <remote-rt-1-ip>:341 UP IA sec ac Po20.341:341(Eth VLAN) UP mpls <remote-rt-2-ip>:341 SB UP pri ac Po20.348:348(Eth VLAN) UP mpls <remote-rt-1-ip>:348 UP IA sec ac Po20.348:348(Eth VLAN) UP mpls <remote-rt-2-ip>:348 SB
So the question I guess is, do the OIDs show the third status of ‘SB’ (or the ‘XC’ and ‘ST’ states of, ‘IA’ & ‘sec’ = Standby/Backup etc) so we can have something like; ‘UP’ = Green (Up), ‘SB’ = Amber (Up), ‘DN’ = Red (Down/Alert)? Without this pseudowire alerting and state monitoring is broken ☹
As always, Thanks for all your time and effort with Observium Kind regards, Andy.
Hi Andy,
until now there is no way for monitoring the “real/correct” states of a PW trough SNMP on Cisco devices. We are monitoring it by checking if the PW is “UP” and if not: Alert. All false-positive on a device reboot, new PW or sth similar will be “Ignored until OK” suppressed.
Mit freundlichem Gruß Stadtwerke Norderstedt
Denis Klimek
Professional Network Engineer IP-Systemtechnik
Tel: 040 / 521 04 – 1049 Mobil: 0151 / 652 219 06
dklimek@stadtwerke-norderstedt.demailto:dklimek@stadtwerke-norderstedt.de www.stadtwerke-norderstedt.dehttp://www.stadtwerke-norderstedt.de/
Von: Andrew Lemin [mailto:AndrewL@4d-dc.com] Gesendet: Mittwoch, 12. Dezember 2018 15:37 An: Klimek, Denis; 'Observium' Betreff: Re: Possible pseudowire bug - Backup PSW's classed as DOWN/ALERT, results in broken PW alerting
Hi Klimek
Ahh, So Mr.Cisco how are we supposed to actually monitor the stuff you support..! And don’t say screen-scraping (we are not stuck in the 90’s).
Another reason to move away from Cisco then.. ☹
For others, I can confirm by walking the OID “1.3.6.1.4.1.9.10.106.1.2.1.26” I only see Up or Down in IOS-XE, IOS-XR and IOS! At least they have failed consistently.. 😉
This is also an issue for our VPLS PW’s and not just the P2P PW’s..
Thanks for your help 😊
From: "Klimek, Denis" DKlimek@Stadtwerke-Norderstedt.de Date: Tuesday, 11 December 2018 at 13:53 To: 'Observium' observium@observium.org Cc: Andrew Lemin AndrewL@4d-dc.com Subject: AW: Possible pseudowire bug - Backup PSW's classed as DOWN/ALERT, results in broken PW alerting
Hi Andy,
I had a ticket (#684907603) with Cisco TAC for IOS-XR about this “issue”. Looks similar to our VPLS. Final answer: will not be fixed.
Original quote: “ Hello Denis Below is the feedback I received from the BU Unfortunately, there are just a few available values in the MIB... up/local-up maps to cpwVcOperStatus "up" anything else maps to "down", including standby. The MIB defines the "dormant" value but it's description does not match the state of a PW in standby. You can also check this further on the below URL http://snmp.cloudapps.cisco.com/Support/SNMP/do/BrowseOID.do?objectInput=1.3... “
Mit freundlichem Gruß Stadtwerke Norderstedt
Denis Klimek
Professional Network Engineer IP-Systemtechnik
Tel: 040 / 521 04 – 1049 Mobil: 0151 / 652 219 06
dklimek@stadtwerke-norderstedt.demailto:dklimek@stadtwerke-norderstedt.de www.stadtwerke-norderstedt.dehttp://www.stadtwerke-norderstedt.de/
Von: observium [mailto:observium-bounces@observium.org] Im Auftrag von Andrew Lemin via observium Gesendet: Dienstag, 11. Dezember 2018 14:27 An: 'observium@observium.org' Cc: Andrew Lemin Betreff: [Observium] Possible pseudowire bug - Backup PSW's classed as DOWN/ALERT, results in broken PW alerting
Hi List,
Just a quick one to check that I do not need to configure something extra for this to work?
We have lots of Pseudowire’s configured using IOS-XR, IOS-XE and IOS, and these all show in Observium with the pwID, and the Local and Remote details (so Observium is correctly matching the stanzas across the two endpoints as being the same service) which is awesome ☺
However when configuring ‘Backup’ Pseudowire, the standby/backup Pseudowire is shown as ‘Down’ and ‘Alert’ in Observium? This is really confusing for support staff who do not understand if down/alert means something is wrong or not, and as such we are currently unable to monitor any pseudowire and have all alerts disabled for them ☹
The attached image shows an example, of how all the PWs to the secondary device are reported as down and in an alert state! They are all down but in the normal state as standby/backup pseudowires.
Below shows the output for the PW states, and it can be seen that a status of ‘sec’ and ‘SB’ is provided for Standby/Backup (the Pseudowire is explicitly configured with a ‘Backup’ path etc) pseudowires.
BYF-CGW-2#show xconnect all Legend: XC ST=Xconnect State S1=Segment1 State S2=Segment2 State UP=Up DN=Down AD=Admin Down IA=Inactive SB=Standby HS=Hot Standby RV=Recovering NH=No Hardware
XC ST Segment 1 S1 Segment 2 S2 ------+---------------------------------+--+---------------------------------+-- UP pri ac Po20.339:339(Eth VLAN) UP mpls <remote-rt-1-ip>:339 UP IA sec ac Po20.339:339(Eth VLAN) UP mpls <remote-rt-2-ip>:339 SB UP pri ac Po20.341:341(Eth VLAN) UP mpls <remote-rt-1-ip>:341 UP IA sec ac Po20.341:341(Eth VLAN) UP mpls <remote-rt-2-ip>:341 SB UP pri ac Po20.348:348(Eth VLAN) UP mpls <remote-rt-1-ip>:348 UP IA sec ac Po20.348:348(Eth VLAN) UP mpls <remote-rt-2-ip>:348 SB
So the question I guess is, do the OIDs show the third status of ‘SB’ (or the ‘XC’ and ‘ST’ states of, ‘IA’ & ‘sec’ = Standby/Backup etc) so we can have something like; ‘UP’ = Green (Up), ‘SB’ = Amber (Up), ‘DN’ = Red (Down/Alert)? Without this pseudowire alerting and state monitoring is broken ☹
As always, Thanks for all your time and effort with Observium Kind regards, Andy.
Hi
Im trying to create some alerts in Observium 18.10.9472 to alert when a fiber interface goes into a warning or alert state for the receive or transmit power dBm. I am only interested in ports that are in a no-shut config. I have made a port group which contains all ports in a no shutdown state but cant see a way to map this to the associations:
How can I do this as there is no way to map that group to the alert?? The issue is that I need to exclude any ports in no-shut as these ports are reporting an alarm as there is no power being received on these ports.
Thanks
Richard
participants (4)
-
Andrew Lemin -
Klimek, Denis -
Mike Stupalov -
Richard Savage