Spent some time trying to get Observium to deal with MIBs as produced by LLDP (Link Layer Discovery Protocol) on a couple of diverse network equipment types. Running the latest CE edition 0.15.6.6430.
After initial meager success (only two boxes recognized and several L2 neighbours reported with wrong interfaces), a couple of changes in includes/discovery/discovery-protocols/lldp-mib.inc.php brought some sense into results.
There are essentially two problems there:
- some lldpRemoteSystemsData OIDs could not be correlated with an interface on the other side (for various reasons, see below);
- a fallback mechanism which tries to find a match on an interface's MAC address produces a wrong match, because typically all ports on larger switches have the same MAC address and the heuristics just picks the first match, which is almost always wrong. In case of multiple matches the matching on MAC should just fail, which at least would not produce a senseless match causing operators to scratch their heads and wonder who crossed the cables. The comment in lldp-mib.inc.php : // Find the device by MAC address, still matches multiple ports sometimes, we use the first one and hope we're lucky if very much misguided, as all port MAC addresses on a switch are typically the same, at least on all device types that I checked.
The first issue above needs more explanation. I found a couple of different reasons for a failed match, as follows.
Case 1:
$ snmpbulkwalk -v2c -c ... -M mibs -m LLDP-MIB ... lldpRemoteSystemsData LLDP-MIB::lldpRemChassisIdSubtype.5360466.52.2 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisId.5360466.52.2 = Hex-STRING: C4 54 44 1E 71 4E LLDP-MIB::lldpRemPortIdSubtype.5360466.52.2 = INTEGER: local(7) LLDP-MIB::lldpRemPortId.5360466.52.2 = Hex-STRING: 30 2F 34 37 00 LLDP-MIB::lldpRemPortDesc.5360466.52.2 = STRING: "Slot: 0 Port: 47 10G - Level" LLDP-MIB::lldpRemSysName.5360466.52.2 = STRING: "<hidden>" LLDP-MIB::lldpRemSysDesc.5360466.52.2 = STRING: "T3048-LY2R L2/L3 Switch" LLDP-MIB::lldpRemSysCapSupported.5360466.52.2 = BITS: 28 00 bridge(2) router(4) LLDP-MIB::lldpRemSysCapEnabled.5360466.52.2 = BITS: 20 00 bridge(2) LLDP-MIB::lldpRemManAddrIfSubtype.5360466.52.2.6.6.196.84.68.30.113.78 = INTEGER: ifIndex(2) LLDP-MIB::lldpRemManAddrIfId.5360466.52.2.6.6.196.84.68.30.113.78 = INTEGER: 69 LLDP-MIB::lldpRemManAddrOID.5360466.52.2.6.6.196.84.68.30.113.78 = OID: iso.3.6.1.4.1.7244 LLDP-MIB::lldpRemOrgDefInfo.5360466.52.2.0.18.15.1.1 = Hex-STRING: 01 00 00 00 24 LLDP-MIB::lldpRemOrgDefInfo.5360466.52.2.0.18.15.2.1 = Hex-STRING: 00 00 00 LLDP-MIB::lldpRemOrgDefInfo.5360466.52.2.0.18.15.3.1 = Hex-STRING: 01 00 00 00 00 LLDP-MIB::lldpRemOrgDefInfo.5360466.52.2.0.18.15.4.1 = Hex-STRING: 23 F2 LLDP-MIB::lldpRemOrgDefInfo.5360466.52.2.0.128.194.1.1 = Hex-STRING: 00 01 LLDP-MIB::lldpRemOrgDefInfo.5360466.52.2.0.128.194.2.1 = Hex-STRING: 02 00 00 LLDP-MIB::lldpRemOrgDefInfo.5360466.52.2.0.128.194.3.1 = Hex-STRING: 00 01 00
Case 2: LLDP-MIB::lldpRemChassisIdSubtype.9511080.4.5 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9511149.2.6 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9511174.3.7 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9511209.26.8 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9587238.48.9 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9588774.27.10 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9588907.28.11 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9592381.42.12 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9827133.8.14 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9843364.9.16 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisId.9511080.4.5 = Hex-STRING: 00 04 96 34 CB 7D LLDP-MIB::lldpRemChassisId.9511149.2.6 = Hex-STRING: 00 04 96 34 CB 78 LLDP-MIB::lldpRemChassisId.9511174.3.7 = Hex-STRING: 00 04 96 34 CB 8D LLDP-MIB::lldpRemChassisId.9511209.26.8 = Hex-STRING: 00 04 96 34 CB 93 LLDP-MIB::lldpRemChassisId.9587238.48.9 = Hex-STRING: A4 8C DB 0C 4B 00 LLDP-MIB::lldpRemChassisId.9588774.27.10 = Hex-STRING: 00 04 96 34 CB 7F LLDP-MIB::lldpRemChassisId.9588907.28.11 = Hex-STRING: 00 04 96 34 CB 8B LLDP-MIB::lldpRemChassisId.9592381.42.12 = Hex-STRING: 44 AD D9 6C 2D 80 LLDP-MIB::lldpRemChassisId.9827133.8.14 = Hex-STRING: C4 54 44 1E 71 4E LLDP-MIB::lldpRemChassisId.9843364.9.16 = Hex-STRING: 2C 60 0C 33 8F FF LLDP-MIB::lldpRemPortIdSubtype.9511080.4.5 = INTEGER: interfaceName(5) LLDP-MIB::lldpRemPortIdSubtype.9511149.2.6 = INTEGER: interfaceName(5) LLDP-MIB::lldpRemPortIdSubtype.9511174.3.7 = INTEGER: interfaceName(5) LLDP-MIB::lldpRemPortIdSubtype.9511209.26.8 = INTEGER: interfaceName(5) LLDP-MIB::lldpRemPortIdSubtype.9587238.48.9 = INTEGER: local(7) LLDP-MIB::lldpRemPortIdSubtype.9588774.27.10 = INTEGER: interfaceName(5) LLDP-MIB::lldpRemPortIdSubtype.9588907.28.11 = INTEGER: interfaceName(5) LLDP-MIB::lldpRemPortIdSubtype.9592381.42.12 = INTEGER: interfaceName(5) LLDP-MIB::lldpRemPortIdSubtype.9827133.8.14 = INTEGER: interfaceAlias(1) LLDP-MIB::lldpRemPortIdSubtype.9843364.9.16 = INTEGER: interfaceAlias(1) LLDP-MIB::lldpRemPortId.9511080.4.5 = STRING: "25" LLDP-MIB::lldpRemPortId.9511149.2.6 = STRING: "25" LLDP-MIB::lldpRemPortId.9511174.3.7 = STRING: "25" LLDP-MIB::lldpRemPortId.9511209.26.8 = STRING: "25" LLDP-MIB::lldpRemPortId.9587238.48.9 = STRING: "50" LLDP-MIB::lldpRemPortId.9588774.27.10 = STRING: "26" LLDP-MIB::lldpRemPortId.9588907.28.11 = STRING: "26" LLDP-MIB::lldpRemPortId.9592381.42.12 = STRING: "Te1/1/2" LLDP-MIB::lldpRemPortId.9827133.8.14 = Hex-STRING: 4C 69 6E 6B 20 74 6F 20 43 4D 49 2C 20 6E 65 75 68 75 62 00 LLDP-MIB::lldpRemPortId.9843364.9.16 = Hex-STRING: 55 70 6C 69 6E 6B 20 74 6F 20 49 4A 53 2D 4E 45 54 00 LLDP-MIB::lldpRemPortDesc.9511080.4.5 = STRING: LLDP-MIB::lldpRemPortDesc.9511149.2.6 = STRING: LLDP-MIB::lldpRemPortDesc.9511174.3.7 = STRING: LLDP-MIB::lldpRemPortDesc.9511209.26.8 = STRING: LLDP-MIB::lldpRemPortDesc.9587238.48.9 = STRING: XGE2 LLDP-MIB::lldpRemPortDesc.9588774.27.10 = STRING: LLDP-MIB::lldpRemPortDesc.9588907.28.11 = STRING: LLDP-MIB::lldpRemPortDesc.9592381.42.12 = STRING: TenGigabitEthernet1/1/2 LLDP-MIB::lldpRemPortDesc.9827133.8.14 = STRING: Slot: 0 Port: 48 10G - Level LLDP-MIB::lldpRemPortDesc.9843364.9.16 = STRING: Slot: 0 Port: 48 10G - Level LLDP-MIB::lldpRemSysName.9511080.4.5 = STRING: <hidden1>.ijs.si LLDP-MIB::lldpRemSysName.9511149.2.6 = STRING: <hidden2>.ijs.si LLDP-MIB::lldpRemSysName.9511174.3.7 = STRING: <hidden3>.ijs.si LLDP-MIB::lldpRemSysName.9511209.26.8 = STRING: <hidden4>.ijs.si LLDP-MIB::lldpRemSysName.9587238.48.9 = STRING: <hidden5> LLDP-MIB::lldpRemSysName.9588774.27.10 = STRING: <hidden6> LLDP-MIB::lldpRemSysName.9588907.28.11 = STRING: <hidden7> LLDP-MIB::lldpRemSysName.9592381.42.12 = STRING: <hidden8> LLDP-MIB::lldpRemSysName.9827133.8.14 = STRING: <hidden9> LLDP-MIB::lldpRemSysName.9843364.9.16 = STRING: <hidden10> LLDP-MIB::lldpRemSysDesc.9511080.4.5 = STRING: ExtremeXOS (X450a-24t) ... LLDP-MIB::lldpRemSysDesc.9511149.2.6 = STRING: ExtremeXOS (X450a-24t) ... LLDP-MIB::lldpRemSysDesc.9511174.3.7 = STRING: ExtremeXOS (X450a-24t) ... LLDP-MIB::lldpRemSysDesc.9511209.26.8 = STRING: ExtremeXOS (X450a-24t) ... LLDP-MIB::lldpRemSysDesc.9587238.48.9 = STRING: Networking Operating System RackSwitch G7052, IBM Networking OS: version 7.6.5, Boot image: version 7.6.5 LLDP-MIB::lldpRemSysDesc.9588774.27.10 = STRING: ExtremeXOS (X450a-24t) ... LLDP-MIB::lldpRemSysDesc.9588907.28.11 = STRING: ExtremeXOS (X450a-24t) ... LLDP-MIB::lldpRemSysDesc.9592381.42.12 = STRING: Cisco IOS Software, Catalyst L3 Switch Software... LLDP-MIB::lldpRemSysDesc.9827133.8.14 = STRING: T3048-LY2R L2/L3 Switch LLDP-MIB::lldpRemSysDesc.9843364.9.16 = STRING: T3048-LY2R L2/L3 Switch [...]
In both cases 1 & 2 the lldpRemPortIdSubtype of 'interfaceAlias' and 'local' fell under the "default:" branch of a "switch ($lldp['lldpRemPortIdSubtype'])", instead of being treated just like a "case 'interfaceName':" branch. So my solution was to add "case 'interfaceAlias':" and "case 'local':" there;
switch ($lldp['lldpRemPortIdSubtype']) { case 'interfaceName': + case 'interfaceAlias': + case 'local':
A Case 3 is a bit nastier and is specific to our Summit Extreme X450a switches. The problem there is that the interface name (i.e. a port number) as reported in the LLDP MIB does not use the same naming/numbering convention as the IF-MIB and SQL: the LLDP MIB just uses a port number (1 .. 26), while the SQL field ports.ifName prefixes a port number with a chassis number, which is always a 1 in a non-stackable switch. Moreover the ifIndex uses a formula 1000*chassis + port number. On top of it the PortDesc in LLDP MIB is an empty string, so cannot be used to correlate with IF-MIB::ifDescr. To illustrate:
IF-MIB::ifIndex.1001 = INTEGER: 1001 IF-MIB::ifIndex.1002 = INTEGER: 1002 IF-MIB::ifIndex.1003 = INTEGER: 1003 [...] IF-MIB::ifIndex.1026 = INTEGER: 1026
IF-MIB::ifDescr.1001 = STRING: X450a-24t Port 1 IF-MIB::ifDescr.1002 = STRING: X450a-24t Port 2 IF-MIB::ifDescr.1003 = STRING: X450a-24t Port 3 [...] IF-MIB::ifDescr.1026 = STRING: X450a-24t Port 26
LLDP-MIB::lldpLocPortIdSubtype.1 = INTEGER: interfaceName(5) LLDP-MIB::lldpLocPortIdSubtype.2 = INTEGER: interfaceName(5) LLDP-MIB::lldpLocPortIdSubtype.3 = INTEGER: interfaceName(5) [...]
LLDP-MIB::lldpLocPortId.1 = STRING: "1" LLDP-MIB::lldpLocPortId.2 = STRING: "2" LLDP-MIB::lldpLocPortId.3 = STRING: "3" [...] LLDP-MIB::lldpLocPortId.26 = STRING: "26"
LLDP-MIB::lldpLocPortDesc.1 = STRING: LLDP-MIB::lldpLocPortDesc.2 = STRING: LLDP-MIB::lldpLocPortDesc.3 = STRING: [...] LLDP-MIB::lldpLocPortDesc.26 = STRING:
The corresponding SQL field ports.ifName looks like "1:26", i.e. the port number is prefixed by a chassis number and a colon.
So neither the SQL query on ifDescr ("") OR ifName ("26") can succeed, nor the query on ifIndex ("1026"). I ended up with a hackish workaround kludge in lldp-mib.inc.php :
case 'interfaceName': $remote_port_id = dbFetchCell("SELECT `port_id` FROM `ports` WHERE (`ifDescr`= ? OR `ifName`= ?) AND `device_id` = ?",array($if, $id, $remote_device_id)); + if (!$remote_port_id && is_numeric($id) && $id < 1000) { + $remote_port_id = dbFetchCell("SELECT `port_id` FROM `ports` WHERE (`ifIndex`= ? OR `ifName`= ?) AND `device_id` = ?", array(1000+$id, "1:$id", $remote_device_id)); + }
Actually there is also a Case 4, where lldpRemSysName shows a string "(none).(none)" instead of an empty string, when the (optional) system name is not configured. To fix:
- if (empty($lldp['lldpRemSysName']) && $lldp['lldpRemChassisIdSubtype'] == 'macAddress') + if ((empty($lldp['lldpRemSysName']) || $lldp['lldpRemSysName'] == '(none).(none)') && $lldp['lldpRemChassisIdSubtype'] == 'macAddress')
The affected devices were: Summit Extreme X450 switches (Case #3), a Quanta LY2R switch (the 'interfaceAlias' Case #2), and a Networking Operating System RackSwitch G7052 (the 'local', Case #1). Not sure what switch was connected there in Case 4, possibly a smaller Quanta, identifies as "armv5tejl/Linux 2.6.17.WB_WPCM450.1.3".
The full patch is attached (I hope it survives the mailing list). If more information is needed I'd be gladly providing.
Regards Mark
Hi,
try with latest r7130.
I'm added more correct detection of remote ports (accordind to MIB descriptions). But hardcoded search ifIndex as 1000+$id not approved, because it's single platform or your device specific.
Send to me output for debug discovery for local and remote devices (I will check ifIndexes and other ports params) ./discovery.php -d -h <local_device> ./discovery.php -d -h <remote_device>
On Tue, Oct 27, 2015 at 4:25 AM, Mark Martinec < Mark.Martinec+observium@ijs.si> wrote:
Spent some time trying to get Observium to deal with MIBs as produced by LLDP (Link Layer Discovery Protocol) on a couple of diverse network equipment types. Running the latest CE edition 0.15.6.6430.
After initial meager success (only two boxes recognized and several L2 neighbours reported with wrong interfaces), a couple of changes in includes/discovery/discovery-protocols/lldp-mib.inc.php brought some sense into results.
There are essentially two problems there:
some lldpRemoteSystemsData OIDs could not be correlated with an interface on the other side (for various reasons, see below);
a fallback mechanism which tries to find a match on an interface's MAC address produces a wrong match, because typically all ports on larger switches have the same MAC address and the heuristics just picks the first match, which is almost always wrong. In case of multiple matches the matching on MAC should just fail, which at least would not produce a senseless match causing operators to scratch their heads and wonder who crossed the cables. The comment in lldp-mib.inc.php : // Find the device by MAC address, still matches multiple ports sometimes, we use the first one and hope we're lucky if very much misguided, as all port MAC addresses on a switch are typically the same, at least on all device types that I checked.
The first issue above needs more explanation. I found a couple of different reasons for a failed match, as follows.
Case 1:
$ snmpbulkwalk -v2c -c ... -M mibs -m LLDP-MIB ... lldpRemoteSystemsData LLDP-MIB::lldpRemChassisIdSubtype.5360466.52.2 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisId.5360466.52.2 = Hex-STRING: C4 54 44 1E 71 4E LLDP-MIB::lldpRemPortIdSubtype.5360466.52.2 = INTEGER: local(7) LLDP-MIB::lldpRemPortId.5360466.52.2 = Hex-STRING: 30 2F 34 37 00 LLDP-MIB::lldpRemPortDesc.5360466.52.2 = STRING: "Slot: 0 Port: 47 10G - Level" LLDP-MIB::lldpRemSysName.5360466.52.2 = STRING: "<hidden>" LLDP-MIB::lldpRemSysDesc.5360466.52.2 = STRING: "T3048-LY2R L2/L3 Switch" LLDP-MIB::lldpRemSysCapSupported.5360466.52.2 = BITS: 28 00 bridge(2) router(4) LLDP-MIB::lldpRemSysCapEnabled.5360466.52.2 = BITS: 20 00 bridge(2) LLDP-MIB::lldpRemManAddrIfSubtype.5360466.52.2.6.6.196.84.68.30.113.78 = INTEGER: ifIndex(2) LLDP-MIB::lldpRemManAddrIfId.5360466.52.2.6.6.196.84.68.30.113.78 = INTEGER: 69 LLDP-MIB::lldpRemManAddrOID.5360466.52.2.6.6.196.84.68.30.113.78 = OID: iso.3.6.1.4.1.7244 LLDP-MIB::lldpRemOrgDefInfo.5360466.52.2.0.18.15.1.1 = Hex-STRING: 01 00 00 00 24 LLDP-MIB::lldpRemOrgDefInfo.5360466.52.2.0.18.15.2.1 = Hex-STRING: 00 00 00 LLDP-MIB::lldpRemOrgDefInfo.5360466.52.2.0.18.15.3.1 = Hex-STRING: 01 00 00 00 00 LLDP-MIB::lldpRemOrgDefInfo.5360466.52.2.0.18.15.4.1 = Hex-STRING: 23 F2 LLDP-MIB::lldpRemOrgDefInfo.5360466.52.2.0.128.194.1.1 = Hex-STRING: 00 01 LLDP-MIB::lldpRemOrgDefInfo.5360466.52.2.0.128.194.2.1 = Hex-STRING: 02 00 00 LLDP-MIB::lldpRemOrgDefInfo.5360466.52.2.0.128.194.3.1 = Hex-STRING: 00 01 00
Case 2: LLDP-MIB::lldpRemChassisIdSubtype.9511080.4.5 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9511149.2.6 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9511174.3.7 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9511209.26.8 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9587238.48.9 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9588774.27.10 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9588907.28.11 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9592381.42.12 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9827133.8.14 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisIdSubtype.9843364.9.16 = INTEGER: macAddress(4) LLDP-MIB::lldpRemChassisId.9511080.4.5 = Hex-STRING: 00 04 96 34 CB 7D LLDP-MIB::lldpRemChassisId.9511149.2.6 = Hex-STRING: 00 04 96 34 CB 78 LLDP-MIB::lldpRemChassisId.9511174.3.7 = Hex-STRING: 00 04 96 34 CB 8D LLDP-MIB::lldpRemChassisId.9511209.26.8 = Hex-STRING: 00 04 96 34 CB 93 LLDP-MIB::lldpRemChassisId.9587238.48.9 = Hex-STRING: A4 8C DB 0C 4B 00 LLDP-MIB::lldpRemChassisId.9588774.27.10 = Hex-STRING: 00 04 96 34 CB 7F LLDP-MIB::lldpRemChassisId.9588907.28.11 = Hex-STRING: 00 04 96 34 CB 8B LLDP-MIB::lldpRemChassisId.9592381.42.12 = Hex-STRING: 44 AD D9 6C 2D 80 LLDP-MIB::lldpRemChassisId.9827133.8.14 = Hex-STRING: C4 54 44 1E 71 4E LLDP-MIB::lldpRemChassisId.9843364.9.16 = Hex-STRING: 2C 60 0C 33 8F FF LLDP-MIB::lldpRemPortIdSubtype.9511080.4.5 = INTEGER: interfaceName(5) LLDP-MIB::lldpRemPortIdSubtype.9511149.2.6 = INTEGER: interfaceName(5) LLDP-MIB::lldpRemPortIdSubtype.9511174.3.7 = INTEGER: interfaceName(5) LLDP-MIB::lldpRemPortIdSubtype.9511209.26.8 = INTEGER: interfaceName(5) LLDP-MIB::lldpRemPortIdSubtype.9587238.48.9 = INTEGER: local(7) LLDP-MIB::lldpRemPortIdSubtype.9588774.27.10 = INTEGER: interfaceName(5) LLDP-MIB::lldpRemPortIdSubtype.9588907.28.11 = INTEGER: interfaceName(5) LLDP-MIB::lldpRemPortIdSubtype.9592381.42.12 = INTEGER: interfaceName(5) LLDP-MIB::lldpRemPortIdSubtype.9827133.8.14 = INTEGER: interfaceAlias(1) LLDP-MIB::lldpRemPortIdSubtype.9843364.9.16 = INTEGER: interfaceAlias(1) LLDP-MIB::lldpRemPortId.9511080.4.5 = STRING: "25" LLDP-MIB::lldpRemPortId.9511149.2.6 = STRING: "25" LLDP-MIB::lldpRemPortId.9511174.3.7 = STRING: "25" LLDP-MIB::lldpRemPortId.9511209.26.8 = STRING: "25" LLDP-MIB::lldpRemPortId.9587238.48.9 = STRING: "50" LLDP-MIB::lldpRemPortId.9588774.27.10 = STRING: "26" LLDP-MIB::lldpRemPortId.9588907.28.11 = STRING: "26" LLDP-MIB::lldpRemPortId.9592381.42.12 = STRING: "Te1/1/2" LLDP-MIB::lldpRemPortId.9827133.8.14 = Hex-STRING: 4C 69 6E 6B 20 74 6F 20 43 4D 49 2C 20 6E 65 75 68 75 62 00 LLDP-MIB::lldpRemPortId.9843364.9.16 = Hex-STRING: 55 70 6C 69 6E 6B 20 74 6F 20 49 4A 53 2D 4E 45 54 00 LLDP-MIB::lldpRemPortDesc.9511080.4.5 = STRING: LLDP-MIB::lldpRemPortDesc.9511149.2.6 = STRING: LLDP-MIB::lldpRemPortDesc.9511174.3.7 = STRING: LLDP-MIB::lldpRemPortDesc.9511209.26.8 = STRING: LLDP-MIB::lldpRemPortDesc.9587238.48.9 = STRING: XGE2 LLDP-MIB::lldpRemPortDesc.9588774.27.10 = STRING: LLDP-MIB::lldpRemPortDesc.9588907.28.11 = STRING: LLDP-MIB::lldpRemPortDesc.9592381.42.12 = STRING: TenGigabitEthernet1/1/2 LLDP-MIB::lldpRemPortDesc.9827133.8.14 = STRING: Slot: 0 Port: 48 10G - Level LLDP-MIB::lldpRemPortDesc.9843364.9.16 = STRING: Slot: 0 Port: 48 10G - Level LLDP-MIB::lldpRemSysName.9511080.4.5 = STRING: <hidden1>.ijs.si LLDP-MIB::lldpRemSysName.9511149.2.6 = STRING: <hidden2>.ijs.si LLDP-MIB::lldpRemSysName.9511174.3.7 = STRING: <hidden3>.ijs.si LLDP-MIB::lldpRemSysName.9511209.26.8 = STRING: <hidden4>.ijs.si LLDP-MIB::lldpRemSysName.9587238.48.9 = STRING: <hidden5> LLDP-MIB::lldpRemSysName.9588774.27.10 = STRING: <hidden6> LLDP-MIB::lldpRemSysName.9588907.28.11 = STRING: <hidden7> LLDP-MIB::lldpRemSysName.9592381.42.12 = STRING: <hidden8> LLDP-MIB::lldpRemSysName.9827133.8.14 = STRING: <hidden9> LLDP-MIB::lldpRemSysName.9843364.9.16 = STRING: <hidden10> LLDP-MIB::lldpRemSysDesc.9511080.4.5 = STRING: ExtremeXOS (X450a-24t) ... LLDP-MIB::lldpRemSysDesc.9511149.2.6 = STRING: ExtremeXOS (X450a-24t) ... LLDP-MIB::lldpRemSysDesc.9511174.3.7 = STRING: ExtremeXOS (X450a-24t) ... LLDP-MIB::lldpRemSysDesc.9511209.26.8 = STRING: ExtremeXOS (X450a-24t) ... LLDP-MIB::lldpRemSysDesc.9587238.48.9 = STRING: Networking Operating System RackSwitch G7052, IBM Networking OS: version 7.6.5, Boot image: version 7.6.5 LLDP-MIB::lldpRemSysDesc.9588774.27.10 = STRING: ExtremeXOS (X450a-24t) ... LLDP-MIB::lldpRemSysDesc.9588907.28.11 = STRING: ExtremeXOS (X450a-24t) ... LLDP-MIB::lldpRemSysDesc.9592381.42.12 = STRING: Cisco IOS Software, Catalyst L3 Switch Software... LLDP-MIB::lldpRemSysDesc.9827133.8.14 = STRING: T3048-LY2R L2/L3 Switch LLDP-MIB::lldpRemSysDesc.9843364.9.16 = STRING: T3048-LY2R L2/L3 Switch [...]
In both cases 1 & 2 the lldpRemPortIdSubtype of 'interfaceAlias' and 'local' fell under the "default:" branch of a "switch ($lldp['lldpRemPortIdSubtype'])", instead of being treated just like a "case 'interfaceName':" branch. So my solution was to add "case 'interfaceAlias':" and "case 'local':" there;
switch ($lldp['lldpRemPortIdSubtype']) { case 'interfaceName':
case 'interfaceAlias':case 'local':A Case 3 is a bit nastier and is specific to our Summit Extreme X450a switches. The problem there is that the interface name (i.e. a port number) as reported in the LLDP MIB does not use the same naming/numbering convention as the IF-MIB and SQL: the LLDP MIB just uses a port number (1 .. 26), while the SQL field ports.ifName prefixes a port number with a chassis number, which is always a 1 in a non-stackable switch. Moreover the ifIndex uses a formula 1000*chassis + port number. On top of it the PortDesc in LLDP MIB is an empty string, so cannot be used to correlate with IF-MIB::ifDescr. To illustrate:
IF-MIB::ifIndex.1001 = INTEGER: 1001 IF-MIB::ifIndex.1002 = INTEGER: 1002 IF-MIB::ifIndex.1003 = INTEGER: 1003 [...] IF-MIB::ifIndex.1026 = INTEGER: 1026
IF-MIB::ifDescr.1001 = STRING: X450a-24t Port 1 IF-MIB::ifDescr.1002 = STRING: X450a-24t Port 2 IF-MIB::ifDescr.1003 = STRING: X450a-24t Port 3 [...] IF-MIB::ifDescr.1026 = STRING: X450a-24t Port 26
LLDP-MIB::lldpLocPortIdSubtype.1 = INTEGER: interfaceName(5) LLDP-MIB::lldpLocPortIdSubtype.2 = INTEGER: interfaceName(5) LLDP-MIB::lldpLocPortIdSubtype.3 = INTEGER: interfaceName(5) [...]
LLDP-MIB::lldpLocPortId.1 = STRING: "1" LLDP-MIB::lldpLocPortId.2 = STRING: "2" LLDP-MIB::lldpLocPortId.3 = STRING: "3" [...] LLDP-MIB::lldpLocPortId.26 = STRING: "26"
LLDP-MIB::lldpLocPortDesc.1 = STRING: LLDP-MIB::lldpLocPortDesc.2 = STRING: LLDP-MIB::lldpLocPortDesc.3 = STRING: [...] LLDP-MIB::lldpLocPortDesc.26 = STRING:
The corresponding SQL field ports.ifName looks like "1:26", i.e. the port number is prefixed by a chassis number and a colon.
So neither the SQL query on ifDescr ("") OR ifName ("26") can succeed, nor the query on ifIndex ("1026"). I ended up with a hackish workaround kludge in lldp-mib.inc.php :
case 'interfaceName': $remote_port_id = dbFetchCell("SELECT `port_id` FROM`ports` WHERE (`ifDescr`= ? OR `ifName`= ?) AND `device_id` = ?",array($if, $id, $remote_device_id));
if (!$remote_port_id && is_numeric($id) && $id < 1000) {$remote_port_id = dbFetchCell("SELECT `port_id` FROM`ports` WHERE (`ifIndex`= ? OR `ifName`= ?) AND `device_id` = ?", array(1000+$id, "1:$id", $remote_device_id));
}Actually there is also a Case 4, where lldpRemSysName shows a string "(none).(none)" instead of an empty string, when the (optional) system name is not configured. To fix:
if (empty($lldp['lldpRemSysName']) &&$lldp['lldpRemChassisIdSubtype'] == 'macAddress')
if ((empty($lldp['lldpRemSysName']) || $lldp['lldpRemSysName']== '(none).(none)') && $lldp['lldpRemChassisIdSubtype'] == 'macAddress')
The affected devices were: Summit Extreme X450 switches (Case #3), a Quanta LY2R switch (the 'interfaceAlias' Case #2), and a Networking Operating System RackSwitch G7052 (the 'local', Case #1). Not sure what switch was connected there in Case 4, possibly a smaller Quanta, identifies as "armv5tejl/Linux 2.6.17.WB_WPCM450.1.3".
The full patch is attached (I hope it survives the mailing list). If more information is needed I'd be gladly providing.
Regards Mark
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
2015-10-27 09:15, Mike Stupalov wrote:
try with latest r7130.
Hmm, where can I get hold of it? I'm on a CE edition.
I'm added more correct detection of remote ports (accordind to MIB descriptions).
Nice, thanks.
But hardcoded search ifIndex as 1000+$id not approved, because it's single platform or your device specific.
Sure, it was just an ad-hoc fix for our particular case.
A more general solution for Extreme switches is needed to take into account a chassis number. Unfortunately none of our switches is stackable, so I can only provide information for a standalone unit.
Send to me output for debug discovery for local and remote devices (I will check ifIndexes and other ports params) ./discovery.php -d -h <local_device> ./discovery.php -d -h <remote_device>
Will send it privately, There's lots of stuff there and a couple of different cases with different local/remote pairs. Need to figure out some way for you to make sense of it.
Tom Laermans wrote:
- a fallback mechanism which tries to find a match on an interface's MAC address produces a wrong match, because typically all ports on larger switches have the same MAC address and the heuristics just picks the first match, which is almost always wrong. In case of multiple matches the matching on MAC should just fail, which at least would not produce a senseless match causing operators to scratch their heads and wonder who crossed the cables. The comment in lldp-mib.inc.php : // Find the device by MAC address, still matches multiple ports sometimes, we use the first one and hope we're lucky if very much misguided, as all port MAC addresses on a switch are typically the same, at least on all device types that I checked.
This worked for quite a number of devices I've checked, which all had different MACs per port. Indeed I have one that sees at least one other port as "port 1", and was planning on looking into that, pretty convinced it had to do with this too. I'll see later if this fixes it :-)
It works when MAC addresses are distinct, it fails misleadingly when they are the same. Found another case, this one a mismatch on a *local* port in a 'neighbours' report.
It should not be difficult to see if the SQL SELECT returned a single or a number of matches on a MAC address. When more than one port matches a query the result should be discarded. It will still work reliably for devices where each port has a distinct MAC address.
Mark
Hi Mark,
This worked for quite a number of devices I've checked, which all had different MACs per port. Indeed I have one that sees at least one other port as "port 1", and was planning on looking into that, pretty convinced it had to do with this too. I'll see later if this fixes it :-)
It works when MAC addresses are distinct, it fails misleadingly when they are the same. Found another case, this one a mismatch on a *local* port in a 'neighbours' report.
It should not be difficult to see if the SQL SELECT returned a single or a number of matches on a MAC address. When more than one port matches a query the result should be discarded. It will still work reliably for devices where each port has a distinct MAC address.
Yup, that is a good idea indeed.
Also the fix didn't solve my problem as I remembered it wrongly - the switch side is ok but the linux box's interface is shown wrong - the switch however sees the correct one. Will need to debug this further.
Tom
Mark,
Thanks for looking into this.
On 27/10/2015 02:25, Mark Martinec wrote:
- a fallback mechanism which tries to find a match on an interface's MAC address produces a wrong match, because typically all ports on larger switches have the same MAC address and the heuristics just picks the first match, which is almost always wrong. In case of multiple matches the matching on MAC should just fail, which at least would not produce a senseless match causing operators to scratch their heads and wonder who crossed the cables. The comment in lldp-mib.inc.php : // Find the device by MAC address, still matches multiple ports sometimes, we use the first one and hope we're lucky if very much misguided, as all port MAC addresses on a switch are typically the same, at least on all device types that I checked.
This worked for quite a number of devices I've checked, which all had different MACs per port. Indeed I have one that sees at least one other port as "port 1", and was planning on looking into that, pretty convinced it had to do with this too. I'll see later if this fixes it :-)
Actually there is also a Case 4, where lldpRemSysName shows a string "(none).(none)" instead of an empty string, when the (optional) system name is not configured. To fix:
if (empty($lldp['lldpRemSysName']) &&$lldp['lldpRemChassisIdSubtype'] == 'macAddress')
if ((empty($lldp['lldpRemSysName']) ||$lldp['lldpRemSysName'] == '(none).(none)') && $lldp['lldpRemChassisIdSubtype'] == 'macAddress')
I've seen (none).(none) as well, though I don't understand why that happens - I have 2 linux boxes both on the same switch and one shows up as none, the other with its correct hostname, both are running identically configured lldpd.
Thanks! Tom
Mark,
On 2015-10-27 09:24, Tom Laermans wrote:
- a fallback mechanism which tries to find a match on an interface's MAC address produces a wrong match, because typically all ports on larger switches have the same MAC address and the heuristics just picks the first match, which is almost always wrong. In case of multiple matches the matching on MAC should just fail, which at least would not produce a senseless match causing operators to scratch their heads and wonder who crossed the cables. The comment in lldp-mib.inc.php : // Find the device by MAC address, still matches multiple ports sometimes, we use the first one and hope we're lucky if very much misguided, as all port MAC addresses on a switch are typically the same, at least on all device types that I checked.
This worked for quite a number of devices I've checked, which all had different MACs per port. Indeed I have one that sees at least one other port as "port 1", and was planning on looking into that, pretty convinced it had to do with this too. I'll see later if this fixes it :-)
My mismatch actually DID have to do with that multiple hits. LACP bonding sets bonded NICs to all have the same MAC. I've committed a fix in r7138 to not link wrongly anymore when multiple ports have the same MAC.
Thanks for the pointer!
Tom
participants (3)
-
Mark Martinec -
Mike Stupalov -
Tom Laermans