Hey,Got it. Thanks. I think this will work. This box uses ‘other’ ifType quite a bit, and it can be filtered out. Doing so dropped the port count on this device from over 10,000 to just over 1,500.With regards to snmpbulkwalk vs snmpwalk - I’m a little unclear on where the filter is used and when it isn’t; trying to understand how this all works a bit better. So a couple questions, if you don’t mind -- Are you saying that the filter is only used during walk and not during bulkwalk?- The contextual help in the GUI when populating these bad_if* fields suggests that these are considered during discovery, not polling. So 1. Every time discovery runs, it creates a list of ‘valid’ interfaces based on the results of snmp(bulk?)walk, then removing anything in bad_if* 2. Every polling interval it walks only that list for a device if the device has separate_walk enabled, and if the device does not have separate_walk enabled, the poller will just bulkwalk the entire table and stick everything into the database, create graphs for it all, etc?Also, FWIW, manually doing a walk on IF only takes about 10s longer to complete than bulkwalk on just over 48,000 entries in the table for this particular device; this is the most dense device. That’s interesting to me in that I thought a bulkwalk would take much, much less time, or is the time dependant on a bunch of other stuff too?Thanks in advance!_______________________________________________On Oct 23, 2017, at 2:23 AM, Markus Klock <markus@best-practice.se> wrote:Hey,The default way of SNMP-polling ports is by doing a snmpbulkwalk of the whole table of interfaces on a device as this is usually faster.But in some cases the table of interfaces is huge, and you are really only interested in a smaller part of the interfaces, then the separate_walk-feature is usefull.Start by filtering out the interfaces you do not want by using interface filters: http://docs.observium.org/config_options/#filters_1Then when the separate_walk-feature is enabled for the device, Observium will only do single snmpgets of the interfaces not filtered out instead of bulkwalking the whole interface table.You enable the feature by device->properties->modules->separate_walk=enabled/Markus_______________________________________________2017-10-23 3:05 GMT+02:00 Jason Lixfeld <jason@lixfeld.ca>:Hey Adam,To circle back around to this - I’m not sure what you mean by separate walk ports poller method. Is this concept on the site somewhere that I can read up on?Thanks!On Oct 2, 2017, at 3:12 PM, Jason Lixfeld <jason@lixfeld.ca> wrote:_______________________________________________We’d be relying on the transparent bridge interfaces at that point. While that might work today, each customer occupies a > 1 number of transparent bridge interfaces, depending on the number of services they have subscribed to. At a certain point, the number of transparent bridge ports would eclipse the number of GPON ports.On Oct 2, 2017, at 3:06 PM, Adam Armstrong <adama@memetic.org> wrote:_______________________________________________If you can deal with not having the pon interfaces, I'd filter them out somehow and switch to the separate walk ports poller method.Adam.Sent from BlueMailOn 2 Oct 2017, at 20:05, Jason Lixfeld <jason@lixfeld.ca> wrote:It would be each active customer’s port, and the device uplink(s). So pretty much a constantly moving target :/On Oct 2, 2017, at 2:54 PM, Markus Klock <markus@best-practice.se> wrote:Do you want to graph the traffic of all the 10 000 ports? You could filter out only the ports you are interested in and have Observium only poll those. Assuming the ports you are interested in are maybe uplinks and stuff, like <50 ports._______________________________________________Den 2 okt. 2017 20:51 skrev "Jason Lixfeld" <jason@lixfeld.ca>:Mostly GPON ports (basically Ethernet, but I don’t know the ifType off the top of my head). Some transparent bridges too. Now to be fair, not all 10,000 ports are up, but I don’t recall if there’s a way to only poll ports that are up. I suppose though that you need to first poll all ports to determine status, then poll again only the ones you want to really poll. Cart, Horse. :)On Oct 2, 2017, at 2:09 PM, Adam Armstrong <adama@memetic.org> wrote:_______________________________________________10k ports is going to take a long time on a device without a blistering fast control plane.What are the ports, mostly?Adam.Sent from BlueMailOn 2 Oct 2017, at 19:05, Jason Lixfeld <jason@lixfeld.ca> wrote:Hey all,
I’ve got a device (Zhone MXK-F) with over 10,000 ports on it, and Observium is taking well over 5 minutes to poll it (upwards of 15 minutes, actually). While there isn’t a definition specifically for MXK-F, it’s getting detected as MXK. os.inc.php is setting max-rep 50, so it’s at least trying to bulk walk, but I don’t know if it’s actually working. Overriding to max-rep 100 doesn’t have an affect on the polling time.
So, I guess my question is whether or not I’m on the right track in assuming that this issue is likely that of bulk walk on this device, or possibly some other issue, and how would I determine one way or the other? If it’s something that I can pinpoint, I can throw it back to the vendor to fix.
From the digging I’ve done so far, all other poller processes called from poller-wrapper complete in under 5 minutes, so I’m fairly certain that this issue is one of the device, not Observium. But, to be thorough, I moved my database and RRD storage over to local SSDs, and also enabled rrdcached - all that to ensure I didn’t have any I/O issues.
/pollerlog/view=devices indicates that it’s just the one device that has high timing (although other devices of the same type are spots 2-7 on the top 10 list high timers, but none is over 300 seconds except for this guy).
root@monitor:/home/jlixfeld# /opt/observium/poller.php -VV
Observium 17.9.8837
##### Software versions #####
o OS Linux 3.16.0-4-amd64 [amd64] (Debian 8 (jessie))
o Apache 2.4.10
o PHP 5.6.30-0+deb8u1 (OPcache: DISABLED)
o Python 2.7.9
o MySQL 5.5.55-0+deb8u1 (extension: mysqli 5.5.55)
o SNMP NET-SNMP 5.7.2.1
o RRDtool 1.4.8 (rrdcached 1.4.8: unix:/opt/observium/rrdcached.sock)
##### Memory Limit #####
o PHP Unlimited
##### MySQL mode #####
o MySQL
##### Charset info #####
o PHP UTF-8
o MySQL utf8
##### Timezones info #####
o Date Monday, 02-Oct-17 12:55:15 EDT
o PHP -04:00
o MySQL -04:00
root@monitor:/home/jlixfeld#
Thanks all!
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium