Hey Adam,
To circle back around to this - I’m not sure what you mean by separate walk ports poller method. Is this concept on the site somewhere that I can read up on?
Thanks!
On Oct 2, 2017, at 3:12 PM, Jason Lixfeld jason@lixfeld.ca wrote:
We’d be relying on the transparent bridge interfaces at that point. While that might work today, each customer occupies a > 1 number of transparent bridge interfaces, depending on the number of services they have subscribed to. At a certain point, the number of transparent bridge ports would eclipse the number of GPON ports.
On Oct 2, 2017, at 3:06 PM, Adam Armstrong <adama@memetic.org mailto:adama@memetic.org> wrote:
If you can deal with not having the pon interfaces, I'd filter them out somehow and switch to the separate walk ports poller method.
Adam.
Sent from BlueMail http://www.bluemail.me/r?b=10777 On 2 Oct 2017, at 20:05, Jason Lixfeld <jason@lixfeld.ca mailto:jason@lixfeld.ca> wrote: It would be each active customer’s port, and the device uplink(s). So pretty much a constantly moving target :/
On Oct 2, 2017, at 2:54 PM, Markus Klock <markus@best-practice.se mailto:markus@best-practice.se> wrote:
Do you want to graph the traffic of all the 10 000 ports? You could filter out only the ports you are interested in and have Observium only poll those. Assuming the ports you are interested in are maybe uplinks and stuff, like <50 ports.
Den 2 okt. 2017 20:51 skrev "Jason Lixfeld" <jason@lixfeld.ca mailto:jason@lixfeld.ca>: Mostly GPON ports (basically Ethernet, but I don’t know the ifType off the top of my head). Some transparent bridges too. Now to be fair, not all 10,000 ports are up, but I don’t recall if there’s a way to only poll ports that are up. I suppose though that you need to first poll all ports to determine status, then poll again only the ones you want to really poll. Cart, Horse. :)
On Oct 2, 2017, at 2:09 PM, Adam Armstrong <adama@memetic.org mailto:adama@memetic.org> wrote:
10k ports is going to take a long time on a device without a blistering fast control plane.
What are the ports, mostly?
Adam.
Sent from BlueMail http://www.bluemail.me/r?b=10777 On 2 Oct 2017, at 19:05, Jason Lixfeld <jason@lixfeld.ca mailto:jason@lixfeld.ca> wrote: Hey all,
I’ve got a device (Zhone MXK-F) with over 10,000 ports on it, and Observium is taking well over 5 minutes to poll it (upwards of 15 minutes, actually). While there isn’t a definition specifically for MXK-F, it’s getting detected as MXK. os.inc.php is setting max-rep 50, so it’s at least trying to bulk walk, but I don’t know if it’s actually working. Overriding to max-rep 100 doesn’t have an affect on the polling time.
So, I guess my question is whether or not I’m on the right track in assuming that this issue is likely that of bulk walk on this device, or possibly some other issue, and how would I determine one way or the other? If it’s something that I can pinpoint, I can throw it back to the vendor to fix.
From the digging I’ve done so far, all other poller processes called from poller-wrapper complete in under 5 minutes, so I’m fairly certain that this issue is one of the device, not Observium. But, to be thorough, I moved my database and RRD storage over to local SSDs, and also enabled rrdcached - all that to ensure I didn’t have any I/O issues.
/pollerlog/view=devices indicates that it’s just the one device that has high timing (although other devices of the same type are spots 2-7 on the top 10 list high timers, but none is over 300 seconds except for this guy).
root@monitor:/home/jlixfeld# /opt/observium/poller.php -VV Observium 17.9.8837
##### Software versions #####
o OS Linux 3.16.0-4-amd64 [amd64] (Debian 8 (jessie)) o Apache 2.4.10 o PHP 5.6.30-0+deb8u1 (OPcache: DISABLED) o Python 2.7.9 o MySQL 5.5.55-0+deb8u1 (extension: mysqli 5.5.55) o SNMP NET-SNMP 5.7.2.1 http://5.7.2.1/ o RRDtool 1.4.8 (rrdcached 1.4.8: unix:/opt/observium/rrdcached.sock)
##### Memory Limit #####
o PHP Unlimited
##### MySQL mode #####
o MySQL
##### Charset info #####
o PHP UTF-8 o MySQL utf8
##### Timezones info #####
o Date Monday, 02-Oct-17 12:55:15 EDT o PHP -04:00 o MySQL -04:00
root@monitor:/home/jlixfeld#
Thanks all!
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium