You see, for my setup, about 150 devices, default poller makes 9393 calls to snmp*.
That means about 30 forks per second for snmp calls, and about the same number of rrdtool forks. Running lots of forks for short-lived programs is a sure way to bring system to its knees, and the more processors you have, the heavier is fork load. Running many pollers in parallel does nothing to ease this burden, for we are to make over 9000 forks every 5 minutes.
Anyway, I'm not into proving you that one language is better than annother, or perusade you to do anything. If someone need quick and light poller, one can git clone it. If someone is interested in collaboration, I'm open. If someone is content with python wrapper around php wrapper of snmp* and rrdtool, than it's just fine.
On 24.07.2014 20:46, Adam Armstrong wrote:
Not to mention that everything he isn't rewriting is covered by QPL and is not distributable outside of Observium. :)
Historically the least successful contributions have been made by people trying to do this sort of thing. They often make a lot of noise and then completely disappear. If they don't, they get commit. You can see how many times that's happened in the commit log.
I honestly don't see the point here. Nothing can speed up the control planes of the devices involved. We already make efficient use of snmpbulkwalk and perform relatively few queries. The poller can run as parallel as you have I/O for. No amount of rewriting is going to solve the primary scalability issue we have, which is raw rrd writes.
Much equal or greater benefit could be attained by rewriting and/or improving existing code, but of course, some people just want their own sand pit.
So on balance, this is just an exercise in reducing features, reducing maintainability and reducing reliability because e-penis.
Especially so as we'd heard nothing about it at all until a random mailing list post!
adam.
On 2014-07-24 16:01, Tom Laermans wrote:
Very nice email.
I don't think #3 is an actual goal for Alex. Problem is that makes #4 and #5 all the more problematic, even for people who agree with #6.
On 07/24/2014 04:25 PM, Moore, Cameron wrote: Alex, Part of what makes OSS so awesome is that people can create these little side projects to test out new ways of doing things. It's also what makes OSS frustrating. I applaud your hubris in one sense, but things may not go well for this side project. Here's why:
- It's written in Perl. I love Perl for many things, but the
Observium guys generally hate it (or that's my take from IRC conversations). 2) The job of the poller is complex. 3) Given #1 and #2, it will probably never be merged upstream. They won't want to maintain it. 4) Given #3, you will have to track upstream changes to the database and RRD structures, which tend to happen without notice. 5) Given #4, users will have to be very careful to match Observium versions with matching Perl poller versions. 6) You only care about Cisco. 7) Resolving #6 makes #2, 3 & 4 worse.
I'm not meaning to put a dark cloud over your project. I'm just pointing out the one that's already there.
Maybe you can prove us all wrong with a whizbang Perl poller that really is 50x faster and does all the things the current poller does while making it maintainable by guys who hate Perl. Go prove us all wrong, but know you have a very steep hill to climb if you want to get past #3. -- Cameron Moore
-----Original Message----- From: observium [mailto:observium-bounces@observium.org] On Behalf Of Alex Povolotsky Sent: Thursday, July 24, 2014 1:48 AM To: observium@observium.org Subject: Re: [Observium] Better poller for Observium
Old poller runs hundreds of forks. It is efficient as a dead cow. And database really lacks foreign keys (I did not check indexes yet. I'm afraid there will be lots of problems)
On 24.07.2014 2:36, Adam Armstrong wrote: How totally pointless.
adam.
On 2014-07-23 20:55, Alex Povolotsky wrote: Hello
While Observium has a nice interface, it's database lacks foreign keys a lot, and poller.php is AWFUL. It's slow and unreadable.
I've started a new, about 40-50 times lighter observium poller
https://github.com/tarkhil/observium-poller
It supports only SNMP for now, and is work-in-progress. Only os, processor and memory modules are working, and ports is about 50% finished.
If anyone is interested, please join. I'm sure to complete SNMP support for Cisco, but quite unsure if I'll be interested in implementing much else.
Alex
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium