I was just about to suggest the same thing :) A galera cluster would solve it for you. But managing a multi master DB cluster spread over the globe is hard, you might create more problems than you solve. :)
/Markus
Den tis 6 juli 2021 kl 22:18 skrev Michael via observium < observium@observium.org>:
You could always try separating off the mariadb onto a galera cluster. Have mutliple nodes/replicas distributed around the place and point each poller to the local/nearest one.
It's unlikely that you'd get much in the way of contention as each poller is updating it's own device table entries. They will then be replicated back to the other db nodes.
Michael
On 7 Jul 2021, at 4:04 am, Milton Ngan via observium < observium@observium.org> wrote:
Thanks. I think we are ok maintaining the centralized polling, but we have a few devices that are right on the edge of the 5min polling period. For now, we can just trim back the modules polled on a few of those to give us some head room.
On Jul 6, 2021, at 10:26 AM, Adam Armstrong adama@observium.org wrote:
There's not a lot of scope for wholesale changing how the pollers work.
We can probably optimise individual poller sections, but there is big risk in overcomplicating things and decoupling how things work for what would probably just be small gains.
adam.
On Jul 6 2021, at 6:06 pm, Milton Ngan via observium < observium@observium.org> wrote:
I was looking at the distributed polling feature as we have a geographically distributed network and polling slow devices from half way across the globe takes a long time. It turns out that putting a poller closer to the device doesn’t actually help. The SNMP polling times go down a bit, but the SQL times go up a lot because there are hundreds of SQL queries that get run for every polled device. This can easily add another 1-2 minutes to your polls depending on how many metrics are being polled. So, it seems to me that the distributed feature only works if the poller is still reasonably close to the database, otherwise the latency per query eats up any gains you might have from reducing the SNMP latency.
Are there plans to try to reduce the number of queries a poller makes to make geographically distributed polling less latency sensitive?
Cheers
Milton
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium