I know in a standard install with a single server the poller_id is 0 in the DB. For a distributed poller setup do we need to specify $config[poller_id] in the config.php file on each poller? And how do we specify the poller_id when devices are added to the system? Thanks
Ron Culler
If you add devices from the cli of the poller, it'll be automatically added to that poller, using the poller_id defined in that install's config.php (that's why the poller_id is in the config, rather than being a command line option)
It doesn't really make much sense to add it from the central system, since it (often) can't do tests and initial poll/discover.
Adding from the UI might come in the future, I have a plan for that.
adam. On 2019-04-04 21:05:55, Ron Culler via observium observium@observium.org wrote: I know in a standard install with a single server the poller_id is 0 in the DB. For a distributed poller setup do we need to specify $config[poller_id] in the config.php file on each poller? And how do we specify the poller_id when devices are added to the system? Thanks
Ron Culler
_______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hi,
what if a poller already has collected lots of local hosts and relevant data?
How could I transfer those data to the master (if possible)? Should they be merged with the master data, or do they have separate tables?
Thanks!
Luca
From: observium observium-bounces@observium.org On Behalf Of Adam Armstrong via observium Sent: Thursday, April 4, 2019 10:47 PM To: Ron Culler via observium observium@observium.org Cc: Adam Armstrong adama@memetic.org Subject: Re: [Observium] Distributed Poller
If you add devices from the cli of the poller, it'll be automatically added to that poller, using the poller_id defined in that install's config.php (that's why the poller_id is in the config, rather than being a command line option)
It doesn't really make much sense to add it from the central system, since it (often) can't do tests and initial poll/discover.
Adding from the UI might come in the future, I have a plan for that.
adam.
On 2019-04-04 21:05:55, Ron Culler via observium <observium@observium.org mailto:observium@observium.org > wrote:
I know in a standard install with a single server the poller_id is 0 in the DB. For a distributed poller setup do we need to specify $config[poller_id] in the config.php file on each poller? And how do we specify the poller_id when devices are added to the system? Thanks
Ron Culler
_______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
I’ve deployed 2 pollers and adjusted the cronjobs on them and see in the Observium.log that half of the devices are polled from one and the other half from the second poller. On the device’s ‘General device settings’ tab there is dropdown for assigning(?) a poller but it’s greyed out with default poller. What do I miss to get that enabled and assign devices to a specific poller instance or that they stick with the poller they were added? Haven’t added anything poller related to the cofig file (also didn’t find anything documented) and noticed there is a table named pollers in the db but it’s empty.
Thanks, Alex
From: observium observium-bounces@observium.org On Behalf Of Luca Sasdelli via observium Sent: Freitag, 05. April 2019 15:48 To: 'Observium' observium@observium.org Cc: Luca Sasdelli luca@sasdelli.it; 'Adam Armstrong' adama@memetic.org Subject: Re: [Observium] Distributed Poller
Hi,
what if a poller already has collected lots of local hosts and relevant data? How could I transfer those data to the master (if possible)? Should they be merged with the master data, or do they have separate tables?
Thanks! Luca
From: observium <observium-bounces@observium.orgmailto:observium-bounces@observium.org> On Behalf Of Adam Armstrong via observium Sent: Thursday, April 4, 2019 10:47 PM To: Ron Culler via observium <observium@observium.orgmailto:observium@observium.org> Cc: Adam Armstrong <adama@memetic.orgmailto:adama@memetic.org> Subject: Re: [Observium] Distributed Poller
If you add devices from the cli of the poller, it'll be automatically added to that poller, using the poller_id defined in that install's config.php (that's why the poller_id is in the config, rather than being a command line option)
It doesn't really make much sense to add it from the central system, since it (often) can't do tests and initial poll/discover.
Adding from the UI might come in the future, I have a plan for that.
adam.
On 2019-04-04 21:05:55, Ron Culler via observium <observium@observium.orgmailto:observium@observium.org> wrote: I know in a standard install with a single server the poller_id is 0 in the DB. For a distributed poller setup do we need to specify $config[poller_id] in the config.php file on each poller? And how do we specify the poller_id when devices are added to the system? Thanks
Ron Culler
_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hi Alexander,
If you currently have two pollers which are sharing the workload, this is what we call "distributed polling", which distributes the polling workload between several systems in the same location.
This is done using command line parameters on the poller cron jobs to tell the processes how many total pollers there are, and which instance that particular poller is. You seem to have worked that part out.
Assigning devices to specific pollers is part of another set of features, what we call "partitioned pollers". This is where a satellite poller instance only polls devices directly assigned to it (and usually added via its console), this is useful behind firewalls and the like.
To turn a poller into a "partitioned poller", you smply add a poller_name to its configuration. This will prevent it from polling any device which isn't assigned to it. Usually you wouldn't use special cron job options with this.
For example, in config.php :
$config['poller_name'] = "poller_name_here";
For semi-obvious reasons, this option can't be configured in the database :)
You can't fix a device to a specific instance of a "distributed poller", because distributed pollers are just to share device polling load.
I actually had a bit of trouble trying to pick names for these features that weren't confusing!
Thanks, adam. On 2019-04-08 16:55:56, Joelly, Alexander via observium observium@observium.org wrote: I’ve deployed 2 pollers and adjusted the cronjobs on them and see in the Observium.log that half of the devices are polled from one and the other half from the second poller. On the device’s ‘General device settings’ tab there is dropdown for assigning(?) a poller but it’s greyed out with default poller. What do I miss to get that enabled and assign devices to a specific poller instance or that they stick with the poller they were added? Haven’t added anything poller related to the cofig file (also didn’t find anything documented) and noticed there is a table named pollers in the db but it’s empty.
Thanks, Alex
From: observium observium-bounces@observium.org On Behalf Of Luca Sasdelli via observium Sent: Freitag, 05. April 2019 15:48 To: 'Observium' observium@observium.org Cc: Luca Sasdelli luca@sasdelli.it; 'Adam Armstrong' adama@memetic.org Subject: Re: [Observium] Distributed Poller Hi, what if a poller already has collected lots of local hosts and relevant data? How could I transfer those data to the master (if possible)? Should they be merged with the master data, or do they have separate tables? Thanks! Luca From: observium <observium-bounces@observium.org [mailto:observium-bounces@observium.org]> On Behalf Of Adam Armstrong via observium Sent: Thursday, April 4, 2019 10:47 PM To: Ron Culler via observium <observium@observium.org [mailto:observium@observium.org]> Cc: Adam Armstrong <adama@memetic.org [mailto:adama@memetic.org]> Subject: Re: [Observium] Distributed Poller If you add devices from the cli of the poller, it'll be automatically added to that poller, using the poller_id defined in that install's config.php (that's why the poller_id is in the config, rather than being a command line option) It doesn't really make much sense to add it from the central system, since it (often) can't do tests and initial poll/discover. Adding from the UI might come in the future, I have a plan for that. adam. On 2019-04-04 21:05:55, Ron Culler via observium <observium@observium.org [mailto:observium@observium.org]> wrote: I know in a standard install with a single server the poller_id is 0 in the DB. For a distributed poller setup do we need to specify $config[poller_id] in the config.php file on each poller? And how do we specify the poller_id when devices are added to the system? Thanks
Ron Culler
_______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
Partitioned polling is definitely awesome for our needs.
Do you think it will be possible to add “partitioned” devices from “master” poller, by stating the poller name together with SNMP data? This would prevent to access the partitioned pollers each time a modification is needed there.
Cheers
Luca
Da: observium observium-bounces@observium.org Per conto di Adam Armstrong via observium Inviato: lunedì 8 aprile 2019 20:42 A: Joelly, Alexander via observium observium@observium.org Cc: Adam Armstrong adama@memetic.org Oggetto: Re: [Observium] Distributed Poller
Hi Alexander,
If you currently have two pollers which are sharing the workload, this is what we call "distributed polling", which distributes the polling workload between several systems in the same location.
This is done using command line parameters on the poller cron jobs to tell the processes how many total pollers there are, and which instance that particular poller is. You seem to have worked that part out.
Assigning devices to specific pollers is part of another set of features, what we call "partitioned pollers". This is where a satellite poller instance only polls devices directly assigned to it (and usually added via its console), this is useful behind firewalls and the like.
To turn a poller into a "partitioned poller", you smply add a poller_name to its configuration. This will prevent it from polling any device which isn't assigned to it. Usually you wouldn't use special cron job options with this.
For example, in config.php :
$config['poller_name'] = "poller_name_here";
For semi-obvious reasons, this option can't be configured in the database :)
You can't fix a device to a specific instance of a "distributed poller", because distributed pollers are just to share device polling load.
I actually had a bit of trouble trying to pick names for these features that weren't confusing!
Thanks,
adam.
On 2019-04-08 16:55:56, Joelly, Alexander via observium <observium@observium.org mailto:observium@observium.org > wrote:
I’ve deployed 2 pollers and adjusted the cronjobs on them and see in the Observium.log that half of the devices are polled from one and the other half from the second poller. On the device’s ‘General device settings’ tab there is dropdown for assigning(?) a poller but it’s greyed out with default poller. What do I miss to get that enabled and assign devices to a specific poller instance or that they stick with the poller they were added? Haven’t added anything poller related to the cofig file (also didn’t find anything documented) and noticed there is a table named pollers in the db but it’s empty.
Thanks, Alex
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Luca Sasdelli via observium Sent: Freitag, 05. April 2019 15:48 To: 'Observium' <observium@observium.org mailto:observium@observium.org > Cc: Luca Sasdelli <luca@sasdelli.it mailto:luca@sasdelli.it >; 'Adam Armstrong' <adama@memetic.org mailto:adama@memetic.org > Subject: Re: [Observium] Distributed Poller
Hi,
what if a poller already has collected lots of local hosts and relevant data?
How could I transfer those data to the master (if possible)? Should they be merged with the master data, or do they have separate tables?
Thanks!
Luca
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Adam Armstrong via observium Sent: Thursday, April 4, 2019 10:47 PM To: Ron Culler via observium <observium@observium.org mailto:observium@observium.org > Cc: Adam Armstrong <adama@memetic.org mailto:adama@memetic.org > Subject: Re: [Observium] Distributed Poller
If you add devices from the cli of the poller, it'll be automatically added to that poller, using the poller_id defined in that install's config.php (that's why the poller_id is in the config, rather than being a command line option)
It doesn't really make much sense to add it from the central system, since it (often) can't do tests and initial poll/discover.
Adding from the UI might come in the future, I have a plan for that.
adam.
On 2019-04-04 21:05:55, Ron Culler via observium <observium@observium.org mailto:observium@observium.org > wrote:
I know in a standard install with a single server the poller_id is 0 in the DB. For a distributed poller setup do we need to specify $config[poller_id] in the config.php file on each poller? And how do we specify the poller_id when devices are added to the system? Thanks
Ron Culler
_______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
This is difficult because the central system has no ability to communicate with the satellite system, the communication only goes in the other direction.
It's possible we could add this in future via a few different methods, but at the moment you need to add directly from the partitioned poller's console.
adam. On 2019-04-08 22:20:25, Luca Sasdelli via observium observium@observium.org wrote: Partitioned polling is definitely awesome for our needs. Do you think it will be possible to add “partitioned” devices from “master” poller, by stating the poller name together with SNMP data? This would prevent to access the partitioned pollers each time a modification is needed there. Cheers Luca Da: observium observium-bounces@observium.org Per conto di Adam Armstrong via observium Inviato: lunedì 8 aprile 2019 20:42 A: Joelly, Alexander via observium observium@observium.org Cc: Adam Armstrong adama@memetic.org Oggetto: Re: [Observium] Distributed Poller Hi Alexander, If you currently have two pollers which are sharing the workload, this is what we call "distributed polling", which distributes the polling workload between several systems in the same location. This is done using command line parameters on the poller cron jobs to tell the processes how many total pollers there are, and which instance that particular poller is. You seem to have worked that part out. Assigning devices to specific pollers is part of another set of features, what we call "partitioned pollers". This is where a satellite poller instance only polls devices directly assigned to it (and usually added via its console), this is useful behind firewalls and the like. To turn a poller into a "partitioned poller", you smply add a poller_name to its configuration. This will prevent it from polling any device which isn't assigned to it. Usually you wouldn't use special cron job options with this. For example, in config.php : $config['poller_name'] = "poller_name_here"; For semi-obvious reasons, this option can't be configured in the database :) You can't fix a device to a specific instance of a "distributed poller", because distributed pollers are just to share device polling load. I actually had a bit of trouble trying to pick names for these features that weren't confusing! Thanks, adam. On 2019-04-08 16:55:56, Joelly, Alexander via observium <observium@observium.org [mailto:observium@observium.org]> wrote: I’ve deployed 2 pollers and adjusted the cronjobs on them and see in the Observium.log that half of the devices are polled from one and the other half from the second poller. On the device’s ‘General device settings’ tab there is dropdown for assigning(?) a poller but it’s greyed out with default poller. What do I miss to get that enabled and assign devices to a specific poller instance or that they stick with the poller they were added? Haven’t added anything poller related to the cofig file (also didn’t find anything documented) and noticed there is a table named pollers in the db but it’s empty.
Thanks, Alex
From: observium <observium-bounces@observium.org [mailto:observium-bounces@observium.org]> On Behalf Of Luca Sasdelli via observium Sent: Freitag, 05. April 2019 15:48 To: 'Observium' <observium@observium.org [mailto:observium@observium.org]> Cc: Luca Sasdelli <luca@sasdelli.it [mailto:luca@sasdelli.it]>; 'Adam Armstrong' <adama@memetic.org [mailto:adama@memetic.org]> Subject: Re: [Observium] Distributed Poller Hi, what if a poller already has collected lots of local hosts and relevant data? How could I transfer those data to the master (if possible)? Should they be merged with the master data, or do they have separate tables? Thanks! Luca From: observium <observium-bounces@observium.org [mailto:observium-bounces@observium.org]> On Behalf Of Adam Armstrong via observium Sent: Thursday, April 4, 2019 10:47 PM To: Ron Culler via observium <observium@observium.org [mailto:observium@observium.org]> Cc: Adam Armstrong <adama@memetic.org [mailto:adama@memetic.org]> Subject: Re: [Observium] Distributed Poller If you add devices from the cli of the poller, it'll be automatically added to that poller, using the poller_id defined in that install's config.php (that's why the poller_id is in the config, rather than being a command line option) It doesn't really make much sense to add it from the central system, since it (often) can't do tests and initial poll/discover. Adding from the UI might come in the future, I have a plan for that. adam. On 2019-04-04 21:05:55, Ron Culler via observium <observium@observium.org [mailto:observium@observium.org]> wrote: I know in a standard install with a single server the poller_id is 0 in the DB. For a distributed poller setup do we need to specify $config[poller_id] in the config.php file on each poller? And how do we specify the poller_id when devices are added to the system? Thanks
Ron Culler
_______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
Ok, thanks.
At present I see all partitioned poller device port graphs in error. By monitoring with tcpdump there is regular data flow from partitioned poller to master on port 42217. What else could I check?
Cheers
Luca
From: observium observium-bounces@observium.org On Behalf Of Adam Armstrong via observium Sent: Tuesday, April 9, 2019 6:35 AM To: Luca Sasdelli via observium observium@observium.org Cc: Adam Armstrong adama@memetic.org Subject: Re: [Observium] R: Distributed Poller
This is difficult because the central system has no ability to communicate with the satellite system, the communication only goes in the other direction.
It's possible we could add this in future via a few different methods, but at the moment you need to add directly from the partitioned poller's console.
adam.
On 2019-04-08 22:20:25, Luca Sasdelli via observium <observium@observium.org mailto:observium@observium.org > wrote:
Partitioned polling is definitely awesome for our needs.
Do you think it will be possible to add “partitioned” devices from “master” poller, by stating the poller name together with SNMP data? This would prevent to access the partitioned pollers each time a modification is needed there.
Cheers
Luca
Da: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > Per conto di Adam Armstrong via observium Inviato: lunedì 8 aprile 2019 20:42 A: Joelly, Alexander via observium <observium@observium.org mailto:observium@observium.org > Cc: Adam Armstrong <adama@memetic.org mailto:adama@memetic.org > Oggetto: Re: [Observium] Distributed Poller
Hi Alexander,
If you currently have two pollers which are sharing the workload, this is what we call "distributed polling", which distributes the polling workload between several systems in the same location.
This is done using command line parameters on the poller cron jobs to tell the processes how many total pollers there are, and which instance that particular poller is. You seem to have worked that part out.
Assigning devices to specific pollers is part of another set of features, what we call "partitioned pollers". This is where a satellite poller instance only polls devices directly assigned to it (and usually added via its console), this is useful behind firewalls and the like.
To turn a poller into a "partitioned poller", you smply add a poller_name to its configuration. This will prevent it from polling any device which isn't assigned to it. Usually you wouldn't use special cron job options with this.
For example, in config.php :
$config['poller_name'] = "poller_name_here";
For semi-obvious reasons, this option can't be configured in the database :)
You can't fix a device to a specific instance of a "distributed poller", because distributed pollers are just to share device polling load.
I actually had a bit of trouble trying to pick names for these features that weren't confusing!
Thanks,
adam.
On 2019-04-08 16:55:56, Joelly, Alexander via observium <observium@observium.org mailto:observium@observium.org > wrote:
I’ve deployed 2 pollers and adjusted the cronjobs on them and see in the Observium.log that half of the devices are polled from one and the other half from the second poller. On the device’s ‘General device settings’ tab there is dropdown for assigning(?) a poller but it’s greyed out with default poller. What do I miss to get that enabled and assign devices to a specific poller instance or that they stick with the poller they were added? Haven’t added anything poller related to the cofig file (also didn’t find anything documented) and noticed there is a table named pollers in the db but it’s empty.
Thanks, Alex
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Luca Sasdelli via observium Sent: Freitag, 05. April 2019 15:48 To: 'Observium' <observium@observium.org mailto:observium@observium.org > Cc: Luca Sasdelli <luca@sasdelli.it mailto:luca@sasdelli.it >; 'Adam Armstrong' <adama@memetic.org mailto:adama@memetic.org > Subject: Re: [Observium] Distributed Poller
Hi,
what if a poller already has collected lots of local hosts and relevant data?
How could I transfer those data to the master (if possible)? Should they be merged with the master data, or do they have separate tables?
Thanks!
Luca
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Adam Armstrong via observium Sent: Thursday, April 4, 2019 10:47 PM To: Ron Culler via observium <observium@observium.org mailto:observium@observium.org > Cc: Adam Armstrong <adama@memetic.org mailto:adama@memetic.org > Subject: Re: [Observium] Distributed Poller
If you add devices from the cli of the poller, it'll be automatically added to that poller, using the poller_id defined in that install's config.php (that's why the poller_id is in the config, rather than being a command line option)
It doesn't really make much sense to add it from the central system, since it (often) can't do tests and initial poll/discover.
Adding from the UI might come in the future, I have a plan for that.
adam.
On 2019-04-04 21:05:55, Ron Culler via observium <observium@observium.org mailto:observium@observium.org > wrote:
I know in a standard install with a single server the poller_id is 0 in the DB. For a distributed poller setup do we need to specify $config[poller_id] in the config.php file on each poller? And how do we specify the poller_id when devices are added to the system? Thanks
Ron Culler
_______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Well, more in details.
At present I’ve got two partitioned pollers, both revision 9817 as per the master one.
The A poller works without any issue, and relevant rrds are created and displayed on master console as expected.
The B poller causes repeated “rrdcached handle_request_update: stat (rrd_path) failed”, and it’s true, because the relevant rrd_path is empty.
Permissions on rrd directory are as usual, so www-data:www-data 00775.
Either on observium log on master and failing poller I can’t see any issue.
Any thoughts?
Thanks!
Luca
From: observium observium-bounces@observium.org On Behalf Of Luca Sasdelli via observium Sent: Tuesday, April 9, 2019 9:18 AM To: 'Observium' observium@observium.org Cc: Luca Sasdelli luca@sasdelli.it; 'Adam Armstrong' adama@memetic.org Subject: Re: [Observium] R: Distributed Poller
Ok, thanks.
At present I see all partitioned poller device port graphs in error. By monitoring with tcpdump there is regular data flow from partitioned poller to master on port 42217. What else could I check?
Cheers
Luca
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Adam Armstrong via observium Sent: Tuesday, April 9, 2019 6:35 AM To: Luca Sasdelli via observium <observium@observium.org mailto:observium@observium.org > Cc: Adam Armstrong <adama@memetic.org mailto:adama@memetic.org > Subject: Re: [Observium] R: Distributed Poller
This is difficult because the central system has no ability to communicate with the satellite system, the communication only goes in the other direction.
It's possible we could add this in future via a few different methods, but at the moment you need to add directly from the partitioned poller's console.
adam.
On 2019-04-08 22:20:25, Luca Sasdelli via observium <observium@observium.org mailto:observium@observium.org > wrote:
Partitioned polling is definitely awesome for our needs.
Do you think it will be possible to add “partitioned” devices from “master” poller, by stating the poller name together with SNMP data? This would prevent to access the partitioned pollers each time a modification is needed there.
Cheers
Luca
Da: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > Per conto di Adam Armstrong via observium Inviato: lunedì 8 aprile 2019 20:42 A: Joelly, Alexander via observium <observium@observium.org mailto:observium@observium.org > Cc: Adam Armstrong <adama@memetic.org mailto:adama@memetic.org > Oggetto: Re: [Observium] Distributed Poller
Hi Alexander,
If you currently have two pollers which are sharing the workload, this is what we call "distributed polling", which distributes the polling workload between several systems in the same location.
This is done using command line parameters on the poller cron jobs to tell the processes how many total pollers there are, and which instance that particular poller is. You seem to have worked that part out.
Assigning devices to specific pollers is part of another set of features, what we call "partitioned pollers". This is where a satellite poller instance only polls devices directly assigned to it (and usually added via its console), this is useful behind firewalls and the like.
To turn a poller into a "partitioned poller", you smply add a poller_name to its configuration. This will prevent it from polling any device which isn't assigned to it. Usually you wouldn't use special cron job options with this.
For example, in config.php :
$config['poller_name'] = "poller_name_here";
For semi-obvious reasons, this option can't be configured in the database :)
You can't fix a device to a specific instance of a "distributed poller", because distributed pollers are just to share device polling load.
I actually had a bit of trouble trying to pick names for these features that weren't confusing!
Thanks,
adam.
On 2019-04-08 16:55:56, Joelly, Alexander via observium <observium@observium.org mailto:observium@observium.org > wrote:
I’ve deployed 2 pollers and adjusted the cronjobs on them and see in the Observium.log that half of the devices are polled from one and the other half from the second poller. On the device’s ‘General device settings’ tab there is dropdown for assigning(?) a poller but it’s greyed out with default poller. What do I miss to get that enabled and assign devices to a specific poller instance or that they stick with the poller they were added? Haven’t added anything poller related to the cofig file (also didn’t find anything documented) and noticed there is a table named pollers in the db but it’s empty.
Thanks, Alex
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Luca Sasdelli via observium Sent: Freitag, 05. April 2019 15:48 To: 'Observium' <observium@observium.org mailto:observium@observium.org > Cc: Luca Sasdelli <luca@sasdelli.it mailto:luca@sasdelli.it >; 'Adam Armstrong' <adama@memetic.org mailto:adama@memetic.org > Subject: Re: [Observium] Distributed Poller
Hi,
what if a poller already has collected lots of local hosts and relevant data?
How could I transfer those data to the master (if possible)? Should they be merged with the master data, or do they have separate tables?
Thanks!
Luca
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Adam Armstrong via observium Sent: Thursday, April 4, 2019 10:47 PM To: Ron Culler via observium <observium@observium.org mailto:observium@observium.org > Cc: Adam Armstrong <adama@memetic.org mailto:adama@memetic.org > Subject: Re: [Observium] Distributed Poller
If you add devices from the cli of the poller, it'll be automatically added to that poller, using the poller_id defined in that install's config.php (that's why the poller_id is in the config, rather than being a command line option)
It doesn't really make much sense to add it from the central system, since it (often) can't do tests and initial poll/discover.
Adding from the UI might come in the future, I have a plan for that.
adam.
On 2019-04-04 21:05:55, Ron Culler via observium <observium@observium.org mailto:observium@observium.org > wrote:
I know in a standard install with a single server the poller_id is 0 in the DB. For a distributed poller setup do we need to specify $config[poller_id] in the config.php file on each poller? And how do we specify the poller_id when devices are added to the system? Thanks
Ron Culler
_______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Are the config.php files identical on both partitioned pollers, except for the poller name?
The config.php should contain almost nothing, except the settings required to reach the central site and set the poller_name.
adam.
Sent from Mailbird [http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm...] On 2019-04-09 17:48:04, Luca Sasdelli via observium observium@observium.org wrote: Well, more in details. At present I’ve got two partitioned pollers, both revision 9817 as per the master one. The A poller works without any issue, and relevant rrds are created and displayed on master console as expected. The B poller causes repeated “rrdcached handle_request_update: stat (rrd_path) failed”, and it’s true, because the relevant rrd_path is empty. Permissions on rrd directory are as usual, so www-data:www-data 00775. Either on observium log on master and failing poller I can’t see any issue. Any thoughts? Thanks! Luca From: observium observium-bounces@observium.org On Behalf Of Luca Sasdelli via observium Sent: Tuesday, April 9, 2019 9:18 AM To: 'Observium' observium@observium.org Cc: Luca Sasdelli luca@sasdelli.it; 'Adam Armstrong' adama@memetic.org Subject: Re: [Observium] R: Distributed Poller Ok, thanks. At present I see all partitioned poller device port graphs in error. By monitoring with tcpdump there is regular data flow from partitioned poller to master on port 42217. What else could I check? Cheers Luca From: observium <observium-bounces@observium.org [mailto:observium-bounces@observium.org]> On Behalf Of Adam Armstrong via observium Sent: Tuesday, April 9, 2019 6:35 AM To: Luca Sasdelli via observium <observium@observium.org [mailto:observium@observium.org]> Cc: Adam Armstrong <adama@memetic.org [mailto:adama@memetic.org]> Subject: Re: [Observium] R: Distributed Poller This is difficult because the central system has no ability to communicate with the satellite system, the communication only goes in the other direction. It's possible we could add this in future via a few different methods, but at the moment you need to add directly from the partitioned poller's console. adam. On 2019-04-08 22:20:25, Luca Sasdelli via observium <observium@observium.org [mailto:observium@observium.org]> wrote: Partitioned polling is definitely awesome for our needs. Do you think it will be possible to add “partitioned” devices from “master” poller, by stating the poller name together with SNMP data? This would prevent to access the partitioned pollers each time a modification is needed there. Cheers Luca Da: observium <observium-bounces@observium.org [mailto:observium-bounces@observium.org]> Per conto di Adam Armstrong via observium Inviato: lunedì 8 aprile 2019 20:42 A: Joelly, Alexander via observium <observium@observium.org [mailto:observium@observium.org]> Cc: Adam Armstrong <adama@memetic.org [mailto:adama@memetic.org]> Oggetto: Re: [Observium] Distributed Poller Hi Alexander, If you currently have two pollers which are sharing the workload, this is what we call "distributed polling", which distributes the polling workload between several systems in the same location. This is done using command line parameters on the poller cron jobs to tell the processes how many total pollers there are, and which instance that particular poller is. You seem to have worked that part out. Assigning devices to specific pollers is part of another set of features, what we call "partitioned pollers". This is where a satellite poller instance only polls devices directly assigned to it (and usually added via its console), this is useful behind firewalls and the like. To turn a poller into a "partitioned poller", you smply add a poller_name to its configuration. This will prevent it from polling any device which isn't assigned to it. Usually you wouldn't use special cron job options with this. For example, in config.php : $config['poller_name'] = "poller_name_here"; For semi-obvious reasons, this option can't be configured in the database :) You can't fix a device to a specific instance of a "distributed poller", because distributed pollers are just to share device polling load. I actually had a bit of trouble trying to pick names for these features that weren't confusing! Thanks, adam. On 2019-04-08 16:55:56, Joelly, Alexander via observium <observium@observium.org [mailto:observium@observium.org]> wrote: I’ve deployed 2 pollers and adjusted the cronjobs on them and see in the Observium.log that half of the devices are polled from one and the other half from the second poller. On the device’s ‘General device settings’ tab there is dropdown for assigning(?) a poller but it’s greyed out with default poller. What do I miss to get that enabled and assign devices to a specific poller instance or that they stick with the poller they were added? Haven’t added anything poller related to the cofig file (also didn’t find anything documented) and noticed there is a table named pollers in the db but it’s empty.
Thanks, Alex
From: observium <observium-bounces@observium.org [mailto:observium-bounces@observium.org]> On Behalf Of Luca Sasdelli via observium Sent: Freitag, 05. April 2019 15:48 To: 'Observium' <observium@observium.org [mailto:observium@observium.org]> Cc: Luca Sasdelli <luca@sasdelli.it [mailto:luca@sasdelli.it]>; 'Adam Armstrong' <adama@memetic.org [mailto:adama@memetic.org]> Subject: Re: [Observium] Distributed Poller Hi, what if a poller already has collected lots of local hosts and relevant data? How could I transfer those data to the master (if possible)? Should they be merged with the master data, or do they have separate tables? Thanks! Luca From: observium <observium-bounces@observium.org [mailto:observium-bounces@observium.org]> On Behalf Of Adam Armstrong via observium Sent: Thursday, April 4, 2019 10:47 PM To: Ron Culler via observium <observium@observium.org [mailto:observium@observium.org]> Cc: Adam Armstrong <adama@memetic.org [mailto:adama@memetic.org]> Subject: Re: [Observium] Distributed Poller If you add devices from the cli of the poller, it'll be automatically added to that poller, using the poller_id defined in that install's config.php (that's why the poller_id is in the config, rather than being a command line option) It doesn't really make much sense to add it from the central system, since it (often) can't do tests and initial poll/discover. Adding from the UI might come in the future, I have a plan for that. adam. On 2019-04-04 21:05:55, Ron Culler via observium <observium@observium.org [mailto:observium@observium.org]> wrote: I know in a standard install with a single server the poller_id is 0 in the DB. For a distributed poller setup do we need to specify $config[poller_id] in the config.php file on each poller? And how do we specify the poller_id when devices are added to the system? Thanks
Ron Culler
_______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
Hi Adam,
yes, pollers config.php are as per your recommendation.
Only modified rows are database server, rrdcached and poller name.
The central site does get data from both pollers, but only one ends in updating rrd files: the other’s devices got nothing in respective rrd folders.
I’ve also tried to delete a device, check for deleted rrd folder on central one, re-added device: the folder is created on central site, but it stays empty while mentioned errors are shown in syslog.
Cheers
Luca
Da: observium observium-bounces@observium.org Per conto di Adam Armstrong via observium Inviato: mercoledì 10 aprile 2019 00:08 A: Adam Ward via observium observium@observium.org Cc: Adam Armstrong adama@memetic.org Oggetto: Re: [Observium] R: Distributed Poller
Are the config.php files identical on both partitioned pollers, except for the poller name?
The config.php should contain almost nothing, except the settings required to reach the central site and set the poller_name.
adam.
Sent from Mailbird http://www.getmailbird.com/?utm_source=Mailbird&utm_medium=email&utm_campaign=sent-from-mailbird
On 2019-04-09 17:48:04, Luca Sasdelli via observium <observium@observium.org mailto:observium@observium.org > wrote:
Well, more in details.
At present I’ve got two partitioned pollers, both revision 9817 as per the master one.
The A poller works without any issue, and relevant rrds are created and displayed on master console as expected.
The B poller causes repeated “rrdcached handle_request_update: stat (rrd_path) failed”, and it’s true, because the relevant rrd_path is empty.
Permissions on rrd directory are as usual, so www-data:www-data 00775.
Either on observium log on master and failing poller I can’t see any issue.
Any thoughts?
Thanks!
Luca
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Luca Sasdelli via observium Sent: Tuesday, April 9, 2019 9:18 AM To: 'Observium' <observium@observium.org mailto:observium@observium.org > Cc: Luca Sasdelli <luca@sasdelli.it mailto:luca@sasdelli.it >; 'Adam Armstrong' <adama@memetic.org mailto:adama@memetic.org > Subject: Re: [Observium] R: Distributed Poller
Ok, thanks.
At present I see all partitioned poller device port graphs in error. By monitoring with tcpdump there is regular data flow from partitioned poller to master on port 42217. What else could I check?
Cheers
Luca
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Adam Armstrong via observium Sent: Tuesday, April 9, 2019 6:35 AM To: Luca Sasdelli via observium <observium@observium.org mailto:observium@observium.org > Cc: Adam Armstrong <adama@memetic.org mailto:adama@memetic.org > Subject: Re: [Observium] R: Distributed Poller
This is difficult because the central system has no ability to communicate with the satellite system, the communication only goes in the other direction.
It's possible we could add this in future via a few different methods, but at the moment you need to add directly from the partitioned poller's console.
adam.
On 2019-04-08 22:20:25, Luca Sasdelli via observium <observium@observium.org mailto:observium@observium.org > wrote:
Partitioned polling is definitely awesome for our needs.
Do you think it will be possible to add “partitioned” devices from “master” poller, by stating the poller name together with SNMP data? This would prevent to access the partitioned pollers each time a modification is needed there.
Cheers
Luca
Da: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > Per conto di Adam Armstrong via observium Inviato: lunedì 8 aprile 2019 20:42 A: Joelly, Alexander via observium <observium@observium.org mailto:observium@observium.org > Cc: Adam Armstrong <adama@memetic.org mailto:adama@memetic.org > Oggetto: Re: [Observium] Distributed Poller
Hi Alexander,
If you currently have two pollers which are sharing the workload, this is what we call "distributed polling", which distributes the polling workload between several systems in the same location.
This is done using command line parameters on the poller cron jobs to tell the processes how many total pollers there are, and which instance that particular poller is. You seem to have worked that part out.
Assigning devices to specific pollers is part of another set of features, what we call "partitioned pollers". This is where a satellite poller instance only polls devices directly assigned to it (and usually added via its console), this is useful behind firewalls and the like.
To turn a poller into a "partitioned poller", you smply add a poller_name to its configuration. This will prevent it from polling any device which isn't assigned to it. Usually you wouldn't use special cron job options with this.
For example, in config.php :
$config['poller_name'] = "poller_name_here";
For semi-obvious reasons, this option can't be configured in the database :)
You can't fix a device to a specific instance of a "distributed poller", because distributed pollers are just to share device polling load.
I actually had a bit of trouble trying to pick names for these features that weren't confusing!
Thanks,
adam.
On 2019-04-08 16:55:56, Joelly, Alexander via observium <observium@observium.org mailto:observium@observium.org > wrote:
I’ve deployed 2 pollers and adjusted the cronjobs on them and see in the Observium.log that half of the devices are polled from one and the other half from the second poller. On the device’s ‘General device settings’ tab there is dropdown for assigning(?) a poller but it’s greyed out with default poller. What do I miss to get that enabled and assign devices to a specific poller instance or that they stick with the poller they were added? Haven’t added anything poller related to the cofig file (also didn’t find anything documented) and noticed there is a table named pollers in the db but it’s empty.
Thanks, Alex
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Luca Sasdelli via observium Sent: Freitag, 05. April 2019 15:48 To: 'Observium' <observium@observium.org mailto:observium@observium.org > Cc: Luca Sasdelli <luca@sasdelli.it mailto:luca@sasdelli.it >; 'Adam Armstrong' <adama@memetic.org mailto:adama@memetic.org > Subject: Re: [Observium] Distributed Poller
Hi,
what if a poller already has collected lots of local hosts and relevant data?
How could I transfer those data to the master (if possible)? Should they be merged with the master data, or do they have separate tables?
Thanks!
Luca
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Adam Armstrong via observium Sent: Thursday, April 4, 2019 10:47 PM To: Ron Culler via observium <observium@observium.org mailto:observium@observium.org > Cc: Adam Armstrong <adama@memetic.org mailto:adama@memetic.org > Subject: Re: [Observium] Distributed Poller
If you add devices from the cli of the poller, it'll be automatically added to that poller, using the poller_id defined in that install's config.php (that's why the poller_id is in the config, rather than being a command line option)
It doesn't really make much sense to add it from the central system, since it (often) can't do tests and initial poll/discover.
Adding from the UI might come in the future, I have a plan for that.
adam.
On 2019-04-04 21:05:55, Ron Culler via observium <observium@observium.org mailto:observium@observium.org > wrote:
I know in a standard install with a single server the poller_id is 0 in the DB. For a distributed poller setup do we need to specify $config[poller_id] in the config.php file on each poller? And how do we specify the poller_id when devices are added to the system? Thanks
Ron Culler
_______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Thanks Adam, got it.
From: observium observium-bounces@observium.org On Behalf Of Adam Armstrong via observium Sent: Montag, 08. April 2019 20:42 To: Joelly, Alexander via observium observium@observium.org Cc: Adam Armstrong adama@memetic.org Subject: Re: [Observium] Distributed Poller
Hi Alexander,
If you currently have two pollers which are sharing the workload, this is what we call "distributed polling", which distributes the polling workload between several systems in the same location.
This is done using command line parameters on the poller cron jobs to tell the processes how many total pollers there are, and which instance that particular poller is. You seem to have worked that part out.
Assigning devices to specific pollers is part of another set of features, what we call "partitioned pollers". This is where a satellite poller instance only polls devices directly assigned to it (and usually added via its console), this is useful behind firewalls and the like.
To turn a poller into a "partitioned poller", you smply add a poller_name to its configuration. This will prevent it from polling any device which isn't assigned to it. Usually you wouldn't use special cron job options with this.
For example, in config.php :
$config['poller_name'] = "poller_name_here";
For semi-obvious reasons, this option can't be configured in the database :)
You can't fix a device to a specific instance of a "distributed poller", because distributed pollers are just to share device polling load.
I actually had a bit of trouble trying to pick names for these features that weren't confusing!
Thanks, adam.
On 2019-04-08 16:55:56, Joelly, Alexander via observium <observium@observium.orgmailto:observium@observium.org> wrote: I’ve deployed 2 pollers and adjusted the cronjobs on them and see in the Observium.log that half of the devices are polled from one and the other half from the second poller. On the device’s ‘General device settings’ tab there is dropdown for assigning(?) a poller but it’s greyed out with default poller. What do I miss to get that enabled and assign devices to a specific poller instance or that they stick with the poller they were added? Haven’t added anything poller related to the cofig file (also didn’t find anything documented) and noticed there is a table named pollers in the db but it’s empty.
Thanks, Alex
From: observium <observium-bounces@observium.orgmailto:observium-bounces@observium.org> On Behalf Of Luca Sasdelli via observium Sent: Freitag, 05. April 2019 15:48 To: 'Observium' <observium@observium.orgmailto:observium@observium.org> Cc: Luca Sasdelli <luca@sasdelli.itmailto:luca@sasdelli.it>; 'Adam Armstrong' <adama@memetic.orgmailto:adama@memetic.org> Subject: Re: [Observium] Distributed Poller
Hi,
what if a poller already has collected lots of local hosts and relevant data? How could I transfer those data to the master (if possible)? Should they be merged with the master data, or do they have separate tables?
Thanks! Luca
From: observium <observium-bounces@observium.orgmailto:observium-bounces@observium.org> On Behalf Of Adam Armstrong via observium Sent: Thursday, April 4, 2019 10:47 PM To: Ron Culler via observium <observium@observium.orgmailto:observium@observium.org> Cc: Adam Armstrong <adama@memetic.orgmailto:adama@memetic.org> Subject: Re: [Observium] Distributed Poller
If you add devices from the cli of the poller, it'll be automatically added to that poller, using the poller_id defined in that install's config.php (that's why the poller_id is in the config, rather than being a command line option)
It doesn't really make much sense to add it from the central system, since it (often) can't do tests and initial poll/discover.
Adding from the UI might come in the future, I have a plan for that.
adam.
On 2019-04-04 21:05:55, Ron Culler via observium <observium@observium.orgmailto:observium@observium.org> wrote: I know in a standard install with a single server the poller_id is 0 in the DB. For a distributed poller setup do we need to specify $config[poller_id] in the config.php file on each poller? And how do we specify the poller_id when devices are added to the system? Thanks
Ron Culler
_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
participants (5)
-
Adam Armstrong -
Joelly, Alexander -
Luca Sasdelli -
Luca Sasdelli
-
Ron Culler