Hi,
A solution for a large network like yours could be using a distributed architecture. Here is a feedback from our usage : Current architecture in our organisation graphs 2000 devices and 70k ports and target is 6000 devices. We have built a farm of Observium pollers ( 6 physical servers ) and 1 more for mysql server (+Apache for GUI). Our 6 poller servers are 70% CPU idle and disks are ok for now. We progressively are adding new devices.
We did a small patch in mysql scheme, to add a "poller_id" into devices table. At each new device added, script "add_device.php" choose the poller the least used (based on numbers of devices per poller server). Each poller asks to SQL Server devices with its poller_id. The GUI server can read via NFS all rrd files stored in pollers. Hardware used for each poller is HP DL360 Gen7 / 4 GB RAM / 6 disks SAS 10k Raid 1+0 / 2*E5620 2.40Ghz (4 cores each). Here is a diagram : http://imagizer.imageshack.us/a/img538/3225/iMXiYb.png
By the way, may it be a good idea to add this distributed feature for monitoring of large platforms (adding the poller_id field into devices table) ?
Cheers, Antoine Desir
________________________________
Ce message et les pi?ces jointes sont confidentiels et r?serv?s ? l'usage exclusif de ses destinataires. Il peut ?galement ?tre prot?g? par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir imm?diatement l'exp?diteur et de le d?truire. L'int?grit? du message ne pouvant ?tre assur?e sur Internet, la responsabilit? de Worldline ne pourra ?tre recherch?e quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'exp?diteur ne donne aucune garantie ? cet ?gard et sa responsabilit? ne saurait ?tre recherch?e pour tout dommage r?sultant d'un virus transmis.
This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.
Hi,
I love this kind of solution :D
On my own I have added rrdcached and fix (for now) lots of IO issues... :)
Xavier
----- Mail original -----
De: "Desir Antoine" Antoine.Desir@worldline.com À: observium@observium.org Envoyé: Lundi 17 Novembre 2014 10:42:34 Objet: [Observium] Performance
Hi,
A solution for a large network like yours could be using a distributed architecture.
Here is a feedback from our usage :
Current architecture in our organisation graphs 2000 devices and 70k ports and target is 6000 devices.
We have built a farm of Observium pollers ( 6 physical servers ) and 1 more for mysql server (+Apache for GUI).
Our 6 poller servers are 70% CPU idle and disks are ok for now. We progressively are adding new devices.
We did a small patch in mysql scheme, to add a "poller_id" into devices table.
At each new device added, script "add_device.php" choose the poller the least used (based on numbers of devices per poller server).
Each poller asks to SQL Server devices with its poller_id.
The GUI server can read via NFS all rrd files stored in pollers.
Hardware used for each poller is HP DL360 Gen7 / 4 GB RAM / 6 disks SAS 10k Raid 1+0 / 2*E5620 2.40Ghz (4 cores each).
Here is a diagram : http://imagizer.imageshack.us/a/img538/3225/iMXiYb.png
By the way, may it be a good idea to add this distributed feature for monitoring of large platforms (adding the poller_id field into devices table) ?
Cheers,
Antoine Desir
Ce message et les pièces jointes sont confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut également être protégé par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire. L'intégrité du message ne pouvant être assurée sur Internet, la responsabilité de Worldline ne pourra être recherchée quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à cet égard et sa responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis.
This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Thank you for sharing your architecture. I have several questions about your MySQL server machine. Is it the same configurations as in poller servers? 1. How much % of CPU is in use? 2. How much memory is in use?
2014-11-17 13:42 GMT+04:00 Desir Antoine Antoine.Desir@worldline.com:
Hi,
A solution for a large network like yours could be using a distributed architecture.
Here is a feedback from our usage :
Current architecture in our organisation graphs 2000 devices and 70k ports and target is 6000 devices.
We have built a farm of Observium pollers ( 6 physical servers ) and 1 more for mysql server (+Apache for GUI).
Our 6 poller servers are 70% CPU idle and disks are ok for now. We progressively are adding new devices.
We did a small patch in mysql scheme, to add a "poller_id" into devices table.
At each new device added, script "add_device.php" choose the poller the least used (based on numbers of devices per poller server).
Each poller asks to SQL Server devices with its poller_id.
The GUI server can read via NFS all rrd files stored in pollers.
Hardware used for each poller is HP DL360 Gen7 / 4 GB RAM / 6 disks SAS 10k Raid 1+0 / 2*E5620 2.40Ghz (4 cores each).
Here is a diagram : http://imagizer.imageshack.us/a/img538/3225/iMXiYb.png
By the way, may it be a good idea to add this distributed feature for monitoring of large platforms (adding the poller_id field into devices table) ?
Cheers,
Antoine Desir
Ce message et les pièces jointes sont confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut également être protégé par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire. L'intégrité du message ne pouvant être assurée sur Internet, la responsabilité de Worldline ne pourra être recherchée quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à cet égard et sa responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis.
This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
participants (3)
-
Desir Antoine -
Xavier Beaudouin -
Муслим Меджлумов