Indeed.

Or if it's super important that you can see graphs when your monitoring system explodes, run two copies, assuming your infrastructure can handle being polled twice.

adam.

On 2014-01-30 14:24, Johnathon Voegeli wrote:

My suggestion is to monitor your monitoring tool... Use site24x7.com and use the unix monitor to make sure observium works. Works great for us.

Sent from my iPhone

On Jan 30, 2014, at 14:32, "Adam Armstrong" adama@memetic.org wrote:

Nope. The level of complexity (and by extension, evil, evil bugs) introduced by such schemes outweigh the benefits, imo.

I'm not entirely sure why anyone would even want such a thing, your monitoring system is an operational tool not a production service, but there we go.

adam.

On 2014-01-30 05:37, Adriaan Smuts wrote: Good day, We are on the subscription version of Observium and loving the product so far! I need to make Observium Highly Available as this is becoming one of our main monitoring systems. Is there any documentation on how to implement this? Regards ADRIAAN SMUTS SYSTEMS ADMINISTRATOR

---

Direct Line +27 21 464 9565 Reception 086 000 9500 Website WWW.WEBAFRICA.CO.ZA [1] Links:

---

[1] http://www.webafrica.co.za/ _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

On 31/01/2014 19:17, Scott Brawner wrote:

Or script a failover server that polls the primary and replicates config, then stands up and takes over if the primary dies.

Database consistency might be an issue unless you use a 3rd server for db.

You'll need 2 in master-master (uh oh) to not make that a SPoF... ;)

I run 2 observiums side by side. I usually forget to delete stuff from the second, but it picks up most new devices through autodiscovery. Sometimes I sync DB and RRD manually. :-)

Tom

Thanks, Scott Brawner

-----Original Message----- From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Thursday, January 30, 2014 3:25 PM To: Observium Network Observation System Subject: Re: [Observium] Observium HA

Indeed.

Or if it's super important that you can see graphs when your monitoring system explodes, run two copies, assuming your infrastructure can handle being polled twice.

adam.

On 2014-01-30 14:24, Johnathon Voegeli wrote:

...

My suggestion is to monitor your monitoring tool... Use site24x7.com and use the unix monitor to make sure observium works. Works great for us.

Sent from my iPhone

On Jan 30, 2014, at 14:32, "Adam Armstrong" adama@memetic.org wrote:

Nope. The level of complexity (and by extension, evil, evil bugs) introduced by such schemes outweigh the benefits, imo.

I'm not entirely sure why anyone would even want such a thing, your monitoring system is an operational tool not a production service, but there we go.

adam.

On 2014-01-30 05:37, Adriaan Smuts wrote: Good day, We are on the subscription version of Observium and loving the product so far! I need to make Observium Highly Available as this is becoming one of our main monitoring systems. Is there any documentation on how to implement this? Regards ADRIAAN SMUTS SYSTEMS ADMINISTRATOR

---

Direct Line +27 21 464 9565 Reception 086 000 9500 Website WWW.WEBAFRICA.CO.ZA [1] Links:

---

[1] http://www.webafrica.co.za/ _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

---

observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

---

observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

What Gareth said. load balance in the front, shared file system, some hacking, and shared DB, but who knows what would actually happen without testing.

Some of the other players (Orion, PRTG) support their own clustering within their server and secondary monitoring nodes so you're actually monitoring from two locations. Once Observium takes off and Adam & crew become millionaires, maybe we'll get some of the uber-fancy stuff too.

Robbie Wright Siuslaw Broadband http://siuslawbroadband.com 541-902-5101

**For support issues, please email support@siuslawbroadband.com.**

On Fri, Jan 31, 2014 at 11:50 AM, Gareth.Saxby@bluesky.co.uk wrote:

Like scripting, but 'proper' and active/active;

• MySQL replication and clustering with Corosync/Pacemaker
• Crontabs managed by Pacemaker
• haproxy to the front end servers
• gluster or another clusters storage solution for the RRDs (iSCSI, fiber

LUNs, whatever)

There's far too many ways to HA a PHP/MySQL app like Observium, and I'm sure it's full of gotchas along the way. I'd look into HAing the server itself ultimately; we have VMWare HA and it vmotions our servers if needs be so we don't have to manage the app and run conf management just for tertiary monitoring... Add multipathed RAID storage and regular backups and we have no need to cluster the actual services.

You're never going to get 99.999% uptime out of a system not designed with HA in mind from the start, you'll always end up being caught on a snag with the code itself, so it's a matter of making the best of what you have and not wasting all your time on it when you could be improving other things.

-Gareth

...

On 31 Jan 2014, at 18:18, "Scott Brawner"

Scott.Brawner@kratosdefense.com wrote:

...

Or script a failover server that polls the primary and replicates config, then stands up and takes over if the primary dies.

Database consistency might be an issue unless you use a 3rd server for

db.

...

Thanks, Scott Brawner

-----Original Message----- From: observium [mailto:observium-bounces@observium.org] On Behalf Of

Adam

...

Armstrong Sent: Thursday, January 30, 2014 3:25 PM To: Observium Network Observation System Subject: Re: [Observium] Observium HA

Indeed.

Or if it's super important that you can see graphs when your monitoring system explodes, run two copies, assuming your infrastructure can handle being polled twice.

adam.

On 2014-01-30 14:24, Johnathon Voegeli wrote:

...

My suggestion is to monitor your monitoring tool... Use site24x7.com and use the unix monitor to make sure observium works. Works great for us.

Sent from my iPhone

On Jan 30, 2014, at 14:32, "Adam Armstrong" adama@memetic.org wrote:

Nope. The level of complexity (and by extension, evil, evil bugs) introduced by such schemes outweigh the benefits, imo.

I'm not entirely sure why anyone would even want such a thing, your monitoring system is an operational tool not a production service, but there we go.

adam.

On 2014-01-30 05:37, Adriaan Smuts wrote: Good day, We are on the subscription version of Observium and loving the product so far! I need to make Observium Highly Available as this is becoming one of our main monitoring systems. Is there any documentation on how to implement this? Regards ADRIAAN SMUTS SYSTEMS ADMINISTRATOR

---

Direct Line +27 21 464 9565 Reception 086 000 9500 Website WWW.WEBAFRICA.CO.ZA [1] Links:

---

[1] http://www.webafrica.co.za/ _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

---

observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium <smime.p7s>

Blue Sky Hosting Ltd is registered in England & Wales with company no. 04753004. Registered office: 19 Catherine Place, London, SW1E 6DX.

---

observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

For large installations I tend to recommend SSD (which needs frequent backups because of the style of the RRD write load) or ramdisk (which needs frequent backups, because, well, lol) RRD storage and as an Observium installation consists of db, rrd and a single config file, it's trivially easy to restore in a couple of minutes.

Backing up large amounts of RRD from a fast read storage device like ramdisk or SSD works best if you compress them before writing them, as they compress really well.

adam.

On 2014-02-02 15:04, Tom Henderson wrote:

OBSERVIUM NETWORK OBSERVATION SYSTEM OBSERVIUM@OBSERVIUM.ORG ON SATURDAY, 1 FEB AT 8:50 AM +1300 WROTE: we have VMWare HA and it vmotions our servers if needs be

Combined with good backups this seems like the easiest option if you're already virtualised (works fine for us), but as observium grows you may find you need the performance of dedicated hardware.

Cheers

Tom Henderson

IT Manager | Pack & Company | 027 461 4403 | 09 929 2750 | tom@pack.co.nz

--

---

If you are not the intended recipient of this email, you must not disclose, copy, or distribute any information contained in or attached to it, or take any action/inaction based on it. Please contact us immediately, and delete this email from your system. Any views or opinions presented in this email are solely those of the author and not of Pack & Company (NZ) Limited (the "Company"), however any intellectual property contained in or attached to this email is the property of the Company. Thank you. _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium