We are currently testing Graylog2 on dedicated hardware. Our Observium instance wasn’t able to take the extra load and IO.

Would be nice to integrate the Syslog and Observium servers…

Adriaan Smuts

Systems Administrator - Windows

Direct Line:	+27 21 464 9565
Reception:	086 000 9500
www.webafrica.co.za

From: observium [mailto:observium-bounces@observium.org] On Behalf Of Tristan Rhodes
Sent: 10 November 2014 11:07 PM
To: Observium Network Observation System
Subject: Re: [Observium] Syslogging & Expansion

With that huge amount of logs, you might require a dedicated server (or more) for logging.  I have been testing ELMA, which is a complete Linux distribution (based on Suse) that has very similar components to what Joseph described.

http://enterprise-log-management-appliance.org/

Cheers,

Tristan

Tristan Rhodes
Network Engineer

Weber State University

801.626.8549

On Mon, Nov 10, 2014 at 2:03 PM, Lane Eckley <lane@staff.hypernia.com> wrote:

Thanks!

I am not aware of a way to integrate it with Observium though which is rather unfortunate as that's the primary reason of not wanting to go with something like splunk,etc.

Maybe I am missing something?

On Nov 10, 2014 3:59 PM, "Joseph L. Brunner" <joe@affirmedsystems.com> wrote:

You really want to use the “ELK” based logging with a scale-out infrastructure for this volume.

You can start  here https://blog.devita.co/2014/09/04/monitoring-pfsense-firewall-logs-with-elk-logstash-kibana-elasticsearch/

From: observium [mailto:observium-bounces@observium.org] On Behalf Of Lane Eckley
Sent: Monday, November 10, 2014 03:04 PM
To: Observium Network Observation System
Subject: [Observium] Syslogging & Expansion

Hi Everyone,

We are considering the possibility of using Observium's syslog capability to handle the syslog output of 50 machines which will be spitting out roughly 300K lines of log per hour for a grand total in the ball park 360,000,000 log entries per 24 hours.

Has anyone used/attempted to handle this volume of logging with Observium in the past? If so, would you mind sharing your experience?

I am also looking for feedback on hardware suggestions for both the Observium machine as well for the standalone database server.

Any feedback is appreciated!

Thanks,

-Lane

_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium