With that huge amount of logs, you might require a dedicated server (or more) for logging.  I have been testing ELMA, which is a complete Linux distribution (based on Suse) that has very similar components to what Joseph described.

http://enterprise-log-management-appliance.org/

Cheers,

Tristan

Tristan Rhodes
Network Engineer
Weber State University
801.626.8549


On Mon, Nov 10, 2014 at 2:03 PM, Lane Eckley <lane@staff.hypernia.com> wrote:

Thanks!

I am not aware of a way to integrate it with Observium though which is rather unfortunate as that's the primary reason of not wanting to go with something like splunk,etc.

Maybe I am missing something?

On Nov 10, 2014 3:59 PM, "Joseph L. Brunner" <joe@affirmedsystems.com> wrote:

You really want to use the “ELK” based logging with a scale-out infrastructure for this volume.

 

You can start  here https://blog.devita.co/2014/09/04/monitoring-pfsense-firewall-logs-with-elk-logstash-kibana-elasticsearch/

 

 

 

From: observium [mailto:observium-bounces@observium.org] On Behalf Of Lane Eckley
Sent: Monday, November 10, 2014 03:04 PM
To: Observium Network Observation System
Subject: [Observium] Syslogging & Expansion

 

Hi Everyone,

 

We are considering the possibility of using Observium's syslog capability to handle the syslog output of 50 machines which will be spitting out roughly 300K lines of log per hour for a grand total in the ball park 360,000,000 log entries per 24 hours.

 

Has anyone used/attempted to handle this volume of logging with Observium in the past? If so, would you mind sharing your experience?

 

I am also looking for feedback on hardware suggestions for both the Observium machine as well for the standalone database server.

 

Any feedback is appreciated!

 

Thanks,

 

-Lane


_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium


_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium