Probably your RAID array gets killed when the poller start and trying to write to 1 billion RRD-files.
You could check 'top' in linux on your box when you start the poller and watch the iowait. If thats goes through the roof to problem is I/O. Your disks are to slow

/Markus

2015-01-29 15:09 GMT+01:00 Jerome Jenkins <JJENKINS@wpahs.org>:

Appologies for lack of info:

 

Shows total of 1027 devices, NOT 1200 (original list was 1200 but not all of it imported)

 

Observium is stored on /opt (which is a raid-array of (6) 146GB drives)

 

root@observium:~# df -h

Filesystem                                                          Size        Used     Avail Use% Mounted on

/dev/mapper/root-root                               9.7G       2.4G       6.8G  27%            /

none                                                                     4.0K         0            4.0K   0%              /sys/fs/cgroup

udev                                                                      12G        4.0K       12G    1%              /dev

tmpfs                                                                    2.4G       612K      2.4G   1%              /run

none                                                                     5.0M     0              5.0M   0%             /run/lock

none                                                                     12G        0              12G     0%             /run/shm

none                                                                     100M     0             100M   0%            /run/user

/dev/mapper/observium--data-data      684G      195G      455G  30%           /opt

 

 

(2) Quad-core processors:

 

cpu family      : 6

model           : 23

model name      : Intel(R) Xeon(R) CPU           L5420  @ 2.50GHz

stepping        : 10

microcode       : 0xa07

cpu MHz         : 2499.803

cache size      : 6144 KB

 

 

 

Meminfo:

 

MemTotal:       24689672 kB

MemFree:        19383548 kB

Buffers:          124572 kB

Cached:          3489096 kB

SwapCached:            0 kB

Active:          1476312 kB

Inactive:        3352276 kB

Active(anon):    1220624 kB

Inactive(anon):    23092 kB

Active(file):     255688 kB

Inactive(file):  3329184 kB

Unevictable:           0 kB

Mlocked:               0 kB

SwapTotal:      25161724 kB

SwapFree:       25161724 kB

Dirty:            353976 kB

Writeback:          2704 kB

AnonPages:       1214728 kB

Mapped:            56804 kB

Shmem:             28796 kB

Slab:             210032 kB

SReclaimable:     178204 kB

SUnreclaim:        31828 kB

KernelStack:        3552 kB

PageTables:        30896 kB

NFS_Unstable:          0 kB

Bounce:                0 kB

WritebackTmp:          0 kB

CommitLimit:    37506560 kB

Committed_AS:    2858996 kB

VmallocTotal:   34359738367 kB

VmallocUsed:      328028 kB

VmallocChunk:   34359404252 kB

HardwareCorrupted:     0 kB

AnonHugePages:    106496 kB

HugePages_Total:       0

HugePages_Free:        0

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:       2048 kB

DirectMap4k:       59728 kB

DirectMap2M:    25104384 kB

 

 

 

From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong
Sent: Tuesday, January 27, 2015 5:18 PM
To: Observium Network Observation System
Subject: Re: [Observium] webpage no longer loads or takes forever

 

Nice that you told us what ram and CPU you have. Less nice that you didn't tell us what storage you're using.

Thus, insufficient information...

Adam.

Sent with AquaMail for Android
http://www.aqua-mail.com

On 27 January 2015 13:56:17 Jerome Jenkins <JJENKINS@wpahs.org> wrote:

Running on Ubuntu Server LTS 14.0.1

 

Server is 8 cores with 24GB of memory

 

Was working perfectly during initial install of the software. Once I let it discover all of our routers and switches (approx. 1200 devices) things went downhill. Login to the cli of the server takes very long and most times will time-out. Webpage never loads. Only way to fix is to reboot the server and once it begins polling everything again…nothing is accessible. I changed the setting for php to use 3GB of memory and change the poller instances per the performance tuning guide for observium to

 

*/5 * * * * root /opt/observium/poller-wrapper.py 10 >> /dev/null 2>&1

 

Seems that the memory and processor are probably being consumed by the poller?

 

Jerome S. Jenkins, Sr.

Sr. Network Engineer

jjenkins@wpahs.org

 

*************************************************************************
Confidentiality Notice:  This e-mail message and any attachments are
intended solely for the individual or individuals designated above. It may
contain confidential or proprietary information and may be subject to
confidentiality protections.  If you are not the intended recipient, you
are not authorized to read, copy, retain or distribute this message.  If
you receive this message in error, please notify the sender by reply e-mail
and delete this message. The sender does not waive any privilege or right
of privacy or confidentiality that may attach to this communication.  Thank
you.
*************************************************************************
 

_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

*************************************************************************
Confidentiality Notice:  This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections.  If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message.  If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication.  Thank you.
*************************************************************************


_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium