webpage no longer loads or takes forever
Running on Ubuntu Server LTS 14.0.1
Server is 8 cores with 24GB of memory
Was working perfectly during initial install of the software. Once I let it discover all of our routers and switches (approx. 1200 devices) things went downhill. Login to the cli of the server takes very long and most times will time-out. Webpage never loads. Only way to fix is to reboot the server and once it begins polling everything again...nothing is accessible. I changed the setting for php to use 3GB of memory and change the poller instances per the performance tuning guide for observium to
*/5 * * * * root /opt/observium/poller-wrapper.py 10 >> /dev/null 2>&1
Seems that the memory and processor are probably being consumed by the poller?
Jerome S. Jenkins, Sr. Sr. Network Engineer jjenkins@wpahs.org
************************************************************************* Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you. *************************************************************************
Whoa, 1200 devices is quite a big amount of stuff to sift through. At that scale, I bet that I/O is probably a big issue and worth looking into.
On 1/27/2015 2:56 PM, Jerome Jenkins wrote:
Running on Ubuntu Server LTS 14.0.1
Server is 8 cores with 24GB of memory
Was working perfectly during initial install of the software. Once I let it discover all of our routers and switches (approx. 1200 devices) things went downhill. Login to the cli of the server takes very long and most times will time-out. Webpage never loads. Only way to fix is to reboot the server and once it begins polling everything again…nothing is accessible. I changed the setting for php to use 3GB of memory and change the poller instances per the performance tuning guide for observium to
*/5 * * * * root /opt/observium/poller-wrapper.py 10 >> /dev/null 2>&1
Seems that the memory and processor are probably being consumed by the poller?
*_Jerome S. Jenkins, Sr._*
Sr. Network Engineer
/jjenkins@wpahs.org/
Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you.
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Nice that you told us what ram and CPU you have. Less nice that you didn't tell us what storage you're using.
Thus, insufficient information...
Adam.
Sent with AquaMail for Android http://www.aqua-mail.com
On 27 January 2015 13:56:17 Jerome Jenkins JJENKINS@wpahs.org wrote:
Running on Ubuntu Server LTS 14.0.1
Server is 8 cores with 24GB of memory
Was working perfectly during initial install of the software. Once I let it discover all of our routers and switches (approx. 1200 devices) things went downhill. Login to the cli of the server takes very long and most times will time-out. Webpage never loads. Only way to fix is to reboot the server and once it begins polling everything again...nothing is accessible. I changed the setting for php to use 3GB of memory and change the poller instances per the performance tuning guide for observium to
*/5 * * * * root /opt/observium/poller-wrapper.py 10 >> /dev/null 2>&1
Seems that the memory and processor are probably being consumed by the poller?
Jerome S. Jenkins, Sr. Sr. Network Engineer jjenkins@wpahs.org
Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you.
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Appologies for lack of info:
Shows total of 1027 devices, NOT 1200 (original list was 1200 but not all of it imported)
Observium is stored on /opt (which is a raid-array of (6) 146GB drives)
root@observium:~# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/root-root 9.7G 2.4G 6.8G 27% / none 4.0K 0 4.0K 0% /sys/fs/cgroup udev 12G 4.0K 12G 1% /dev tmpfs 2.4G 612K 2.4G 1% /run none 5.0M 0 5.0M 0% /run/lock none 12G 0 12G 0% /run/shm none 100M 0 100M 0% /run/user /dev/mapper/observium--data-data 684G 195G 455G 30% /opt
(2) Quad-core processors:
cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5420 @ 2.50GHz stepping : 10 microcode : 0xa07 cpu MHz : 2499.803 cache size : 6144 KB
Meminfo:
MemTotal: 24689672 kB MemFree: 19383548 kB Buffers: 124572 kB Cached: 3489096 kB SwapCached: 0 kB Active: 1476312 kB Inactive: 3352276 kB Active(anon): 1220624 kB Inactive(anon): 23092 kB Active(file): 255688 kB Inactive(file): 3329184 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 25161724 kB SwapFree: 25161724 kB Dirty: 353976 kB Writeback: 2704 kB AnonPages: 1214728 kB Mapped: 56804 kB Shmem: 28796 kB Slab: 210032 kB SReclaimable: 178204 kB SUnreclaim: 31828 kB KernelStack: 3552 kB PageTables: 30896 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 37506560 kB Committed_AS: 2858996 kB VmallocTotal: 34359738367 kB VmallocUsed: 328028 kB VmallocChunk: 34359404252 kB HardwareCorrupted: 0 kB AnonHugePages: 106496 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 59728 kB DirectMap2M: 25104384 kB
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Tuesday, January 27, 2015 5:18 PM To: Observium Network Observation System Subject: Re: [Observium] webpage no longer loads or takes forever
Nice that you told us what ram and CPU you have. Less nice that you didn't tell us what storage you're using.
Thus, insufficient information...
Adam.
Sent with AquaMail for Android http://www.aqua-mail.com
On 27 January 2015 13:56:17 Jerome Jenkins <JJENKINS@wpahs.orgmailto:JJENKINS@wpahs.org> wrote: Running on Ubuntu Server LTS 14.0.1
Server is 8 cores with 24GB of memory
Was working perfectly during initial install of the software. Once I let it discover all of our routers and switches (approx. 1200 devices) things went downhill. Login to the cli of the server takes very long and most times will time-out. Webpage never loads. Only way to fix is to reboot the server and once it begins polling everything again...nothing is accessible. I changed the setting for php to use 3GB of memory and change the poller instances per the performance tuning guide for observium to
*/5 * * * * root /opt/observium/poller-wrapper.py 10 >> /dev/null 2>&1
Seems that the memory and processor are probably being consumed by the poller?
Jerome S. Jenkins, Sr. Sr. Network Engineer jjenkins@wpahs.orgmailto:jjenkins@wpahs.org
*************************************************************************
Confidentiality Notice: This e-mail message and any attachments are
intended solely for the individual or individuals designated above. It may
contain confidential or proprietary information and may be subject to
confidentiality protections. If you are not the intended recipient, you
are not authorized to read, copy, retain or distribute this message. If
you receive this message in error, please notify the sender by reply e-mail
and delete this message. The sender does not waive any privilege or right
of privacy or confidentiality that may attach to this communication. Thank
you.
*************************************************************************
_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
************************************************************************* Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you. *************************************************************************
Probably your RAID array gets killed when the poller start and trying to write to 1 billion RRD-files. You could check 'top' in linux on your box when you start the poller and watch the iowait. If thats goes through the roof to problem is I/O. Your disks are to slow
/Markus
2015-01-29 15:09 GMT+01:00 Jerome Jenkins JJENKINS@wpahs.org:
Appologies for lack of info:
Shows total of 1027 devices, NOT 1200 (original list was 1200 but not all of it imported)
Observium is stored on /opt (which is a raid-array of (6) 146GB drives)
root@observium:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/root-root 9.7G 2.4G 6.8G 27% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 12G 4.0K 12G 1% /dev
tmpfs 2.4G 612K 2.4G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 12G 0 12G 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/mapper/observium--data-data 684G 195G 455G 30% /opt
(2) Quad-core processors:
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU L5420 @ 2.50GHz
stepping : 10
microcode : 0xa07
cpu MHz : 2499.803
cache size : 6144 KB
Meminfo:
MemTotal: 24689672 kB
MemFree: 19383548 kB
Buffers: 124572 kB
Cached: 3489096 kB
SwapCached: 0 kB
Active: 1476312 kB
Inactive: 3352276 kB
Active(anon): 1220624 kB
Inactive(anon): 23092 kB
Active(file): 255688 kB
Inactive(file): 3329184 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 25161724 kB
SwapFree: 25161724 kB
Dirty: 353976 kB
Writeback: 2704 kB
AnonPages: 1214728 kB
Mapped: 56804 kB
Shmem: 28796 kB
Slab: 210032 kB
SReclaimable: 178204 kB
SUnreclaim: 31828 kB
KernelStack: 3552 kB
PageTables: 30896 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 37506560 kB
Committed_AS: 2858996 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 328028 kB
VmallocChunk: 34359404252 kB
HardwareCorrupted: 0 kB
AnonHugePages: 106496 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 59728 kB
DirectMap2M: 25104384 kB
*From:* observium [mailto:observium-bounces@observium.org] *On Behalf Of *Adam Armstrong *Sent:* Tuesday, January 27, 2015 5:18 PM *To:* Observium Network Observation System *Subject:* Re: [Observium] webpage no longer loads or takes forever
Nice that you told us what ram and CPU you have. Less nice that you didn't tell us what storage you're using.
Thus, insufficient information...
Adam.
Sent with AquaMail for Android http://www.aqua-mail.com
On 27 January 2015 13:56:17 Jerome Jenkins JJENKINS@wpahs.org wrote:
Running on Ubuntu Server LTS 14.0.1
Server is 8 cores with 24GB of memory
Was working perfectly during initial install of the software. Once I let it discover all of our routers and switches (approx. 1200 devices) things went downhill. Login to the cli of the server takes very long and most times will time-out. Webpage never loads. Only way to fix is to reboot the server and once it begins polling everything again…nothing is accessible. I changed the setting for php to use 3GB of memory and change the poller instances per the performance tuning guide for observium to
*/5 * * * * root /opt/observium/poller-wrapper.py 10 >> /dev/null 2>&1
Seems that the memory and processor are probably being consumed by the poller?
*Jerome S. Jenkins, Sr.*
Sr. Network Engineer
*jjenkins@wpahs.org jjenkins@wpahs.org*
Confidentiality Notice: This e-mail message and any attachments are
intended solely for the individual or individuals designated above. It may
contain confidential or proprietary information and may be subject to
confidentiality protections. If you are not the intended recipient, you
are not authorized to read, copy, retain or distribute this message. If
you receive this message in error, please notify the sender by reply e-mail
and delete this message. The sender does not waive any privilege or right
of privacy or confidentiality that may attach to this communication. Thank
you.
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you.
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
I was able to gather the stats shortly after rebooting by holding the power button. After running for about 30mins now....the webpage now gives the error:
"MySQL Error: Too many connections"
From: Jerome Jenkins Sent: Thursday, January 29, 2015 9:10 AM To: Observium Network Observation System Subject: RE: [Observium] webpage no longer loads or takes forever
Appologies for lack of info:
Shows total of 1027 devices, NOT 1200 (original list was 1200 but not all of it imported)
Observium is stored on /opt (which is a raid-array of (6) 146GB drives)
root@observium:~# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/root-root 9.7G 2.4G 6.8G 27% / none 4.0K 0 4.0K 0% /sys/fs/cgroup udev 12G 4.0K 12G 1% /dev tmpfs 2.4G 612K 2.4G 1% /run none 5.0M 0 5.0M 0% /run/lock none 12G 0 12G 0% /run/shm none 100M 0 100M 0% /run/user /dev/mapper/observium--data-data 684G 195G 455G 30% /opt
(2) Quad-core processors:
cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5420 @ 2.50GHz stepping : 10 microcode : 0xa07 cpu MHz : 2499.803 cache size : 6144 KB
Meminfo:
MemTotal: 24689672 kB MemFree: 19383548 kB Buffers: 124572 kB Cached: 3489096 kB SwapCached: 0 kB Active: 1476312 kB Inactive: 3352276 kB Active(anon): 1220624 kB Inactive(anon): 23092 kB Active(file): 255688 kB Inactive(file): 3329184 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 25161724 kB SwapFree: 25161724 kB Dirty: 353976 kB Writeback: 2704 kB AnonPages: 1214728 kB Mapped: 56804 kB Shmem: 28796 kB Slab: 210032 kB SReclaimable: 178204 kB SUnreclaim: 31828 kB KernelStack: 3552 kB PageTables: 30896 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 37506560 kB Committed_AS: 2858996 kB VmallocTotal: 34359738367 kB VmallocUsed: 328028 kB VmallocChunk: 34359404252 kB HardwareCorrupted: 0 kB AnonHugePages: 106496 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 59728 kB DirectMap2M: 25104384 kB
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Tuesday, January 27, 2015 5:18 PM To: Observium Network Observation System Subject: Re: [Observium] webpage no longer loads or takes forever
Nice that you told us what ram and CPU you have. Less nice that you didn't tell us what storage you're using.
Thus, insufficient information...
Adam.
Sent with AquaMail for Android http://www.aqua-mail.com
On 27 January 2015 13:56:17 Jerome Jenkins <JJENKINS@wpahs.orgmailto:JJENKINS@wpahs.org> wrote: Running on Ubuntu Server LTS 14.0.1
Server is 8 cores with 24GB of memory
Was working perfectly during initial install of the software. Once I let it discover all of our routers and switches (approx. 1200 devices) things went downhill. Login to the cli of the server takes very long and most times will time-out. Webpage never loads. Only way to fix is to reboot the server and once it begins polling everything again...nothing is accessible. I changed the setting for php to use 3GB of memory and change the poller instances per the performance tuning guide for observium to
*/5 * * * * root /opt/observium/poller-wrapper.py 10 >> /dev/null 2>&1
Seems that the memory and processor are probably being consumed by the poller?
Jerome S. Jenkins, Sr. Sr. Network Engineer jjenkins@wpahs.orgmailto:jjenkins@wpahs.org
*************************************************************************
Confidentiality Notice: This e-mail message and any attachments are
intended solely for the individual or individuals designated above. It may
contain confidential or proprietary information and may be subject to
confidentiality protections. If you are not the intended recipient, you
are not authorized to read, copy, retain or distribute this message. If
you receive this message in error, please notify the sender by reply e-mail
and delete this message. The sender does not waive any privilege or right
of privacy or confidentiality that may attach to this communication. Thank
you.
*************************************************************************
_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
************************************************************************* Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you. *************************************************************************
Jerome,
I ran into this problem too. I think it is common for anyone with a few hundred devices, using the default MYSQL settings.
Edit "/etc/mysql/my.cnf " and add this setting:
max_heap_table_size = 512M
I was told the default is 16M, so you can pick any larger number you want (512M might be over-kill).
Cheers,
Tristan
*Tristan Rhodes* Network Engineer Weber State University 801.626.8549
On Thu, Jan 29, 2015 at 7:37 AM, Jerome Jenkins JJENKINS@wpahs.org wrote:
I was able to gather the stats shortly after rebooting by holding the power button. After running for about 30mins now….the webpage now gives the error:
“MySQL Error: Too many connections”
*From:* Jerome Jenkins *Sent:* Thursday, January 29, 2015 9:10 AM *To:* Observium Network Observation System *Subject:* RE: [Observium] webpage no longer loads or takes forever
Appologies for lack of info:
Shows total of 1027 devices, NOT 1200 (original list was 1200 but not all of it imported)
Observium is stored on /opt (which is a raid-array of (6) 146GB drives)
root@observium:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/root-root 9.7G 2.4G 6.8G 27% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 12G 4.0K 12G 1% /dev
tmpfs 2.4G 612K 2.4G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 12G 0 12G 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/mapper/observium--data-data 684G 195G 455G 30% /opt
(2) Quad-core processors:
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU L5420 @ 2.50GHz
stepping : 10
microcode : 0xa07
cpu MHz : 2499.803
cache size : 6144 KB
Meminfo:
MemTotal: 24689672 kB
MemFree: 19383548 kB
Buffers: 124572 kB
Cached: 3489096 kB
SwapCached: 0 kB
Active: 1476312 kB
Inactive: 3352276 kB
Active(anon): 1220624 kB
Inactive(anon): 23092 kB
Active(file): 255688 kB
Inactive(file): 3329184 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 25161724 kB
SwapFree: 25161724 kB
Dirty: 353976 kB
Writeback: 2704 kB
AnonPages: 1214728 kB
Mapped: 56804 kB
Shmem: 28796 kB
Slab: 210032 kB
SReclaimable: 178204 kB
SUnreclaim: 31828 kB
KernelStack: 3552 kB
PageTables: 30896 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 37506560 kB
Committed_AS: 2858996 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 328028 kB
VmallocChunk: 34359404252 kB
HardwareCorrupted: 0 kB
AnonHugePages: 106496 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 59728 kB
DirectMap2M: 25104384 kB
*From:* observium [mailto:observium-bounces@observium.org observium-bounces@observium.org] *On Behalf Of *Adam Armstrong *Sent:* Tuesday, January 27, 2015 5:18 PM *To:* Observium Network Observation System *Subject:* Re: [Observium] webpage no longer loads or takes forever
Nice that you told us what ram and CPU you have. Less nice that you didn't tell us what storage you're using.
Thus, insufficient information...
Adam.
Sent with AquaMail for Android http://www.aqua-mail.com
On 27 January 2015 13:56:17 Jerome Jenkins JJENKINS@wpahs.org wrote:
Running on Ubuntu Server LTS 14.0.1
Server is 8 cores with 24GB of memory
Was working perfectly during initial install of the software. Once I let it discover all of our routers and switches (approx. 1200 devices) things went downhill. Login to the cli of the server takes very long and most times will time-out. Webpage never loads. Only way to fix is to reboot the server and once it begins polling everything again…nothing is accessible. I changed the setting for php to use 3GB of memory and change the poller instances per the performance tuning guide for observium to
*/5 * * * * root /opt/observium/poller-wrapper.py 10 >> /dev/null 2>&1
Seems that the memory and processor are probably being consumed by the poller?
*Jerome S. Jenkins, Sr.*
Sr. Network Engineer
*jjenkins@wpahs.org jjenkins@wpahs.org*
Confidentiality Notice: This e-mail message and any attachments are
intended solely for the individual or individuals designated above. It may
contain confidential or proprietary information and may be subject to
confidentiality protections. If you are not the intended recipient, you
are not authorized to read, copy, retain or distribute this message. If
you receive this message in error, please notify the sender by reply e-mail
and delete this message. The sender does not waive any privilege or right
of privacy or confidentiality that may attach to this communication. Thank
you.
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you.
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Sorry! I was mixing this up with another error. The heap size fix above is for when you search "/var/log/mysql/error.log" and find:
[ERROR] /usr/sbin/mysqld: The table 'alert_table-state' is full
On the other hand, if you get the “MySQL Error: Too many connections” in the web-page, then you need to increase the number of connections in "/etc/mysql/my.cnf":
max_connections = 500 (I think the default is 151)
I will try to add these common issues to the Observium wiki.
Good luck!
*Tristan Rhodes* Network Engineer Weber State University 801.626.8549
On Thu, Jan 29, 2015 at 9:01 AM, Tristan Rhodes tristanrhodes@weber.edu wrote:
Jerome,
I ran into this problem too. I think it is common for anyone with a few hundred devices, using the default MYSQL settings.
Edit "/etc/mysql/my.cnf " and add this setting:
max_heap_table_size = 512M
I was told the default is 16M, so you can pick any larger number you want (512M might be over-kill).
Cheers,
Tristan
*Tristan Rhodes* Network Engineer Weber State University 801.626.8549
On Thu, Jan 29, 2015 at 7:37 AM, Jerome Jenkins JJENKINS@wpahs.org wrote:
I was able to gather the stats shortly after rebooting by holding the power button. After running for about 30mins now….the webpage now gives the error:
“MySQL Error: Too many connections”
*From:* Jerome Jenkins *Sent:* Thursday, January 29, 2015 9:10 AM *To:* Observium Network Observation System *Subject:* RE: [Observium] webpage no longer loads or takes forever
Appologies for lack of info:
Shows total of 1027 devices, NOT 1200 (original list was 1200 but not all of it imported)
Observium is stored on /opt (which is a raid-array of (6) 146GB drives)
root@observium:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/root-root 9.7G 2.4G 6.8G 27% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 12G 4.0K 12G 1% /dev
tmpfs 2.4G 612K 2.4G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 12G 0 12G 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/mapper/observium--data-data 684G 195G 455G 30% /opt
(2) Quad-core processors:
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU L5420 @ 2.50GHz
stepping : 10
microcode : 0xa07
cpu MHz : 2499.803
cache size : 6144 KB
Meminfo:
MemTotal: 24689672 kB
MemFree: 19383548 kB
Buffers: 124572 kB
Cached: 3489096 kB
SwapCached: 0 kB
Active: 1476312 kB
Inactive: 3352276 kB
Active(anon): 1220624 kB
Inactive(anon): 23092 kB
Active(file): 255688 kB
Inactive(file): 3329184 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 25161724 kB
SwapFree: 25161724 kB
Dirty: 353976 kB
Writeback: 2704 kB
AnonPages: 1214728 kB
Mapped: 56804 kB
Shmem: 28796 kB
Slab: 210032 kB
SReclaimable: 178204 kB
SUnreclaim: 31828 kB
KernelStack: 3552 kB
PageTables: 30896 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 37506560 kB
Committed_AS: 2858996 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 328028 kB
VmallocChunk: 34359404252 kB
HardwareCorrupted: 0 kB
AnonHugePages: 106496 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 59728 kB
DirectMap2M: 25104384 kB
*From:* observium [mailto:observium-bounces@observium.org observium-bounces@observium.org] *On Behalf Of *Adam Armstrong *Sent:* Tuesday, January 27, 2015 5:18 PM *To:* Observium Network Observation System *Subject:* Re: [Observium] webpage no longer loads or takes forever
Nice that you told us what ram and CPU you have. Less nice that you didn't tell us what storage you're using.
Thus, insufficient information...
Adam.
Sent with AquaMail for Android http://www.aqua-mail.com
On 27 January 2015 13:56:17 Jerome Jenkins JJENKINS@wpahs.org wrote:
Running on Ubuntu Server LTS 14.0.1
Server is 8 cores with 24GB of memory
Was working perfectly during initial install of the software. Once I let it discover all of our routers and switches (approx. 1200 devices) things went downhill. Login to the cli of the server takes very long and most times will time-out. Webpage never loads. Only way to fix is to reboot the server and once it begins polling everything again…nothing is accessible. I changed the setting for php to use 3GB of memory and change the poller instances per the performance tuning guide for observium to
*/5 * * * * root /opt/observium/poller-wrapper.py 10 >> /dev/null 2>&1
Seems that the memory and processor are probably being consumed by the poller?
*Jerome S. Jenkins, Sr.*
Sr. Network Engineer
*jjenkins@wpahs.org jjenkins@wpahs.org*
Confidentiality Notice: This e-mail message and any attachments are
intended solely for the individual or individuals designated above. It may
contain confidential or proprietary information and may be subject to
confidentiality protections. If you are not the intended recipient, you
are not authorized to read, copy, retain or distribute this message. If
you receive this message in error, please notify the sender by reply e-mail
and delete this message. The sender does not waive any privilege or right
of privacy or confidentiality that may attach to this communication. Thank
you.
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you.
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Thanks. Looks like my max_connections setting is commented out. I’ll uncomment it and change it to 500 and see if that helps.
Below are the settings in the Fine Tuning section of the my.cnf on my box:
# * Fine Tuning # key_buffer = 16M max_allowed_packet = 16M thread_stack = 192K thread_cache_size = 8 # This replaces the startup script and checks MyISAM tables if needed # the first time they are touched myisam-recover = BACKUP #max_connections = 100 #table_cache = 64 #thread_concurrency = 10 # # * Query Cache Configuration # query_cache_limit = 1M query_cache_size = 16M
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Tristan Rhodes Sent: Thursday, January 29, 2015 11:21 AM To: Observium Network Observation System Subject: Re: [Observium] webpage no longer loads or takes forever
Sorry! I was mixing this up with another error. The heap size fix above is for when you search "/var/log/mysql/error.log" and find:
[ERROR] /usr/sbin/mysqld: The table 'alert_table-state' is full
On the other hand, if you get the “MySQL Error: Too many connections” in the web-page, then you need to increase the number of connections in "/etc/mysql/my.cnf":
max_connections = 500 (I think the default is 151)
I will try to add these common issues to the Observium wiki.
Good luck!
Tristan Rhodes Network Engineer Weber State University 801.626.8549 [http://www.weber.edu/wsuimages/brand/templates/emailsig_sig1.jpg]
On Thu, Jan 29, 2015 at 9:01 AM, Tristan Rhodes <tristanrhodes@weber.edumailto:tristanrhodes@weber.edu> wrote: Jerome,
I ran into this problem too. I think it is common for anyone with a few hundred devices, using the default MYSQL settings.
Edit "/etc/mysql/my.cnf " and add this setting:
max_heap_table_size = 512M
I was told the default is 16M, so you can pick any larger number you want (512M might be over-kill).
Cheers,
Tristan
Tristan Rhodes Network Engineer Weber State University 801.626.8549tel:801.626.8549 [http://www.weber.edu/wsuimages/brand/templates/emailsig_sig1.jpg]
On Thu, Jan 29, 2015 at 7:37 AM, Jerome Jenkins <JJENKINS@wpahs.orgmailto:JJENKINS@wpahs.org> wrote: I was able to gather the stats shortly after rebooting by holding the power button. After running for about 30mins now….the webpage now gives the error:
“MySQL Error: Too many connections”
From: Jerome Jenkins Sent: Thursday, January 29, 2015 9:10 AM To: Observium Network Observation System Subject: RE: [Observium] webpage no longer loads or takes forever
Appologies for lack of info:
Shows total of 1027 devices, NOT 1200 (original list was 1200 but not all of it imported)
Observium is stored on /opt (which is a raid-array of (6) 146GB drives)
root@observium:~# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/root-root 9.7G 2.4G 6.8G 27% / none 4.0K 0 4.0K 0% /sys/fs/cgroup udev 12G 4.0K 12G 1% /dev tmpfs 2.4G 612K 2.4G 1% /run none 5.0M 0 5.0M 0% /run/lock none 12G 0 12G 0% /run/shm none 100M 0 100M 0% /run/user /dev/mapper/observium--data-data 684G 195G 455G 30% /opt
(2) Quad-core processors:
cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU L5420 @ 2.50GHz stepping : 10 microcode : 0xa07 cpu MHz : 2499.803 cache size : 6144 KB
Meminfo:
MemTotal: 24689672 kB MemFree: 19383548 kB Buffers: 124572 kB Cached: 3489096 kB SwapCached: 0 kB Active: 1476312 kB Inactive: 3352276 kB Active(anon): 1220624 kB Inactive(anon): 23092 kB Active(file): 255688 kB Inactive(file): 3329184 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 25161724 kB SwapFree: 25161724 kB Dirty: 353976 kB Writeback: 2704 kB AnonPages: 1214728 kB Mapped: 56804 kB Shmem: 28796 kB Slab: 210032 kB SReclaimable: 178204 kB SUnreclaim: 31828 kB KernelStack: 3552 kB PageTables: 30896 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 37506560 kB Committed_AS: 2858996 kB VmallocTotal: 34359738367 kB VmallocUsed: 328028 kB VmallocChunk: 34359404252 kB HardwareCorrupted: 0 kB AnonHugePages: 106496 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 59728 kB DirectMap2M: 25104384 kB
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Tuesday, January 27, 2015 5:18 PM To: Observium Network Observation System Subject: Re: [Observium] webpage no longer loads or takes forever
Nice that you told us what ram and CPU you have. Less nice that you didn't tell us what storage you're using.
Thus, insufficient information...
Adam.
Sent with AquaMail for Android http://www.aqua-mail.com
On 27 January 2015 13:56:17 Jerome Jenkins <JJENKINS@wpahs.orgmailto:JJENKINS@wpahs.org> wrote: Running on Ubuntu Server LTS 14.0.1
Server is 8 cores with 24GB of memory
Was working perfectly during initial install of the software. Once I let it discover all of our routers and switches (approx. 1200 devices) things went downhill. Login to the cli of the server takes very long and most times will time-out. Webpage never loads. Only way to fix is to reboot the server and once it begins polling everything again…nothing is accessible. I changed the setting for php to use 3GB of memory and change the poller instances per the performance tuning guide for observium to
*/5 * * * * root /opt/observium/poller-wrapper.py 10 >> /dev/null 2>&1
Seems that the memory and processor are probably being consumed by the poller?
Jerome S. Jenkins, Sr. Sr. Network Engineer jjenkins@wpahs.orgmailto:jjenkins@wpahs.org
*************************************************************************
Confidentiality Notice: This e-mail message and any attachments are
intended solely for the individual or individuals designated above. It may
contain confidential or proprietary information and may be subject to
confidentiality protections. If you are not the intended recipient, you
are not authorized to read, copy, retain or distribute this message. If
you receive this message in error, please notify the sender by reply e-mail
and delete this message. The sender does not waive any privilege or right
of privacy or confidentiality that may attach to this communication. Thank
you.
*************************************************************************
_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
*************************************************************************
Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you.
*************************************************************************
_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
************************************************************************* Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you. *************************************************************************
I had this same issue once we upped our device count to roughly 600 devices. Edited the my.cnf with
max_connections = 512
And restarted, and no more problems.
Agree with Tristan, that’s something that needs to be added to the wiki.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Jerome Jenkins Sent: Thursday, January 29, 2015 10:26 AM To: Observium Network Observation System Subject: Re: [Observium] webpage no longer loads or takes forever
Thanks. Looks like my max_connections setting is commented out. I’ll uncomment it and change it to 500 and see if that helps.
Below are the settings in the Fine Tuning section of the my.cnf on my box:
# * Fine Tuning
#
key_buffer = 16M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover = BACKUP
#max_connections = 100
#table_cache = 64
#thread_concurrency = 10
#
# * Query Cache Configuration
#
query_cache_limit = 1M
query_cache_size = 16M
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Tristan Rhodes Sent: Thursday, January 29, 2015 11:21 AM To: Observium Network Observation System Subject: Re: [Observium] webpage no longer loads or takes forever
Sorry! I was mixing this up with another error. The heap size fix above is for when you search "/var/log/mysql/error.log" and find:
[ERROR] /usr/sbin/mysqld: The table 'alert_table-state' is full
On the other hand, if you get the “MySQL Error: Too many connections” in the web-page, then you need to increase the number of connections in "/etc/mysql/my.cnf":
max_connections = 500
(I think the default is 151)
I will try to add these common issues to the Observium wiki.
Good luck!
Tristan Rhodes Network Engineer
Weber State University
801.626.8549
http://www.weber.edu/wsuimages/brand/templates/emailsig_sig1.jpg
On Thu, Jan 29, 2015 at 9:01 AM, Tristan Rhodes tristanrhodes@weber.edu wrote:
Jerome,
I ran into this problem too. I think it is common for anyone with a few hundred devices, using the default MYSQL settings.
Edit "/etc/mysql/my.cnf " and add this setting:
max_heap_table_size = 512M
I was told the default is 16M, so you can pick any larger number you want (512M might be over-kill).
Cheers,
Tristan
Tristan Rhodes Network Engineer
Weber State University
801.626.8549
http://www.weber.edu/wsuimages/brand/templates/emailsig_sig1.jpg
On Thu, Jan 29, 2015 at 7:37 AM, Jerome Jenkins JJENKINS@wpahs.org wrote:
I was able to gather the stats shortly after rebooting by holding the power button. After running for about 30mins now….the webpage now gives the error:
“MySQL Error: Too many connections”
From: Jerome Jenkins Sent: Thursday, January 29, 2015 9:10 AM To: Observium Network Observation System Subject: RE: [Observium] webpage no longer loads or takes forever
Appologies for lack of info:
Shows total of 1027 devices, NOT 1200 (original list was 1200 but not all of it imported)
Observium is stored on /opt (which is a raid-array of (6) 146GB drives)
root@observium:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/root-root 9.7G 2.4G 6.8G 27% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 12G 4.0K 12G 1% /dev
tmpfs 2.4G 612K 2.4G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 12G 0 12G 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/mapper/observium--data-data 684G 195G 455G 30% /opt
(2) Quad-core processors:
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU L5420 @ 2.50GHz
stepping : 10
microcode : 0xa07
cpu MHz : 2499.803
cache size : 6144 KB
Meminfo:
MemTotal: 24689672 kB
MemFree: 19383548 kB
Buffers: 124572 kB
Cached: 3489096 kB
SwapCached: 0 kB
Active: 1476312 kB
Inactive: 3352276 kB
Active(anon): 1220624 kB
Inactive(anon): 23092 kB
Active(file): 255688 kB
Inactive(file): 3329184 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 25161724 kB
SwapFree: 25161724 kB
Dirty: 353976 kB
Writeback: 2704 kB
AnonPages: 1214728 kB
Mapped: 56804 kB
Shmem: 28796 kB
Slab: 210032 kB
SReclaimable: 178204 kB
SUnreclaim: 31828 kB
KernelStack: 3552 kB
PageTables: 30896 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 37506560 kB
Committed_AS: 2858996 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 328028 kB
VmallocChunk: 34359404252 kB
HardwareCorrupted: 0 kB
AnonHugePages: 106496 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 59728 kB
DirectMap2M: 25104384 kB
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Tuesday, January 27, 2015 5:18 PM To: Observium Network Observation System Subject: Re: [Observium] webpage no longer loads or takes forever
Nice that you told us what ram and CPU you have. Less nice that you didn't tell us what storage you're using.
Thus, insufficient information...
Adam.
Sent with AquaMail for Android http://www.aqua-mail.com
On 27 January 2015 13:56:17 Jerome Jenkins JJENKINS@wpahs.org wrote:
Running on Ubuntu Server LTS 14.0.1
Server is 8 cores with 24GB of memory
Was working perfectly during initial install of the software. Once I let it discover all of our routers and switches (approx. 1200 devices) things went downhill. Login to the cli of the server takes very long and most times will time-out. Webpage never loads. Only way to fix is to reboot the server and once it begins polling everything again…nothing is accessible. I changed the setting for php to use 3GB of memory and change the poller instances per the performance tuning guide for observium to
*/5 * * * * root /opt/observium/poller-wrapper.py 10 >> /dev/null 2>&1
Seems that the memory and processor are probably being consumed by the poller?
Jerome S. Jenkins, Sr.
Sr. Network Engineer
jjenkins@wpahs.org
************************************************************************* Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you. *************************************************************************
_______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
************************************************************************* Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you. *************************************************************************
_______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
************************************************************************* Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you. *************************************************************************
Done!
http://observium.org/wiki/FAQs#I_am_getting_a_.22MySQL_Error:_Too_many_conne...
*Tristan Rhodes* Network Engineer Weber State University 801.626.8549
On Thu, Jan 29, 2015 at 9:37 AM, Ron Marosko ron@rjr-services.com wrote:
I had this same issue once we upped our device count to roughly 600 devices. Edited the my.cnf with
max_connections = 512
And restarted, and no more problems.
Agree with Tristan, that’s something that needs to be added to the wiki.
*From:* observium [mailto:observium-bounces@observium.org] *On Behalf Of *Jerome Jenkins *Sent:* Thursday, January 29, 2015 10:26 AM
*To:* Observium Network Observation System *Subject:* Re: [Observium] webpage no longer loads or takes forever
Thanks. Looks like my max_connections setting is commented out. I’ll uncomment it and change it to 500 and see if that helps.
Below are the settings in the Fine Tuning section of the my.cnf on my box:
# * Fine Tuning
#
key_buffer = 16M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 8
# This replaces the startup script and checks MyISAM tables if needed
# the first time they are touched
myisam-recover = BACKUP
#max_connections = 100
#table_cache = 64
#thread_concurrency = 10
#
# * Query Cache Configuration
#
query_cache_limit = 1M
query_cache_size = 16M
*From:* observium [mailto:observium-bounces@observium.org] *On Behalf Of *Tristan Rhodes *Sent:* Thursday, January 29, 2015 11:21 AM *To:* Observium Network Observation System *Subject:* Re: [Observium] webpage no longer loads or takes forever
Sorry! I was mixing this up with another error. The heap size fix above is for when you search "/var/log/mysql/error.log" and find:
[ERROR] /usr/sbin/mysqld: The table 'alert_table-state' is full
On the other hand, if you get the “MySQL Error: Too many connections” in the web-page, then you need to increase the number of connections in "/etc/mysql/my.cnf":
max_connections = 500
(I think the default is 151)
I will try to add these common issues to the Observium wiki.
Good luck!
*Tristan Rhodes* Network Engineer
Weber State University
801.626.8549
On Thu, Jan 29, 2015 at 9:01 AM, Tristan Rhodes tristanrhodes@weber.edu wrote:
Jerome,
I ran into this problem too. I think it is common for anyone with a few hundred devices, using the default MYSQL settings.
Edit "/etc/mysql/my.cnf " and add this setting:
max_heap_table_size = 512M
I was told the default is 16M, so you can pick any larger number you want (512M might be over-kill).
Cheers,
Tristan
*Tristan Rhodes* Network Engineer
Weber State University
801.626.8549
On Thu, Jan 29, 2015 at 7:37 AM, Jerome Jenkins JJENKINS@wpahs.org wrote:
I was able to gather the stats shortly after rebooting by holding the power button. After running for about 30mins now….the webpage now gives the error:
“MySQL Error: Too many connections”
*From:* Jerome Jenkins *Sent:* Thursday, January 29, 2015 9:10 AM *To:* Observium Network Observation System *Subject:* RE: [Observium] webpage no longer loads or takes forever
Appologies for lack of info:
Shows total of 1027 devices, NOT 1200 (original list was 1200 but not all of it imported)
Observium is stored on /opt (which is a raid-array of (6) 146GB drives)
root@observium:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/root-root 9.7G 2.4G 6.8G 27% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 12G 4.0K 12G 1% /dev
tmpfs 2.4G 612K 2.4G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 12G 0 12G 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/mapper/observium--data-data 684G 195G 455G 30% /opt
(2) Quad-core processors:
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU L5420 @ 2.50GHz
stepping : 10
microcode : 0xa07
cpu MHz : 2499.803
cache size : 6144 KB
Meminfo:
MemTotal: 24689672 kB
MemFree: 19383548 kB
Buffers: 124572 kB
Cached: 3489096 kB
SwapCached: 0 kB
Active: 1476312 kB
Inactive: 3352276 kB
Active(anon): 1220624 kB
Inactive(anon): 23092 kB
Active(file): 255688 kB
Inactive(file): 3329184 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 25161724 kB
SwapFree: 25161724 kB
Dirty: 353976 kB
Writeback: 2704 kB
AnonPages: 1214728 kB
Mapped: 56804 kB
Shmem: 28796 kB
Slab: 210032 kB
SReclaimable: 178204 kB
SUnreclaim: 31828 kB
KernelStack: 3552 kB
PageTables: 30896 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 37506560 kB
Committed_AS: 2858996 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 328028 kB
VmallocChunk: 34359404252 kB
HardwareCorrupted: 0 kB
AnonHugePages: 106496 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 59728 kB
DirectMap2M: 25104384 kB
*From:* observium [mailto:observium-bounces@observium.org observium-bounces@observium.org] *On Behalf Of *Adam Armstrong *Sent:* Tuesday, January 27, 2015 5:18 PM *To:* Observium Network Observation System *Subject:* Re: [Observium] webpage no longer loads or takes forever
Nice that you told us what ram and CPU you have. Less nice that you didn't tell us what storage you're using.
Thus, insufficient information...
Adam.
Sent with AquaMail for Android http://www.aqua-mail.com
On 27 January 2015 13:56:17 Jerome Jenkins JJENKINS@wpahs.org wrote:
Running on Ubuntu Server LTS 14.0.1
Server is 8 cores with 24GB of memory
Was working perfectly during initial install of the software. Once I let it discover all of our routers and switches (approx. 1200 devices) things went downhill. Login to the cli of the server takes very long and most times will time-out. Webpage never loads. Only way to fix is to reboot the server and once it begins polling everything again…nothing is accessible. I changed the setting for php to use 3GB of memory and change the poller instances per the performance tuning guide for observium to
*/5 * * * * root /opt/observium/poller-wrapper.py 10 >> /dev/null 2>&1
Seems that the memory and processor are probably being consumed by the poller?
*Jerome S. Jenkins, Sr.*
Sr. Network Engineer
*jjenkins@wpahs.org jjenkins@wpahs.org*
Confidentiality Notice: This e-mail message and any attachments are
intended solely for the individual or individuals designated above. It may
contain confidential or proprietary information and may be subject to
confidentiality protections. If you are not the intended recipient, you
are not authorized to read, copy, retain or distribute this message. If
you receive this message in error, please notify the sender by reply e-mail
and delete this message. The sender does not waive any privilege or right
of privacy or confidentiality that may attach to this communication. Thank
you.
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you.
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Confidentiality Notice: This e-mail message and any attachments are intended solely for the individual or individuals designated above. It may contain confidential or proprietary information and may be subject to confidentiality protections. If you are not the intended recipient, you are not authorized to read, copy, retain or distribute this message. If you receive this message in error, please notify the sender by reply e-mail and delete this message. The sender does not waive any privilege or right of privacy or confidentiality that may attach to this communication. Thank you.
-- This message has been scanned for viruses and dangerous content by *E.F.A. Project* http://www.efa-project.org, and is believed to be clean. Click here to report this message as spam. http://smtp.rjr-services.com/cgi-bin/learn-msg.cgi?id=7808B604CC.A41F0&token=1e4038420555cae1179341165fb67e8c
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
participants (6)
-
Adam Armstrong -
Jerome Jenkins -
Laurent Dumont -
Markus Klock -
Ron Marosko -
Tristan Rhodes