Hi Team,

 

I have a performance issue with the web interface.

My Observium is monitoring 3095 devices from which 168 are switches (Just a bit less than 20000 ports total).

 

My server is an IBM X3650 M4, 32G of ram, 6 cores  and a raid 10 of SSD.

I am more than happy with the polling performance has it finishing in under 200seconds.

 

But the web interface is quite annoying : I have a minimum Time to first bit of ~2seconds.

Once these two seconds have passed the page load almost instantly, graphs included.

 

So the question is, can I speed things up ?

And if yes how ?

 

It looks like Observium is spending about a second in the database and says that the Cache took 2 seconds to process (?).

Do I have some leverage on these points to make it faster ?

 

 

Page load time (Just a device page) :

 

Connection Setup                            TIME

Queueing                                           5.24 ms

Stalled                                                1.26 ms

Request sent                                     0.80 ms

Waiting (TTFB)                                 3.29 s

Content Download                          26.62 ms

Explanation                                       3.33 s

 

Observium Performance report :

 

Page	2.281s
Cache	2.047s

 

MySQL
Cell	50/0.0633s
Row	35/0.0113s
Rows	49/0.7822s
Column	8/0.0564s

 

Memory
Cached	180MB
Page	189MB
Peak	229MB

 

Versions:

 

Observium	0.16.1.7535 (27th January 2016)
OS	Linux 3.10.0-327.el7.x86_64 [amd64] (CentOS 7.2.1511)
Apache	2.4.6 (CentOS) PHP/5.4.16
PHP	5.4.16
Python	2.7.5
MySQL	5.5.44-MariaDB-log (extension: mysqli 5.5.44-MariaDB)
SNMP	NET-SNMP 5.7.2
RRDtool	1.4.8

 

 

Stats :

 

Statistics

DB size	606MB	RRD size	134GB
Devices	3095	Ports	19390
IPv4 Addresses	6148	IPv4 Networks	44
IPv6 Addresses	2959	IPv6 Networks	1
Services	0	Applications	0
Processors	61262	Memory	15563
Storage	11742	Disk I/O	97461
HR-MIB	96815	Entity-MIB	7292
Syslog Entries	0	Eventlog Entries	172938
Sensors	3863	Toner

 

 

Mysqltuner output  :

 

-------- Performance Metrics -------------------------------------------------

[--] Up for: 2d 21h 57m 9s (260M q [1K qps], 5K conn, TX: 283B, RX: 45B)

[--] Reads / Writes: 59% / 41%

[--] Binary logging is disabled

[--] Total buffers: 10.5G global + 4.7M per thread (650 max threads)

[OK] Maximum reached memory usage: 11.2G (35.94% of installed RAM)

[OK] Maximum possible memory usage: 13.5G (43.37% of installed RAM)

[OK] Slow queries: 0% (14K/260M)

[OK] Highest usage of available connections: 21% (142/650)

[OK] Aborted connections: 0.00%  (0/5477)

[OK] Query cache efficiency: 32.3% (74M cached / 230M selects)

[OK] Query cache prunes per day: 0

[OK] Sorts requiring temporary tables: 0% (129 temp sorts / 182K sorts)

[OK] Temporary tables created on disk: 0% (201 on disk / 112K total)

[OK] Thread cache hit rate: 97% (142 created / 5K connections)

[OK] Table cache hit rate: 102% (922 open / 896 opened)

[OK] Open file limit used: 5% (52/1K)

[OK] Table locks acquired immediately: 97% (244M immediate / 249M locks)

 

-------- MyISAM Metrics -----------------------------------------------------

[!!] Key buffer used: 18.0% (28M used / 157M cache)

[OK] Key buffer size / total MyISAM indexes: 150.0M/99.0K

[!!] Read Key buffer hit rate: 93.3% (15 cached / 1 reads)

 

-------- InnoDB Metrics -----------------------------------------------------

[--] InnoDB is enabled.

[OK] InnoDB buffer pool / data size: 8.0G/579.1M

[!!] InnoDB buffer pool instances: 1

[!!] InnoDB Used buffer: 6.67% (34952 used/ 524287 total)

[OK] InnoDB Read buffer efficiency: 100.00% (2079385726 hits/ 2079395120 total)

[!!] InnoDB Write buffer efficiency: 0.00% (0 hits/ 1 total)

[OK] InnoDB log waits: 0.00% (0 waits / 11253445 writes)

 

-------- AriaDB Metrics -----------------------------------------------------

[--] AriaDB is disabled.

 

-------- Replication Metrics -------------------------------------------------

[--] No replication slave(s) for this server.

[--] This is a standalone server..

 

-------- Recommendations -----------------------------------------------------

General recommendations:

    Run OPTIMIZE TABLE to defragment tables for better performance

    Remove Anonymous User accounts - there are 2 anonymous accounts.

    Set up a Password for user with the following SQL statement ( SET PASSWORD FOR 'user'@'SpecificDNSorIp' = PASSWORD('secure_password'); )

    Set up a Secure Password for user@host ( SET PASSWORD FOR 'user'@'SpecificDNSorIp' = PASSWORD('secure_password'); )

    Restrict Host for user@% to user@SpecificDNSorIp

Variables to adjust:

    innodb_buffer_pool_instances(=8)

-------------------------------------------------------------------------------------

 

 

 

Thanks.

 

Best regards,

Louis