Check /var/log and see what is in there that is usually what fills stuff up.

-- 
Ray Poorman, CTO
CSS Integration & Communications
1911 C St
Bellingham WA 98225
360-671-4878

On March 5, 2020 at 12:15:13 PM, Gordon Cheng (gocheng) via observium (observium@observium.org) wrote:

Hi all,

 

Disk space is full on our Observium and we lost the GUI access (only CLI).

 

I’m trying to use “housekeeping.php -a” to clean up the space, and use “delete_device.php” to delete a few offending devices using up lots of space.

 

But it didn’t help much yet.  For example, even if I tried to delete a device “phx-nx5596-04”:

 

sjc-observium-1:/opt/observium# ./delete_device.php phx-nx5596-04

Observium 19.5.9872

Remove Device

 

CACHE CLEAR SET. Cache clear set.

 

* Deleted interfaces: id=13437 (mgmt0), id=13438 (Vlan1), id=13439 (Vlan2283), id=13440 (port-channel1), id=13441 (port-channel2), id=13442 (port-channel5), id=13443 (port-channel6), id=13444 (port-channel7), id=13445 (port-channel8), id=13446 (port-channel10), id=13447 (port-channel12), id=13448 (port-channel13), id=13449 (port-channel14), id=13450 (port-channel15),

<snip>

64898 (), id=64899 (), id=64900 ()

* Deleted common entity entries linked to device: device

* Deleted device entries from tables: devices_locations, device_graphs, entPhysical

* Deleted device: phx-nx5596-04

Device phx-nx5596-04 removed.

sjc-observium-1:/opt/observium#

 

It’s still showing up undeleted and using a lot of disk space:

 

sjc-observium-1:/opt/observium/rrd# du -sk * | sort -nr | head

2276640 phx-core03

2256244 phx-core04

2203544 sjc-core03

2199116 phx-spider01

2196748 sjc-core04

2169864 sjc-spider01-old

1885692 phx-spider02

1873348 phx-spider01-new

1862500 sjc-spider01

1452708 phx-nx5596-04

sjc-observium-1:/opt/observium/rrd#

sjc-observium-1:/opt/observium/rrd# df -h

Filesystem      Size  Used Avail Use% Mounted on

/dev/vda3       378G  359G     0 100% /

devtmpfs         32G     0   32G   0% /dev

tmpfs            32G     0   32G   0% /dev/shm

tmpfs            32G  8.7M   32G   1% /run

tmpfs            32G     0   32G   0% /sys/fs/cgroup

/dev/vda1       488M  189M  264M  42% /boot

tmpfs           6.3G     0  6.3G   0% /run/user/15115

tmpfs           6.3G     0  6.3G   0% /run/user/0

sjc-observium-1:/opt/observium/rrd#

 

Any suggestions how we can fix and restore it?

 

Thanks.

 

- Gordon

_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium