as i managed to see the type of that files, so yeah, they're all alert-*.rrd ones. The housekeeping scripts didn't delete them (because i assume the scripts sees them as the more recent than the retention time)

infra [root@observ001prvitx rrd]# find jaynet01b -type f | sort | uniq -c | awk '{print $2}' | awk -F"/" '{print $2}' | awk -F"-" '{print $1}' | sort | uniq -c  | sort
    938 port
 993164 alert

i think i will try to delete them. Btw, how to know which interfaces caused them to be recreated to be able to disable ? i'm attaching the graph which shows how the space was increasing in 4months period:
image.png

On Wed, Nov 6, 2019 at 5:22 PM Adam Armstrong <adama@memetic.org> wrote:
They're just the availability data for an alert entry.

If your device is creating lots of alert entries, I assume they're port entries and it's creating and destroying lots of temporary interfaces for some reason.

If this is happening, you should block these interfaces from being discovered by filtering them out using the relevant config options.

Adam.

Sent from BlueMail
On 6 Nov 2019, at 14:24, Edvinas Kairys via observium <observium@observium.org> wrote:
would i be safe to delete alert-*.rrd files ?

afaik these files are not related to interfaces traffic graph history ?

Thank you

On Wed, Nov 6, 2019 at 4:00 PM Edvinas Kairys < edvinas.email@gmail.com> wrote:
managed to see count of files:

so, in mirrored devices there are this count of files:

infra [root@observ001prvitx rrd]# find jaynet01a -type f | wc -l
13249

and in the problematic one:

infra [root@observ001prvitx rrd]# find jaynet01b -type f | wc -l
995349 






On Wed, Nov 6, 2019 at 3:53 PM Edvinas Kairys < edvinas.email@gmail.com> wrote:
thank you,

Something is very strange space in that folder started to increase rapidly ~3months ago.

i even can't to a list of files: of that particular folder.

Now i ran the housekeeping script separately in a screen mode, and it's already running ~3hours in that particular folder and still deleting items:

[2019/11/06 13:52:21 +0000] housekeeping.php(30530): File /opt/observium/rrd/jaynet01b/alert-6873343.rrd modification time is 2019-10-07 08:22:09 - deleting
[2019/11/06 13:52:24 +0000] housekeeping.php(30530): File /opt/observium/rrd/jaynet01b/alert-6871338.rrd modification time is 2019-10-07 07:21:54 - deleting
[2019/11/06 13:52:26 +0000] housekeeping.php(30530): File /opt/observium/rrd/jaynet01b/alert-6878336.rrd modification time is 2019-10-07 08:21:50 - deleting
[2019/11/06 13:52:28 +0000] housekeeping.php(30530): File /opt/observium/rrd/jaynet01b/alert-6877497.rrd modification time is 2019-10-07 08:22:02 - deleting
[2019/11/06 13:52:28 +0000] housekeeping.php(30530): File /opt/observium/rrd/jaynet01b/alert-6867753.rrd modification time is 2019-10-07 04:22:05 - deleting
[2019/11/06 13:52:29 +0000] housekeeping.php(30530): File /opt/observium/rrd/jaynet01b/alert-6871233.rrd modification time is 2019-10-07 07:21:48 - deleting
[2019/11/06 13:52:29 +0000] housekeeping.php(30530): File /opt/observium/rrd/jaynet01b/alert-6867316.rrd modification time is 2019-10-07 04:22:09 - deleting
[2019/11/06 13:52:31 +0000] housekeeping.php(30530): File /opt/observium/rrd/jaynet01b/alert-6873301.rrd modification time is 2019-10-07 08:22:06 - deleting
[2019/11/06 13:52:31 +0000] housekeeping.php(30530): File /opt/observium/rrd/jaynet01b/alert-6865505.rrd modification time is 2019-10-07 04:22:00 - deleting
[2019/11/06 13:52:32 +0000] housekeeping.php(30530): File /opt/observium/rrd/jaynet01b/alert-6873140.rrd modification time is 2019-10-07 08:21:58 - deleting
[2019/11/06 13:52:32 +0000] housekeeping.php(30530): File /opt/observium/rrd/jaynet01b/alert-6877296.rrd modification time is 2019-10-07 08:21:52 - deleting

etc, etc..




On Wed, Nov 6, 2019 at 1:25 PM Markus Klock < markus@best-practice.se> wrote:
Hey,
Well your housekeeping settings are to not delete RRDs unless they are not updated for 2 months.
Did this one device with 200GB RRDs have files older than 2 months?
If not then none of those RRDs will be deleted for now.
/Markus

Den ons 6 nov. 2019 kl 10:45 skrev Edvinas Kairys via observium < observium@observium.org>:
Hello,

I noticed that Observium started to use up lots of storage space. From ~200Gb it started to use ~400Gbs. Found interested thing, that one device RRD folder tooks ~200GBs instead of ~4-5Gbs. I'm always running housekeeping scripts in cron:

# Run housekeeping script daily for syslog, eventlog and alert log
13 5 * * * root /opt/observium/housekeeping.php -ysel >> /dev/null 2>&1

# Run housekeeping script daily for rrds, ports, orphaned entries in the database and performance data
47 4 * * * root /opt/observium/housekeeping.php -yrptib >> /dev/null 2>&1

But seems it doesnt help, and housekeeping log doesn't show errors.

Could someone advice how to clear that files in RRD folder according to housekeeping settings which are here:

infra [root@observ001prvitx logs]# cat /opt/observium/includes/defaults.inc.php  | grep housekeep

$config['housekeeping']['syslog']['age'] = '1M';         // Maximum age of syslog entries; 0 to disable
$config['housekeeping']['eventlog']['age'] = '3M';       // Maximum age of event log entries; 0 to disable
$config['housekeeping']['alertlog']['age'] = '3M';       // Maximum age of alert log entries; 0 to disable
$config['housekeeping']['authlog']['age'] = '6M';        // Maximum age of authlog entries; 0 to disable
$config['housekeeping']['inventory']['age'] = '1M';      // Maximum age of deleted inventory entries; 0 to disable
$config['housekeeping']['deleted_ports']['age'] = '1M';  // Maximum age of deleted ports before automatically purging; 0 to disable
$config['housekeeping']['rrd']['age'] = '2M';            // Maximum age of unused rrd files before automatically purging; 0 to disable
$config['housekeeping']['rrd']['invalid'] = TRUE;        // Delete .rrd files that are not valid RRD files (eg created with a full disk)

afaik M = month ?
_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium 
 

observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium