Hello, we're trying to implement observium and it is about 400 devices. At the moment there are 226 but 2 of them suddenly went offline and cannot be brought back up with polling and I'm not sure if the snmp gets blocked on out end or on the devices end.  There are also Palo Alto firewalls present at the location and I'm thinking they're causing the issue but just in case im posting this here also. Here is the ./poller with debug if it helps:
DEBUG!
DEFINITIONS Time  : 0.056319 ms

SQL[SELECT * FROM `alert_tests`]
SQL RUNTIME[0.00016594s]
Cached 0 alert rules.

SQL[SELECT * FROM `alert_table` WHERE `device_id` = '1052']
SQL RUNTIME[0.00018501s]
#####  upssrv2.komunala.local [1052]  #####

 o OS                   generic
 o OS Group             unix
 o Last poll duration   3.06 seconds
 o Last Polled          2025-06-12 16:27:21
 o SNMP Version         v2c
Host 'upssrv2.komunala.local' found in /etc/hosts: 213.250.36.26

CMD[/usr/bin/fping -t 500 -c 1 -q 213.250.36.26 2>&1]

CMD EXITCODE[0]
CMD RUNTIME[0.0117s]
STDOUT[
213.250.36.26 : xmt/rcv/%loss = 1/1/0%, min/avg/max = 9.11/9.11/9.11
]
Return cached 'upssrv2.komunala.local' = '213.250.36.26'

CMD[/usr/bin/snmpget -t '15' -r '5' -v2c -c *** -Pud -OQUsn -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'213.250.36.26':'10181' .1.3.6.1.2.1.1.2.0 .1.3.6.1.2.1.1.3.0]

CMD EXITCODE[1]
CMD RUNTIME[90.0902s]
STDOUT[

]
STDERR[
Timeout: No Response from udp:213.250.36.26:10181.
]
SNMP STATUS[FALSE]
 o Device status        Device is not responding to SNMP requests
Return cached 'upssrv2.komunala.local' = '213.250.36.26'

SQL[UPDATE `devices` SET `status_type` ='snmp' WHERE device_id = '1052']
SQL RUNTIME[0.00130391s]

SQL[SELECT * FROM `devices` WHERE `devices`.`device_id` = '1052']
SQL RUNTIME[0.00032711s]

SQL[INSERT INTO `eventlog` (`device_id`,`entity_id`,`entity_type`,`timestamp`,`severity`,`message`) VALUES ('1052','1052','device',NOW(),'3','Device status changed to Down (ping -> snmp)')]
SQL RUNTIME[0.00673699s]
RRD /opt/observium/rrd/upssrv2.komunala.local/status.rrd already exists - no need to create.

RRD CMD[update /opt/observium/rrd/upssrv2.komunala.local/status.rrd N:0]
RRD RUNTIME[0.0011s]
RRD STDOUT[ERROR: opening '/opt/observium/rrd/upssrv2.komunala.local/status.rrd': Permission denied]
RRD_STATUS[FALSE]

RRD /opt/observium/rrd/upssrv2.komunala.local/ping.rrd already exists - no need to create.

RRD CMD[update /opt/observium/rrd/upssrv2.komunala.local/ping.rrd N:9.11]
RRD RUNTIME[0.0011s]
RRD STDOUT[ERROR: opening '/opt/observium/rrd/upssrv2.komunala.local/ping.rrd': Permission denied]
RRD_STATUS[FALSE]

RRD /opt/observium/rrd/upssrv2.komunala.local/ping_snmp.rrd already exists - no need to create.

RRD CMD[update /opt/observium/rrd/upssrv2.komunala.local/ping_snmp.rrd N:U]
RRD RUNTIME[0.0011s]
RRD STDOUT[ERROR: opening '/opt/observium/rrd/upssrv2.komunala.local/ping_snmp.rrd': Permission denied]
RRD_STATUS[FALSE]

 o Poller time          90.1356 seconds
RRD /opt/observium/rrd/upssrv2.komunala.local/perf-poller.rrd already exists - no need to create.

RRD CMD[update /opt/observium/rrd/upssrv2.komunala.local/perf-poller.rrd N:90.1356]
RRD RUNTIME[0.0011s]
RRD STDOUT[ERROR: opening '/opt/observium/rrd/upssrv2.komunala.local/perf-poller.rrd': Permission denied]
RRD_STATUS[FALSE]

SQL[DELETE FROM `observium_processes` WHERE `process_pid` = '1574' AND `process_name` = 'poller.php' AND `device_id` = '1052' AND `poller_id` = '0']
SQL RUNTIME[0.00112414s]
poller.php: upssrv2.komunala.local - 1 devices polled in 90.2475 secs
#####  Completed polling run at 2025-06-13 13:02:53  #####

NOTE, $config['snmp']['hide_auth'] is set to TRUE, snmp community and snmp v3 auth hidden from debug output.
 o Devices Polled       1
 o Poller Time          90.2475 secs
 o Definitions          0.056318998336792 secs
 o Memory usage         26MB (peak: 27.11MB)
 o MySQL Usage          Insert[2/0.015s] Update[1/0.003s] Delete[2/0.004s] Cell[4/0.004s] Row[2/0.002s] Rows[7/0.085s] Col[1/0.001s]
                        Connect[1/0s]Func[2/0.018s] (0.132s 0.146%)
 o RRDTool Usage        update[4/0.004s] (0.004s 0.005%)
 o SNMP Usage           errors[/0s] snmpget[1/90.09s] (90.09s 99.826%)
Log file '/opt/observium/observium.log' is not writeable, check file permissions.
_______________________________________________
observium mailing list -- observium@lists.observium.org
To unsubscribe send an email to observium-leave@lists.observium.org