Hi Basile,

Thanks for that patch; that fixed it up for us. Would be great to see that included in Observium soon.

Cheers,

Nathan

From: observium [mailto:observium-bounces@observium.org] On Behalf Of Basile Bluntschli
Sent: Saturday, 9 May 2015 5:26 AM
To: Observium Network Observation System
Subject: Re: [Observium] Fortigate Ports Disappeared

Hi Serghei

I have no idea how to apply a svn diff file to an existing file. I just followed the instructions on the Observium website on how to submit a patch :) I would need to use google as well...

Else you could manually edit the fortigate.inc.pcp file, the patch provides something like three or four lines of code change... That should be doable as well.

Kind regards

Basile

2015-05-08 16:42 GMT+02:00 Serghei Golipad <s.golipad@kelway.com>:

Hi Basile,

Am I doing something wrong here?

observ01:/opt/observium/includes/polling/os# patch -p1 < r6384.patch

can't find file to patch at input line 5

Perhaps you used the wrong -p or --strip option?

The text leading up to this was:

--------------------------

|Index: fortigate.inc.php

|===================================================================

|--- fortigate.inc.php (revision 6384)

|+++ fortigate.inc.php (working copy)

--------------------------

File to patch: fortigate.inc.php

patching file fortigate.inc.php

Hunk #1 FAILED at 11.

patch unexpectedly ends in middle of line

Hunk #2 FAILED at 23.

2 out of 2 hunks FAILED -- saving rejects to file fortigate.inc.php.rej

root@uk1soc-observ01:/opt/observium/includes/polling/os#

From: observium [mailto:observium-bounces@observium.org] On Behalf Of Basile Bluntschli
Sent: 08 May 2015 15:34

To: Observium Network Observation System
Subject: Re: [Observium] Fortigate Ports Disappeared

Hi Serghei

Fortigate CPU was fixed somewhere in August 2014. The same change broke back then the Firewall Session counter.

You may want to update to the current svn version? And then reapply my patch?

This should work.

Kind regards

Basile

2015-05-08 15:23 GMT+02:00 Serghei Golipad <s.golipad@kelway.com>:

Hi Basile,

I’ve tried that one, patch fails. Are you able to send a copy of your fortigate.php.inc?

P.S does it fix CPU graphs too? Also any reason why there’s CPU and Processor graphs?

Thanks

From: observium [mailto:observium-bounces@observium.org] On Behalf Of Basile Bluntschli
Sent: 08 May 2015 13:57

To: Observium Network Observation System
Subject: Re: [Observium] Fortigate Ports Disappeared

Hi Serghei

there is a patch here:
http://jira.observium.org/browse/OBSERVIUM-1278?jql=

It works for me... You can try it on your own risk :)

Kind regards

Basile

2015-05-08 10:22 GMT+02:00 Serghei Golipad <s.golipad@kelway.com>:

Hi Nathan,

That was spot on :D thank you very much! This problem was been screwing our traffic accounting graphs and causing all sorts of aggravation J We are on: v5.0,build3608,140409 (GA Patch 7) time to upgrade, I know.

P.S if only there was a fix to sessions being graphed L

Cheers

From: observium [mailto:observium-bounces@observium.org] On Behalf Of Nathan Phelan
Sent: 08 May 2015 05:58

To: Observium Network Observation System
Subject: Re: [Observium] Fortigate Ports Disappeared

Hi Serghei,

This is an issue we have run into several times. Every time it has been due to a SNMPD bug on the fortigates.

There is an issue with several firmware versions that returns invalid values for interfaces that have been recently deleted (usually vlan subinterfaces or ipsec interfaces).

When this has happened snmp on the fortigates will still return values for everything apart from anything referencing the deleted interface (which is still in snmp somewhere).

I have logged a ticket with Fortinet about this (in Feb 2014) and it is supposed to be fixed in newer firmware versions (although I have seen it crop up again on ‘fixed’ versions).

When it is in this state snmpwalk actually fails the following query

snmpwalk -v2c -c public fortigateIPAddress ifType

Error in packet.
Reason: (genError) A general failure occured
Failed object: IF-MIB::ifType.45
(in this example 45 was snmp index of the deleted interface).

The fix for us has always been to kill and restart snmpd on the Fortigate (on all cluster members if it is a cluster).

You’ll need to find the process ID of the snmpd process and then kill it; watchdog will automatically restart it and then it’ll be fine.

show running processes

diag sys top 30 99

kill snmpd process by process id

diag sys kill 11 <process ID>

I don’t remember if you have to do a discovery after this or if it will return to normal on the next polling cycle.

Interested to hear if this solves it for you – also interested to hear what firmware version you are seeing this on.

Cheers,

Nathan

From: observium [mailto:observium-bounces@observium.org] On Behalf Of Serghei Golipad
Sent: Friday, 8 May 2015 1:17 AM
To: Observium Network Observation System
Subject: Re: [Observium] Fortigate Ports Disappeared

Hi Adam,

Anything I should be looking for in the debug? I can see ifNames correctly displayed during debug however those don’t make it into portal.

Also in event log I get below for all discovered ports:

Interface changed: [ifDescr] TEST_PORT -> NULL; [ifType] other -> NULL; [ifDuplex] -> 0

One of the things I’ve picked up from poller debug:

ignored (by empty ifDescr and ifName).

Thanks

From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong
Sent: 07 May 2015 15:45
To: observium@observium.org
Subject: Re: [Observium] Fortigate Ports Disappeared

snmpwalk doesn't prove anything, since we don't use snmpwalk. Checking the output from the poller module running in debug might, though.

./poller.php -h <host> -m ports -d

adam.

Sent from Mailbird

On 07/05/2015 15:38:44, Serghei Golipad <s.golipad@kelway.com> wrote:

Hi Everyone,

Has anyone else had this problem? All of a sudden some of the critical ports have disappeared, first though it was SNMP process crash on the firewall however simple snmpwalk proves that’s not the case. I then deleted device including RRDs and re-added, now none of the ports are showing L

I’m on 6404.

Thanks

PLEASE CONSIDER THE ENVIRONMENT BEFORE PRINTING THIS EMAIL.

Kelway Ltd is a Company registered in England and Wales.
This email is sent for and on behalf of Kelway Holdings Limited. Kelway Holdings Limited is the UK holding company for Kelway Limited trading as Kelway. The registered office for Kelway Holdings Limited (Company Number 05872067) and its UK group undertakings is 10 Fleet Place, London, EC4M 7RB. VAT Registration 902 1949 39.
Confidentiality: This e-mail contains proprietary information, some or all of which may be confidential and/or legally privileged. It is for the intended recipient only. If an addressing or transmission error has misdirected this e-mail, please notify the author by replying to this e-mail and then delete the original. If you are not the intended recipient you may not use, disclose, distribute, copy, print or rely on any information contained in this e-mail. You must not inform any other person other than Kelway or the sender of its existence.

For more information about Kelway please visit www.kelway.com

_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium