no messages in event log for 2 days

newer
Storage usage with RAID

older
An interesting issue with Cisco...

Simon Smith

3 Feb 2015 3 Feb '15

11:52 p.m.

Hi All

Ive had no Messages appear in my Event Logs panel for the past 2 days?

where would be the best place to start checking if ive got a code problem or maybe an SVN update has goofed it up?

Only reason I ask this is because we have had a few devices today go on and off alot and there are no entries in the Event Log Panel of them going off and on

but yet ive been getting the email alerts?

Regards

Simon

Show replies by date

Adam Armstrong

3 Feb 3 Feb

6:12 p.m.

Seems you updated when there was a sql bug present.

Run this against your database :

ALTER TABLE `eventlog` ADD `severity` TINYINT NOT NULL DEFAULT '6' AFTER `entity_id`; UPDATE `eventlog` SET `severity` = 4 WHERE `message` REGEXP '(above|under) threshold'; UPDATE `eventlog` SET `severity` = 5 WHERE `message` REGEXP ' [.(.]by (user|cron|console)'; UPDATE `eventlog` SET `severity` = 3 WHERE `message` LIKE 'Device status changed to%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE '%probably the device was replaced%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE 'OS changed%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE 'Device rebooted%'; UPDATE `eventlog` SET `severity` = 4 WHERE `entity_type` IN ('bgp_peer', 'sensor') AND `message` REGEXP ' (Up|Down|Alarm): ';

And it should fix it.

adam. On 03/02/2015 22:52:58, Simon Smith simonsmith5521@gmail.com wrote: Hi All

Ive had no Messages appear in my Event Logs panel for the past 2 days?

where would be the best place to start checking if ive got a code problem or maybe an SVN update has goofed it up?

Only reason I ask this is because we have had a few devices today go on and off alot and there are no entries in the Event Log Panel of them going off and on

but yet ive been getting the email alerts?

Regards

Simon

_______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

Simon Smith

4 Feb 4 Feb

12:17 a.m.

No luck adam :(

Error Code: 1060. Duplicate column name 'severity'

and ive also come across this in update-errors.log

[2015/02/01 03:59:18 +0000] discovery.php(554): ====== Schema update 179 -> 180 ============== [2015/02/01 03:59:18 +0000] discovery.php(554): Query: ALTER TABLE `eventlog` ADD `severity` TINYINT NOT NULL DEFAULT '6' AFTER `entity_id`; [2015/02/01 03:59:18 +0000] discovery.php(554): Error: Unknown column 'entity_id' in 'eventlog' [2015/02/01 03:59:18 +0000] discovery.php(554): Query: UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE '%above threshold%' [2015/02/01 03:59:18 +0000] discovery.php(554): Error: Unknown column 'severity' in 'field list' [2015/02/01 03:59:18 +0000] discovery.php(554): Query: UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE '%under threshold%' [2015/02/01 03:59:18 +0000] discovery.php(554): Error: Unknown column 'severity' in 'field list' [2015/02/01 12:59:03 +0000] discovery.php(22292): ====== Schema update 181 -> 182 ============== [2015/02/01 12:59:03 +0000] discovery.php(22292): Query: UPDATE `eventlog` SET `severity` = 5 WHERE `message` REGEXP ' [.(.]by (user|cron|console)'; [2015/02/01 12:59:03 +0000] discovery.php(22292): Error: Unknown column 'severity' in 'field list' [2015/02/01 12:59:03 +0000] discovery.php(22292): Query: UPDATE `eventlog` SET `severity` = 3 WHERE `message` LIKE '%Device status changed to%'; [2015/02/01 12:59:03 +0000] discovery.php(22292): Error: Unknown column 'severity' in 'field list' [2015/02/01 12:59:03 +0000] discovery.php(22292): Query: UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE '%probably the device was replaced%'; [2015/02/01 12:59:03 +0000] discovery.php(22292): Error: Unknown column 'severity' in 'field list' [2015/02/02 12:59:03 +0000] discovery.php(14299): ====== Schema update 182 -> 183 ============== [2015/02/02 12:59:03 +0000] discovery.php(14299): Query: UPDATE `eventlog` SET `severity` = 4 WHERE `entity_type` IN ('bgp_peer', 'sensor') AND `message` REGEXP ' (Up|Down|Alarm): '; [2015/02/02 12:59:03 +0000] discovery.php(14299): Error: Unknown column 'severity' in 'field list'

regards

Simon

...

On 3 Feb 2015, at 5:12 pm, Adam Armstrong adama@memetic.org wrote:

Seems you updated when there was a sql bug present.

Run this against your database :

ALTER TABLE `eventlog` ADD `severity` TINYINT NOT NULL DEFAULT '6' AFTER `entity_id`; UPDATE `eventlog` SET `severity` = 4 WHERE `message` REGEXP '(above|under) threshold'; UPDATE `eventlog` SET `severity` = 5 WHERE `message` REGEXP ' [.(.]by (user|cron|console)'; UPDATE `eventlog` SET `severity` = 3 WHERE `message` LIKE 'Device status changed to%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE '%probably the device was replaced%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE 'OS changed%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE 'Device rebooted%'; UPDATE `eventlog` SET `severity` = 4 WHERE `entity_type` IN ('bgp_peer', 'sensor') AND `message` REGEXP ' (Up|Down|Alarm): ';

And it should fix it.

adam.

...
On 03/02/2015 22:52:58, Simon Smith simonsmith5521@gmail.com wrote:

Hi All

Ive had no Messages appear in my Event Logs panel for the past 2 days?

where would be the best place to start checking if ive got a code problem or maybe an SVN update has goofed it up?

Only reason I ask this is because we have had a few devices today go on and off alot and there are no entries in the Event Log Panel of them going off and on

but yet ive been getting the email alerts?

Regards

Simon

observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

Adam Armstrong

3 Feb 3 Feb

6:21 p.m.

Hmm. What does your eventlog table look like?

I should have just killed the table and started again! :D

adam. On 03/02/2015 23:17:14, Simon Smith simonsmith5521@gmail.com wrote: No luck adam :(

Error Code: 1060. Duplicate column name 'severity'

and ive also come across this in update-errors.log

regards

Simon

On 3 Feb 2015, at 5:12 pm, Adam Armstrong <adama@memetic.org [mailto:adama@memetic.org]> wrote:

Seems you updated when there was a sql bug present.

Run this against your database :

And it should fix it.

adam. On 03/02/2015 22:52:58, Simon Smith <simonsmith5521@gmail.com [mailto:simonsmith5521@gmail.com]> wrote: Hi All

Ive had no Messages appear in my Event Logs panel for the past 2 days?

where would be the best place to start checking if ive got a code problem or maybe an SVN update has goofed it up?

Only reason I ask this is because we have had a few devices today go on and off alot and there are no entries in the Event Log Panel of them going off and on

but yet ive been getting the email alerts?

Regards

Simon

_______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

Simon Smith

4 Feb 4 Feb

12:22 a.m.

ignore my last email adam,

my bad, i checked the table `eventlog` and it already had the column severity in for some reason?

the other commands completed with loads and loads of updates…

however if i goto the EventLog now its still not showing any new information for last two days :(

am i just going to have to wait for something to happen for the DB to be updated?

Simon

...

On 3 Feb 2015, at 5:12 pm, Adam Armstrong adama@memetic.org wrote:

Seems you updated when there was a sql bug present.

Run this against your database :

ALTER TABLE `eventlog` ADD `severity` TINYINT NOT NULL DEFAULT '6' AFTER `entity_id`; UPDATE `eventlog` SET `severity` = 4 WHERE `message` REGEXP '(above|under) threshold'; UPDATE `eventlog` SET `severity` = 5 WHERE `message` REGEXP ' [.(.]by (user|cron|console)'; UPDATE `eventlog` SET `severity` = 3 WHERE `message` LIKE 'Device status changed to%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE '%probably the device was replaced%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE 'OS changed%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE 'Device rebooted%'; UPDATE `eventlog` SET `severity` = 4 WHERE `entity_type` IN ('bgp_peer', 'sensor') AND `message` REGEXP ' (Up|Down|Alarm): ';

And it should fix it.

adam.

...
On 03/02/2015 22:52:58, Simon Smith simonsmith5521@gmail.com wrote:

Hi All

Ive had no Messages appear in my Event Logs panel for the past 2 days?

where would be the best place to start checking if ive got a code problem or maybe an SVN update has goofed it up?

Only reason I ask this is because we have had a few devices today go on and off alot and there are no entries in the Event Log Panel of them going off and on

but yet ive been getting the email alerts?

Regards

Simon

observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

Adam Armstrong

3 Feb 3 Feb

6:25 p.m.

Your table should look like :

http://alpha.memetic.org/~adama/snaps/dev.observium.org___localhost___observ...

adam. On 03/02/2015 23:22:35, Simon Smith simonsmith5521@gmail.com wrote: ignore my last email adam,

my bad, i checked the table `eventlog` and it already had the column severity in for some reason?

the other commands completed with loads and loads of updates…

however if i goto the EventLog now its still not showing any new information for last two days :(

am i just going to have to wait for something to happen for the DB to be updated?

Simon

On 3 Feb 2015, at 5:12 pm, Adam Armstrong <adama@memetic.org [mailto:adama@memetic.org]> wrote:

Seems you updated when there was a sql bug present.

Run this against your database :

And it should fix it.

adam. On 03/02/2015 22:52:58, Simon Smith <simonsmith5521@gmail.com [mailto:simonsmith5521@gmail.com]> wrote: Hi All

Ive had no Messages appear in my Event Logs panel for the past 2 days?

where would be the best place to start checking if ive got a code problem or maybe an SVN update has goofed it up?

Only reason I ask this is because we have had a few devices today go on and off alot and there are no entries in the Event Log Panel of them going off and on

but yet ive been getting the email alerts?

Regards

Simon

_______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

Simon Smith

4 Feb 4 Feb

12:31 a.m.

Hi Adam

i don’t quite know what the problem was with the Table but it seems to have started working again?

i took a few servers SNMPs offline to generate the offline messages and they have suddenly appeared in the Event Logs page !?

no idea what the problem was, but thank you for the help, keep up the brilliant work :)

Simon

...

On 3 Feb 2015, at 5:25 pm, Adam Armstrong adama@memetic.org wrote:

Your table should look like :

http://alpha.memetic.org/~adama/snaps/dev.observium.org___localhost___observ...

adam.

...
On 03/02/2015 23:22:35, Simon Smith simonsmith5521@gmail.com wrote:

ignore my last email adam,

my bad, i checked the table `eventlog` and it already had the column severity in for some reason?

the other commands completed with loads and loads of updates…

however if i goto the EventLog now its still not showing any new information for last two days :(

am i just going to have to wait for something to happen for the DB to be updated?

Simon

...
On 3 Feb 2015, at 5:12 pm, Adam Armstrong <adama@memetic.org mailto:adama@memetic.org> wrote:

Seems you updated when there was a sql bug present.

Run this against your database :

ALTER TABLE `eventlog` ADD `severity` TINYINT NOT NULL DEFAULT '6' AFTER `entity_id`; UPDATE `eventlog` SET `severity` = 4 WHERE `message` REGEXP '(above|under) threshold'; UPDATE `eventlog` SET `severity` = 5 WHERE `message` REGEXP ' [.(.]by (user|cron|console)'; UPDATE `eventlog` SET `severity` = 3 WHERE `message` LIKE 'Device status changed to%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE '%probably the device was replaced%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE 'OS changed%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE 'Device rebooted%'; UPDATE `eventlog` SET `severity` = 4 WHERE `entity_type` IN ('bgp_peer', 'sensor') AND `message` REGEXP ' (Up|Down|Alarm): ';

And it should fix it.

adam.

...
On 03/02/2015 22:52:58, Simon Smith <simonsmith5521@gmail.com mailto:simonsmith5521@gmail.com> wrote:

Hi All

Ive had no Messages appear in my Event Logs panel for the past 2 days?

where would be the best place to start checking if ive got a code problem or maybe an SVN update has goofed it up?

Only reason I ask this is because we have had a few devices today go on and off alot and there are no entries in the Event Log Panel of them going off and on

but yet ive been getting the email alerts?

Regards

Simon

observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

Matt Ladewig

12:38 a.m.

Hey Adam – I’m on 6239 and appear to be in the same boat.

mysql> desc eventlog; +-------------+-------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------------+-------------+------+-----+---------+----------------+ | event_id | int(11) | NO | PRI | NULL | auto_increment | | device_id | int(11) | NO | MUL | 0 | | | timestamp | datetime | YES | MUL | NULL | | | message | text | YES | | NULL | | | entity_type | varchar(64) | YES | MUL | NULL | | | entity_id | int(64) | YES | | NULL | | +-------------+-------------+------+-----+---------+----------------+ 6 rows in set (0.00 sec)

Can you confirm that the script you mentioned below will take care of this?

Thanks,

Matt

From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Tuesday, February 3, 2015 9:25 AM To: observium@observium.org Subject: Re: [Observium] no messages in event log for 2 days

Your table should look like :

http://alpha.memetic.org/~adama/snaps/dev.observium.org___localhost___observ...

adam.

On 03/02/2015 23:22:35, Simon Smith <simonsmith5521@gmail.commailto:simonsmith5521@gmail.com> wrote: ignore my last email adam,

my bad, i checked the table `eventlog` and it already had the column severity in for some reason?

the other commands completed with loads and loads of updates…

however if i goto the EventLog now its still not showing any new information for last two days :(

am i just going to have to wait for something to happen for the DB to be updated?

Simon

On 3 Feb 2015, at 5:12 pm, Adam Armstrong <adama@memetic.orgmailto:adama@memetic.org> wrote:

Seems you updated when there was a sql bug present.

Run this against your database :

And it should fix it.

adam. On 03/02/2015 22:52:58, Simon Smith <simonsmith5521@gmail.commailto:simonsmith5521@gmail.com> wrote: Hi All

Ive had no Messages appear in my Event Logs panel for the past 2 days?

where would be the best place to start checking if ive got a code problem or maybe an SVN update has goofed it up?

Only reason I ask this is because we have had a few devices today go on and off alot and there are no entries in the Event Log Panel of them going off and on

but yet ive been getting the email alerts?

Regards

Simon

_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

Adam Armstrong

3 Feb 3 Feb

6:42 p.m.

Yup, seems the original commit was in the wrong order.

adam. On 03/02/2015 23:38:36, Matt Ladewig matt@univoip.com wrote: Hey Adam – I’m on 6239 and appear to be in the same boat. mysql> desc eventlog; +-------------+-------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +-------------+-------------+------+-----+---------+----------------+ | event_id | int(11) | NO | PRI | NULL | auto_increment | | device_id | int(11) | NO | MUL | 0 | | | timestamp | datetime | YES | MUL | NULL | | | message | text | YES | | NULL | | | entity_type | varchar(64) | YES | MUL | NULL | | | entity_id | int(64) | YES | | NULL | | +-------------+-------------+------+-----+---------+----------------+ 6 rows in set (0.00 sec) Can you confirm that the script you mentioned below will take care of this?

Thanks,

Matt From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Tuesday, February 3, 2015 9:25 AM To: observium@observium.org Subject: Re: [Observium] no messages in event log for 2 days Your table should look like : http://alpha.memetic.org/~adama/snaps/dev.observium.org___localhost___observ... [http://alpha.memetic.org/~adama/snaps/dev.observium.org___localhost___observ...] adam. On 03/02/2015 23:22:35, Simon Smith <simonsmith5521@gmail.com [mailto:simonsmith5521@gmail.com]> wrote: ignore my last email adam, my bad, i checked the table `eventlog` and it already had the column severity in for some reason? the other commands completed with loads and loads of updates… however if i goto the EventLog now its still not showing any new information for last two days :( am i just going to have to wait for something to happen for the DB to be updated? Simon On 3 Feb 2015, at 5:12 pm, Adam Armstrong <adama@memetic.org [mailto:adama@memetic.org]> wrote: Seems you updated when there was a sql bug present. Run this against your database : ALTER TABLE `eventlog` ADD `severity` TINYINT NOT NULL DEFAULT '6' AFTER `entity_id`; UPDATE `eventlog` SET `severity` = 4 WHERE `message` REGEXP '(above|under) threshold'; UPDATE `eventlog` SET `severity` = 5 WHERE `message` REGEXP ' [.(.]by (user|cron|console)'; UPDATE `eventlog` SET `severity` = 3 WHERE `message` LIKE 'Device status changed to%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE '%probably the device was replaced%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE 'OS changed%'; UPDATE `eventlog` SET `severity` = 4 WHERE `message` LIKE 'Device rebooted%'; UPDATE `eventlog` SET `severity` = 4 WHERE `entity_type` IN ('bgp_peer', 'sensor') AND `message` REGEXP ' (Up|Down|Alarm): ';

And it should fix it.

adam. On 03/02/2015 22:52:58, Simon Smith <simonsmith5521@gmail.com [mailto:simonsmith5521@gmail.com]> wrote: Hi All

Ive had no Messages appear in my Event Logs panel for the past 2 days?

where would be the best place to start checking if ive got a code problem or maybe an SVN update has goofed it up?

Only reason I ask this is because we have had a few devices today go on and off alot and there are no entries in the Event Log Panel of them going off and on

but yet ive been getting the email alerts?

Regards

Simon

_______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium] _______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]

Adam Armstrong

6:46 p.m.

I've moved all of these statements to a new sql update file, so they will fix issues people who were hit by the initial issue.

People without issues will get a couple of errors in logs, but nothing to worry about :)

adam. On 03/02/2015 23:42:42, Adam Armstrong adama@memetic.org wrote: Yup, seems the original commit was in the wrong order.

Thanks,

And it should fix it.

adam. On 03/02/2015 22:52:58, Simon Smith <simonsmith5521@gmail.com [mailto:simonsmith5521@gmail.com]> wrote: Hi All

Ive had no Messages appear in my Event Logs panel for the past 2 days?

where would be the best place to start checking if ive got a code problem or maybe an SVN update has goofed it up?

Only reason I ask this is because we have had a few devices today go on and off alot and there are no entries in the Event Log Panel of them going off and on

but yet ive been getting the email alerts?

Regards

Simon

Matt Ladewig

4 Feb 4 Feb

2:15 a.m.

Thank you – Now on 6243 and issue is resolved. Events are recording correctly.

matt

From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Tuesday, February 3, 2015 9:46 AM To: observium@observium.org Subject: Re: [Observium] no messages in event log for 2 days

I've moved all of these statements to a new sql update file, so they will fix issues people who were hit by the initial issue.

People without issues will get a couple of errors in logs, but nothing to worry about :)

adam.

On 03/02/2015 23:42:42, Adam Armstrong <adama@memetic.orgmailto:adama@memetic.org> wrote: Yup, seems the original commit was in the wrong order.

adam.

On 03/02/2015 23:38:36, Matt Ladewig <matt@univoip.commailto:matt@univoip.com> wrote: Hey Adam – I’m on 6239 and appear to be in the same boat.

Can you confirm that the script you mentioned below will take care of this?

Thanks,

Matt

From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong Sent: Tuesday, February 3, 2015 9:25 AM To: observium@observium.orgmailto:observium@observium.org Subject: Re: [Observium] no messages in event log for 2 days

Your table should look like :

http://alpha.memetic.org/~adama/snaps/dev.observium.org___localhost___observ...

adam.

On 03/02/2015 23:22:35, Simon Smith <simonsmith5521@gmail.commailto:simonsmith5521@gmail.com> wrote: ignore my last email adam,

my bad, i checked the table `eventlog` and it already had the column severity in for some reason?

the other commands completed with loads and loads of updates…

however if i goto the EventLog now its still not showing any new information for last two days :(

am i just going to have to wait for something to happen for the DB to be updated?

Simon

On 3 Feb 2015, at 5:12 pm, Adam Armstrong <adama@memetic.orgmailto:adama@memetic.org> wrote:

Seems you updated when there was a sql bug present.

Run this against your database :

And it should fix it.

adam. On 03/02/2015 22:52:58, Simon Smith <simonsmith5521@gmail.commailto:simonsmith5521@gmail.com> wrote: Hi All

Ive had no Messages appear in my Event Logs panel for the past 2 days?

where would be the best place to start checking if ive got a code problem or maybe an SVN update has goofed it up?

Only reason I ask this is because we have had a few devices today go on and off alot and there are no entries in the Event Log Panel of them going off and on

but yet ive been getting the email alerts?

Regards

Simon

3655

Age (days ago)

3656

Last active (days ago)

List overview

Download

10 comments

3 participants

tags (0)

participants (3)

Adam Armstrong
Matt Ladewig
Simon Smith