- observium - lists.observium.org

[SOLVED] Housekeeping not deleting records if you have a stupid number to kill.
by Crozier, Doug 23 Jul '14

23 Jul '14

I have a rather old and crufty installation of Observium (now at 5668) with over 70 million syslog, eventlog, and timing records that are older than 7 days. The first time I ran housekeeping -s to trim syslog messages it took about 30 minutes to return showing 42 million records. Just to be sure I was actually done deleting 42 million records, I re-ran the command. The same 42 million records were found. Meh. I ran the housekeeping in debug mode and noticed that I didn't have posix support installed. So I installed the php-process package and tried again. Still no records deleted. I tried deleting fewer records by keeping 18 months worth and that didn't help either. Here's the debug output from attempting to delete records older than 7 days: [root@xxx ~]# php /opt/observium/housekeeping.php -s -d DEBUG! SQL[SELECT COUNT(*) FROM `syslog` WHERE UNIX_TIMESTAMP(`timestamp`) < 1405322208;] 42123350 syslog entries from before 2014-07-14 15:16:48 will be deleted [y/N]: y SQL[DELETE FROM `syslog` WHERE UNIX_TIMESTAMP(`timestamp`) < 1405322208;] Syslog housekeeping: deleted entries Deleting the eventlog records also fails to remove the records: [root@xxxx observium]# php housekeeping.php -e -d DEBUG! SQL[SELECT COUNT(*) FROM `eventlog` WHERE UNIX_TIMESTAMP(`timestamp`) < 1380059875;] 29289378 eventlog entries from before 2013-09-25 05:57:55 will be deleted [y/N]: y SQL[DELETE FROM `eventlog` WHERE UNIX_TIMESTAMP(`timestamp`) < 1380059875] Event log housekeeping: deleted entries (As an aside, I'm not sure, but there may be a semi-colon missing on the delete statement for event log records.) What I eventually found was that trying to delete this many records exhausts the lock table (?). ERROR 1206 (HY000): The total number of locks exceeds the lock table size Issuing the same command from the mysql CLI and adding a 10,000,000 limit allowed the deletion to work correctly mysql> SELECT COUNT(*) FROM `eventlog` WHERE UNIX_TIMESTAMP(`timestamp`) < 1380059875; +----------+ | COUNT(*) | +----------+ | 28177378 | +----------+ 1 row in set (51.76 sec) mysql> DELETE FROM `eventlog` WHERE UNIX_TIMESTAMP(`timestamp`) < 1380059875 LIMIT 10000000; Query OK, 10000000 rows affected (3 min 40.32 sec) mysql> SELECT COUNT(*) FROM `eventlog` WHERE UNIX_TIMESTAMP(`timestamp`) < 1380059875; +----------+ | COUNT(*) | +----------+ | 18177378 | +----------+ 1 row in set (37.01 sec) Interesting to note that deleting 10 million records at a time with the LIMIT statement doesn't always work. Adjusting the LIMIT up and down eventually got me where I was going. Now that I have it the system down to a reasonable number of records, a nightly/weekly cron job should do just fine. Thanks for the effort you put into this Tom. For me, this was like a hardware upgrade.

3 2

About PPPOE / VPDN
by Paolo Giustiniani 23 Jul '14

23 Jul '14

I would like to monitor several interfaces pppoe connections. I enabled the feature and vpdn pppoe # $ config ['bad_if'] [] = "pppoe-"; $ config ['poller_modules'] ['cisco-vpdn'] = 1; The process of self discovery did not detect it. The command show vpdn order properly connections. How can I do?

5 18

Re: [Observium] observium Digest, Vol 48, Issue 97
by Kiru Pananthan 23 Jul '14

23 Jul '14

Hi Peter the machine we are polling is windows VM and also getting for the network switches also On Sun, Jul 20, 2014 at 6:55 AM, <observium-request(a)observium.org> wrote: > Send observium mailing list submissions to > observium(a)observium.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://postman.memetic.org/cgi-bin/mailman/listinfo/observium > or, via email, send a message with subject or body 'help' to > observium-request(a)observium.org > > You can reach the person managing the list at > observium-owner(a)observium.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of observium digest..." > > > Today's Topics: > > 1. Re: Device up / Down (Peter Persson) > 2. Does this product have server agents? (Chris Lawrence) > 3. Re: Does this product have server agents? (Adam Armstrong) > 4. Re: About PPPOE / VPDN (Adam Armstrong) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 19 Jul 2014 16:09:56 +0200 > From: Peter Persson <peter.persson(a)bredband2.se> > To: Observium Network Observation System <observium(a)observium.org> > Subject: Re: [Observium] Device up / Down > Message-ID: > < > CAPwoXysEEEmTBwWxGZFK6NdnsxoAujz40Oy+s8Pj2wbrWcgdPg(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > The machines doesnt respond as requested. > What type of machines are you polling? > > And thank you for your notice that the mail is confidential, I will not > show this mail for anyone! > > > 2014-07-19 2:41 GMT+02:00 Kiru Pananthan <kirup(a)aism.edu.my>: > > > Dear all > > We are getting an alerts message for every five minutes for 5 > > machines. Stating Device up: / Device Down > > > > Thanks > > Kiru > > > > -- > > *Confidentiality Disclaimer:* This e-mail and any attachments are > > confidential and intended solely for the intended addressee and may also > be > > privileged or exempt from disclosure under applicable law. If you are not > > the intended addressee, or have received this e-mail in error, please > > notify the sender immediately, delete it from your system and do not > copy, > > disclose, distribute or otherwise act in reliance upon any part of this > > e-mail or its attachments. Australian International School Malaysia and > all > > affiliates under Taylor's Education Group does not accept responsibility > > for any loss arising from unauthorised access to, or interference with, > any > > internet communications by any third party in reliance to this email, or > > from the transmission of any viruses. Please note that any views or > > opinions presented in this email are solely those of the author and do > not > > necessarily represent those of Australian International School Malaysia > > and > > all affiliates under Taylor's Education Group. Replies to this e-mail may > > be monitored by Australian International School Malaysia and all > > affiliates under Taylor's Education Group for operational or business > > reasons. > > _______________________________________________ > > observium mailing list > > observium(a)observium.org > > http://postman.memetic.org/cgi-bin/mailman/listinfo/observium > > >

1 0

Meru Networks Controllers + Access Points
by Sean Huggans 22 Jul '14

22 Jul '14

Any chance to have Meru Networks wireless controllers added? Running a model MC1550 managing 11 Aps and would love to be able to pull it in. Sean Huggans I.S. Services Manager Network Operations Engineer Minneola Healthcare shuggans(a)minneolahealthcare.com [image: A+_CE_tiny]

3 2

Login issue. IP works hostname does not?
by Scott Benninger 21 Jul '14

21 Jul '14

I have a strange issue that I cannot resolve. I have a recent install of the community edition on Ubuntu 14.04. Only difference is the sites available directive changes to make it work compared to the official install docs. Everything was working perfectly and I could login via IP, hostname, or FQDN. The server is a VMware VM and I forgot to install the VMware tools so our VMware monitoring software would stop complaining. I installed the tools and rebooted and now I can only login via the IP. If I try via hostname etc the form just clears with no errors. Adding /debug=1/ shows the mysql commands etc but no errors. I have checked all the hostname settings, defaulted the host file, checked the apache and site configs and nothing working. Seems like a php issue to me. Any ideas? Thanks in advance and my apologies if this something simple that I have missed. Scott

4 4

alert checker missing
by Dash Brousseau 19 Jul '14

19 Jul '14

I updated observium thinking I can create alerts based on http://www.observium.org/wiki/Creating_an_Alert_Checker But I don’t have add check on the menu bar Is that only for the professional paid version? Observium CE 0.14.4.5229 Apache 2.2.22 (Ubuntu) PHP 5.4.9-4ubuntu2.4 MySQL 5.5.34-0ubuntu0.13.04.1 SNMP NET-SNMP version: 5.4.3 RRDtool 1.4.7 Dash This message contains information which may be confidential and privileged. Unless you are the intended addressee (or authorized to receive for the intended addressee), you may not use, copy or disclose to anyone the message or any information contained in the message. If you have received the message in error, please advise the sender by e-mail then delete the message immediately. Francais: http://www.fundserv.com/fr/company/disclaimer<http://www.fundserv.com/francais/legal/disclaimer_email_ib.shtml>

3 2

Does this product have server agents?
by Chris Lawrence 19 Jul '14

19 Jul '14

I work for an MSP and am looking for a solution that has agents that act as collectors/relays to the main server. This networks and devices that need to be monitored are not directly connected on. PRTG has something like this - if that helps you understand what I'm asking about. Chris Lawrence Director of IT, Solutions and Services 888.825.6748 Support 480.663.6322 Direct 480.200.7875 Cell Chris.Lawrence(a)ValorGlobal.com<mailto:Chris.Lawrence@ValorGlobal.com> [cid:image001.png@01CF865C.4CF93F30]<http://valorglobal.com/> CONFIDENTIALITY: This e-mail (including any attachments) may contain confidential, proprietary and privileged information, and unauthorized disclosure or use is prohibited. If you received this e-mail in error, please notify the sender and delete this e-mail from your system.

2 1

Device up / Down
by Kiru Pananthan 19 Jul '14

19 Jul '14

Dear all We are getting an alerts message for every five minutes for 5 machines. Stating Device up: / Device Down Thanks Kiru -- *Confidentiality Disclaimer:* This e-mail and any attachments are confidential and intended solely for the intended addressee and may also be privileged or exempt from disclosure under applicable law. If you are not the intended addressee, or have received this e-mail in error, please notify the sender immediately, delete it from your system and do not copy, disclose, distribute or otherwise act in reliance upon any part of this e-mail or its attachments. Australian International School Malaysia and all affiliates under Taylor's Education Group does not accept responsibility for any loss arising from unauthorised access to, or interference with, any internet communications by any third party in reliance to this email, or from the transmission of any viruses. Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Australian International School Malaysia and all affiliates under Taylor's Education Group. Replies to this e-mail may be monitored by Australian International School Malaysia and all affiliates under Taylor's Education Group for operational or business reasons.

2 1

Moving obvsevium to a new host
by Ryan, Spencer J. 19 Jul '14

19 Jul '14

Hi All, We are going to migrate our Observium install (paid) to a different VM, same architecture and everything. I ended up pulling down the latest version from SVN, but to move the actual data over do I need to move anything other than the RRD directory, the config file, and the MySQL database? Thanks! -Spencer _______________________________________________________________ NOTICE TO PERSONS SUBJECT TO UNITED STATES TAXATION (MCPS) DISCLOSURE UNDER TREASURY CIRCULAR 230: The United States Federal tax advice, if any, contained in this document and its attachments may not be used or referred to in the promoting, marketing or recommending of any entity, investment plan or arrangement, nor is such advice intended or written to be used, and may not be used, by a taxpayer for the purpose of avoiding Federal tax penalties. _______________________________________________________________

3 3

Housekeeping script
by Tom Laermans 18 Jul '14

18 Jul '14

Hi everyone, Yesterday I committed a housekeeping script to clean up after Observium. If you have a nice number of devices combined with a long running time of Observium, you're sure to have a gigantic database of event log, syslog and performance timing information. Some of you have resorted to periodically truncating tables - fear no more, housekeeping.php is here! You have a choice of specifying either -a for all modules or s, e, r, p and/or t for specific things. Apart from that, there are options in the configuration to set before most of these things actually do anything. What it does: -s Cleans up syslog data, based on the age of the entries. Configured by $config['housekeeping']['syslog']['age'] = 0; // Maximum age of syslog entries in seconds; 0 to disable i.e. 7*86400 for 7 days. -e Cleans up eventlog data, based on the age of the entries. Configured by $config['housekeeping']['eventlog']['age'] = 0; // Maximum age of eventlog entries in seconds; 0 to disable i.e. 30*86400 for 30 days. -r Cleans up RRD files, there are 2 options here: Cleanup of RRD files with a modification time of a certain age. Ports that are gone, sensors that are gone, erroneaously created RRD files due to bugs or hardware changes, etc. Configured by $config['housekeeping']['rrd']['age'] = 0; // Maximum age of unused rrd files in seconds before automatically purging; 0 to disable i.e. 90*86400 for 90 days. Cleanup of invalid RRD files. Requires $config['file'] to be set correctly to the path of the "file" magic utility. Runs over all .rrd files to see if they are identified as RRDTool Databases. They can sometimes get corrupted (0 bytes, half a file) when your disk/ramdrive is full. When they exist, they are not recreated even if they're invalid. Consequently RRDtool refuses to add data into them as they are not valid RRD database files. This scripts deletes any .rrd file not identified as RRDTool. Configured by $config['housekeeping']['rrd']['invalid'] = TRUE; // Delete .rrd files that are not valid RRD files (eg created with a full disk) Note: this defaults to true currently. Handle with care. -p Cleans up deleted ports. You can do this manually by using the purge button in the web interface, but one could probably safely assume if a port has not returned after 30 days it really is gone. Shorter could be useful in your environment, mine is set to 7 days. Configured by $config['housekeeping']['deleted_ports']['age'] = 0; // Maximum age of deleted ports in seconds before automatically purging; 0 to disable i.e. 30*86400 for 30 days. -t: Cleans up perf time information. Observium keeps performance times per module, per device and per run. This quickly accumulates into a giant number of rows in the database. Configured by $config['housekeeping']['timing']['age'] = 0; // Maximum age of timing (discovery and poll time) entries in seconds; 0 to disable i.e. 7*86400 for 7 days. Housekeeping logs into housekeeping.log in the log_dir. (*) I recommend putting this into cron, daily or weekly (yes, even though the age is set in seconds, I don't recommend running it every minute ;), but only after you've run it manually once. It'll take quite a while the first time to clean up large databases. The "check for invalid RRD files" is probably quite I/O heavy. i'll be documenting this on the wiki soonish. (*) log_dir was also added as a new configuration directive, it defaults to ${installdir}/logs which will be created automatically for you if possible. Tom

6 10