Adam,
[root@obs observium]# cat /etc/cron.d/observium
33 */6 * * * root /opt/observium/discovery.php -h all >> /dev/null 2>&1
*/5 * * * * root /opt/observium/discovery.php -h new >> /dev/null 2>&1
*/5 * * * * root /opt/observium/poller.php -h all >> /dev/null 2>&1
*/5 * * * * root /opt/observium/poll-billing.php >> /dev/null 2>&1
01 * * * * root /opt/observium/billing-calculate.php >> /dev/null 2>&1
Regards,
Cameron Murray
Remote Support Engineer
How am I doing? Please
click here to rate my work.
P 1300 033 300
F 07 3809 3999
E cameron@techpath.com.au
www.techpath.com.au
Unit 2/55 Tradelink Rd, Hillcrest QLD 4118
Formerly Betta Computer Services Pty Ltd
|
|
|
|
Note: This message may contain privileged and confidential information that is the property of the intended recipient. The information herein is intended only for
use of the addressee. If you are not the intended recipient, then you are requested to return e-mail to TechPath Pty Ltd and destroy any copies made. Copying or disseminating any of this message is prohibited. Any views expressed in this message are those
of the individual sender and may not necessarily reflect the views of TechPath Pty Ltd.
From: observium [mailto:observium-bounces@observium.org]
On Behalf Of Adam Armstrong
Sent: Friday, 28 August 2015 9:51 AM
To: Observium Network Observation System
Subject: Re: [Observium] Alert Emails Not sending
What are you running in cron? Poller-wrapper.py or poller.php?
Adam.
Sent with AquaMail for Android
http://www.aqua-mail.com
On 28 August 2015 00:43:27 Cameron Murray <cameron@techpath.com.au> wrote:
Adam,
When I ran that I received the email alerts.
Regards,
Cameron Murray
Remote Support EngineerHow am I doing? Please click here to rate my work.
P 1300 033 300
F 07 3809 3999
E cameron@techpath.com.au
www.techpath.com.au
Unit 2/55 Tradelink Rd, Hillcrest QLD 4118
Formerly Betta Computer Services Pty Ltd
Note: This message may contain privileged and confidential information that is the property of the intended recipient. The information herein is intended only for use of the addressee. If you are not the intended recipient, then you are requested to return e-mail to TechPath Pty Ltd and destroy any copies made. Copying or disseminating any of this message is prohibited. Any views expressed in this message are those of the individual sender and may not necessarily reflect the views of TechPath Pty Ltd.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong
Sent: Friday, 28 August 2015 9:37 AM
To: Observium Network Observation System
Subject: Re: [Observium] Alert Emails Not sending
That alert is still delayed, so not notification yet :)
What happens if you run :
./alerter.php -h all
Adam.
Sent with AquaMail for Android
http://www.aqua-mail.comOn 28 August 2015 00:35:43 Cameron Murray <cameron@techpath.com.au> wrote:
Adam,
I’ve disabled the snmp agent for one device and obs has recorded the issue below:
https://dl.dropboxusercontent.com/u/8790688/obs3.png
Regards,
Cameron Murray
Remote Support EngineerHow am I doing? Please click here to rate my work.
P 1300 033 300
F 07 3809 3999
E cameron@techpath.com.au
www.techpath.com.au
Unit 2/55 Tradelink Rd, Hillcrest QLD 4118
Formerly Betta Computer Services Pty Ltd
Note: This message may contain privileged and confidential information that is the property of the intended recipient. The information herein is intended only for use of the addressee. If you are not the intended recipient, then you are requested to return e-mail to TechPath Pty Ltd and destroy any copies made. Copying or disseminating any of this message is prohibited. Any views expressed in this message are those of the individual sender and may not necessarily reflect the views of TechPath Pty Ltd.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong
Sent: Friday, 28 August 2015 9:13 AM
To: Observium Network Observation System
Subject: Re: [Observium] Alert Emails Not sending
This says the state hasn't changed for over a day.
You need to unfail and refail the alert to see if it sends.
Adam.
Sent with AquaMail for Android
http://www.aqua-mail.comOn 28 August 2015 00:11:13 Cameron Murray <cameron@techpath.com.au> wrote:
Yes the second line entry in the previous picture is post contacts being adjusted.
Regards,
Cameron Murray
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong
Sent: Friday, 28 August 2015 9:03 AM
To: Observium Network Observation System
Subject: Re: [Observium] Alert Emails Not sending
Hmm. That is odd. Are you sure an alert has entered the failed state since you configured the contacts?
Adam.
Sent with AquaMail for Android
http://www.aqua-mail.comOn 27 August 2015 23:58:19 Cameron Murray <cameron@techpath.com.au> wrote:
Adam,
I’m not seeing notifications for “Email” However when I search only for ALERT_NOTIFY there is 0 entries.
Regards,
Cameron Murray
Remote Support EngineerHow am I doing? Please click here to rate my work.
P 1300 033 300
F 07 3809 3999
E cameron@techpath.com.au
www.techpath.com.au
Unit 2/55 Tradelink Rd, Hillcrest QLD 4118
Formerly Betta Computer Services Pty Ltd
Note: This message may contain privileged and confidential information that is the property of the intended recipient. The information herein is intended only for use of the addressee. If you are not the intended recipient, then you are requested to return e-mail to TechPath Pty Ltd and destroy any copies made. Copying or disseminating any of this message is prohibited. Any views expressed in this message are those of the individual sender and may not necessarily reflect the views of TechPath Pty Ltd.
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong
Sent: Friday, 28 August 2015 8:50 AM
To: Observium Network Observation System
Subject: Re: [Observium] Alert Emails Not sending
Are the missing notifications recorded in the alert log?
Adam.
Sent with AquaMail for Android
http://www.aqua-mail.comOn 27 August 2015 23:46:15 Cameron Murray <cameron@techpath.com.au> wrote:
I have all alerts setup to a single for testing. Once I know it works the other alert groups will be added.
Regards,
Cameron Murray
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam Armstrong
Sent: Friday, 28 August 2015 8:43 AM
To: Observium Network Observation System
Subject: Re: [Observium] Alert Emails Not sending
Just out of interest, why are you associating all alerts with something in contacts rather than just setting a default in the config file?
If the default is working, why not just use that?
Adam.
Sent with AquaMail for Android
http://www.aqua-mail.comOn 27 August 2015 23:41:42 Cameron Murray <cameron@techpath.com.au> wrote:
Not showing in any logs – Like it is not being sent at all.
Alerts have been associated with contacts. https://dl.dropboxusercontent.com/u/8790688/obs.png
Regards,
Cameron Murray
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Casper Allerslev Jensen
Sent: Friday, 28 August 2015 2:21 AM
To: Observium Network Observation System
Subject: Re: [Observium] Alert Emails Not sending
Cameron have you checked that the email is not being captured by the spam filter?
I'm asking because associating the contact with the alert worked for me.. I can now see that postfix is sending mail to my contact email address..
my problem, and maybe Cameron, is that the from filed still is from root@observium01 which is being banned in my spam filter. I can't whitelist it because it has to be a propper domain syntax to do so.. like root@observium01.org or something like that..
I know that tom said it was the default from field from postfix and not the actual from field but i do see a root@observium01 in my spam filter that is being blocked... so something is wrong..
Aug 27 18:10:05 observium01 postfix/cleanup[2022]: B9291A532E: message-id=<e103e994b1f0ee8f892ba6a0b6ef6cbb@observium01>
Aug 27 18:10:05 observium01 postfix/qmgr[1101]: B9291A532E: from=<root@observium01>, size=15353, nrcpt=2 (queue active)
Aug 27 18:10:06 observium01 postfix/smtp[2025]: B9291A532E: to=<casper@dingit.dk>, relay=mxpool2.spamgateway.comodo.com[178.33.199.66]:25, delay=0.53, delays=0.05/0.04/0.23/0.21, dsn=5.0.0, status=bounced (host mxpool2.spamgateway.comodo.com[178.33.199.66] said: 550 Access denied - Invalid HELO name (See RFC5321 4.1.1.1) (in reply to RCPT TO command))
Aug 27 18:10:06 observium01 postfix/smtp[2025]: B9291A532E: lost connection with mxpool2.spamgateway.comodo.com[178.33.199.66] while sending RCPT TO
Aug 27 18:10:06 observium01 postfix/smtp[2025]: B9291A532E: to=<it@dingit.dk>, relay=mxpool2.spamgateway.comodo.com[178.33.199.65]:25, delay=0.91, delays=0.05/0.04/0.73/0.09, dsn=5.0.0, status=bounced (host mxpool2.spamgateway.comodo.com[178.33.199.65] said: 550 Access denied - Invalid HELO name (See RFC5321 4.1.1.1) (in reply to RCPT TO command))
Aug 27 18:10:06 observium01 postfix/cleanup[2022]: 9C34EA532F: message-id=<20150827161006.9C34EA532F@observium01>
Aug 27 18:10:06 observium01 postfix/qmgr[1101]: 9C34EA532F: from=<>, size=17621, nrcpt=1 (queue active)
Aug 27 18:10:06 observium01 postfix/bounce[2054]: B9291A532E: sender non-delivery notification: 9C34EA532F
Aug 27 18:10:06 observium01 postfix/qmgr[1101]: B9291A532E: removed
Aug 27 18:10:06 observium01 postfix/local[2058]: 9C34EA532F: to=<root@observium01>, relay=local, delay=0.04, delays=0/0.03/0/0.01, dsn=2.0.0, status=sent (delivered to mailbox)
Aug 27 18:10:06 observium01 postfix/qmgr[1101]: 9C34EA532F: removed
Now im happy with only one email for alerting so i could go with configuring it in the config.php file.. but nothing will work aslong as the from field is from root@observium01
I really don't care what is in the from field as long as its propper syntax. I would just whitelist it... its somewhat of a stupid problem actually :)
Fra: "Cameron Murray" <cameron@techpath.com.au>
Til: "Observium Network Observation System" <observium@observium.org>
Sendt: torsdag, 27. august 2015 12:12:25
Emne: Re: [Observium] Alert Emails Not sending
Certainly have for me
-------- Original message --------
From: Adam Armstrong
Date:27/08/2015 20:09 (GMT+10:00)
To: Observium Network Observation System
Subject: Re: [Observium] Alert Emails Not sending
Have the contacts been associated with any alerts?
Adam.
Sent with AquaMail for Android
http://www.aqua-mail.com
On 27 August 2015 8:38:28 am Casper Allerslev Jensen <casper@dingit.dk> wrote:
> Hi Tom,
>
> Thanks for tryng but it didn't help.. I set it to false but from postfix i
> can see my from field is still root@observium01 - this was tested after reboot
>
> Aug 27 09:35:05 observium01 postfix/pickup[1100]: 49784A532E: uid=0 from=<root>
> Aug 27 09:35:05 observium01 postfix/cleanup[2466]: 49784A532E:
> message-id=<532453c60007a53981b676a34a5dac3e@observium01>
> Aug 27 09:35:05 observium01 postfix/qmgr[1101]: 49784A532E:
> from=<root@observium01>, size=15223, nrcpt=1 (queue active)
> Aug 27 09:35:05 observium01 postfix/smtp[2470]: 49784A532E:
> to=<it@dingit.dk>, relay=XXXXXXX, delay=0.32, delays=0.06/0.02/0.16/0.09,
> dsn=5.0.0, status=bounced (host XXXXX said: 550 Access denied - Invalid
> HELO name (See RFC5321 4.1.1.1) (in reply to RCPT TO command))
> Aug 27 09:35:05 observium01 postfix/cleanup[2466]: 90B5FA532F:
> message-id=<20150827073505.90B5FA532F@observium01>
> Aug 27 09:35:05 observium01 postfix/qmgr[1101]: 90B5FA532F: from=<>,
> size=17097, nrcpt=1 (queue active)
> Aug 27 09:35:05 observium01 postfix/bounce[2491]: 49784A532E: sender
> non-delivery notification: 90B5FA532F
> Aug 27 09:35:05 observium01 postfix/qmgr[1101]: 49784A532E: removed
> Aug 27 09:35:05 observium01 postfix/local[2496]: 90B5FA532F:
> to=<root@observium01>, relay=local, delay=0.34, delays=0.01/0.31/0/0.02,
> dsn=2.0.0, status=sent (delivered to mailbox)
> Aug 27 09:35:05 observium01 postfix/qmgr[1101]: 90B5FA532F: removed
>
>
>
>
> ----- Original meddelelse -----
> Fra: "Tom Laermans" <tom.laermans@powersource.cx>
> Til: observium@observium.org
> Sendt: torsdag, 27. august 2015 09:15:35
> Emne: Re: [Observium] Alert Emails Not sending
>
> I'm no expert on this but maybe because you set default_only to true in the
> config?
>
> On Aug 27, 2015 7:05 AM, Casper Allerslev Jensen <casper@dingit.dk> wrote:
>>
>> Hi,
>>
>> I can confirm that installing postfix and hardcode the reciving mailbox in
>> config.php works. But it doesn't use the contacts im making in the
>> observium gui which seems to be the same issue Cameron has.
>>
>> the funny thing is my spam filter blocks the email, well to be expected,
>> but it shows it comming from root@observium even though my config.php says:
>>
>> // Set up a default alerter (email to a single address)
>> $config['email']['default'] = "it@dingit.dk";
>> $config['email']['from'] = "Observium <observium@dingit.dk>";
>> $config['email']['default_only'] = TRUE;
>>
>> any ideas?
>>
>> Also i couldn't find any documentation on how to set up email service in
>> observium. I figured it out from Adams and Cameron conversation on the mail
>> list.
>>
>>
>> ________________________________
>> Fra: "Adam Armstrong" <adama@memetic.org>
>> Til: "Observium Network Observation System" <observium@observium.org>
>> Sendt: torsdag, 27. august 2015 01:30:22
>> Emne: Re: [Observium] Alert Emails Not sending
>>
>> Does the alert log show them as being sent?
>>
>> They will only be sent when they first occur, so you'll need to have new
>> ones for any new alerts to be sent.
>>
>> Adam.
>>
>> Sent with AquaMail for Android
>> http://www.aqua-mail.com
>>
>> On 26 August 2015 11:46:40 pm Cameron Murray <cameron@techpath.com.au> wrote:
>>>
>>> Adam,
>>>
>>>
>>>
>>> Thanks yes – When I test with the ./test_mail.php -h bne-b1-lvl4-r1-apc2 I
>>> receive the email correctly however none of the alerts come through.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Cameron Murray
>>> Remote Support Engineer
>>>
>>> How am I doing? Please click here to rate my work.
>>>
>>>
>>>
>>> P 1300 033 300
>>> F 07 3809 3999
>>> E cameron@techpath.com.au
>>>
>>> www.techpath.com.au
>>>
>>> Unit 2/55 Tradelink Rd, Hillcrest QLD 4118
>>> Formerly Betta Computer Services Pty Ltd
>>>
>>> Note: This message may contain privileged and confidential information that
>>> is the property of the intended recipient. The information herein is
>>> intended only for use of the addressee. If you are not the intended
>>> recipient, then you are requested to return e-mail to TechPath Pty Ltd and
>>> destroy any copies made. Copying or disseminating any of this message is
>>> prohibited. Any views expressed in this message are those of the individual
>>> sender and may not necessarily reflect the views of TechPath Pty Ltd.
>>>
>>> From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam
>>> Armstrong
>>> Sent: Thursday, 27 August 2015 8:41 AM
>>> To: Observium Network Observation System
>>> Subject: Re: [Observium] Alert Emails Not sending
>>>
>>>
>>>
>>> Ahh indeed, that's true. You never actually tell it to use SMTP, even
>>> though you set the SMTP options!
>>>
>>> In that case, make sure the local mail relay is installed and configured
>>> (probably just apt-get install postfix) and check /var/log/mail.log
>>>
>>> Adam.
>>>
>>> Sent with AquaMail for Android
>>> http://www.aqua-mail.com
>>>
>>> On 26 August 2015 11:28:51 pm Cameron Murray <cameron@techpath.com.au> wrote:
>>>>
>>>> Thanks Adam,
>>>>
>>>>
>>>>
>>>> It appears to already be using the php backend to send emails:
>>>>
>>>>
>>>>
>>>> E-mails ENABLED globally ($config['email']['enable'])
>>>>
>>>> E-mails use PHP-builtin MAIL backend ($config['email']['backend'])
>>>>
>>>> E-mails send only to default email ($config['email']['default_only'])
>>>>
>>>> E-mails use these default recipient(s) ($config['email']['default']):
>>>>
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Cameron Murray
>>>>
>>>>
>>>> From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam
>>>> Armstrong
>>>> Sent: Thursday, 27 August 2015 8:20 AM
>>>> To: Observium Network Observation System
>>>> Subject: Re: [Observium] Alert Emails Not sending
>>>>
>>>>
>>>>
>>>> Remove all of the SMTP configutation and just let php send the mail using
>>>> the local MTA itself.
>>>>
>>>> You'll need to make sure one is installed, but its fairly straightforward
>>>> with Debian/Ubuntu.
>>>>
>>>> Adam.
>>>>
>>>> Sent with AquaMail for Android
>>>> http://www.aqua-mail.com
>>>>
>>>> On 26 August 2015 11:18:57 pm Cameron Murray <cameron@techpath.com.au> wrote:
>>>>>
>>>>> Adam,
>>>>>
>>>>>
>>>>>
>>>>> Can you recommend a working configuration?
>>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Cameron Murray
>>>>>
>>>>>
>>>>>
>>>>> From: observium [mailto:observium-bounces@observium.org] On Behalf Of Adam
>>>>> Armstrong
>>>>> Sent: Thursday, 27 August 2015 8:08 AM
>>>>> To: Observium Network Observation System
>>>>> Subject: Re: [Observium] Alert Emails Not sending
>>>>>
>>>>>
>>>>>
>>>>> Don't use remote SMTP It never works properly.
>>>>>
>>>>> Adam.
>>>>>
>>>>> Sent with AquaMail for Android
>>>>> http://www.aqua-mail.com
>>>>>
>>>>> On 26 August 2015 10:47:42 pm Cameron Murray <cameron@techpath.com.au> wrote:
>>>>>>
>>>>>> Guys,
>>>>>>
>>>>>>
>>>>>>
>>>>>> Only new to Observium and it’s a great product!
>>>>>>
>>>>>>
>>>>>>
>>>>>> Setting up the email alerts and testing from the cli works perfectly
>>>>>> however automated alerts configured via the interface do not send. The
>>>>>> alerts trigger in the interface and the contact groups are configured.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Test Email from cli:
>>>>>>
>>>>>>
>>>>>>
>>>>>> [root@obs observium]# ./test_mail.php -h bne-b1-lvl4-r1-apc2
>>>>>>
>>>>>> Observium 0.15.8.6920
>>>>>>
>>>>>> Test E-mail Configuration
>>>>>>
>>>>>>
>>>>>>
>>>>>> ##### Software versions #####
>>>>>>
>>>>>>
>>>>>>
>>>>>> o OS Linux 2.6.32-504.23.4.el6.x86_64 [amd64] (CentOS 6.7)
>>>>>>
>>>>>> o Apache 2.2.15
>>>>>>
>>>>>> o PHP 5.5.28
>>>>>>
>>>>>> o Python 2.6.6
>>>>>>
>>>>>> o MySQL 5.1.73 (extension: mysqli 5.1.73)
>>>>>>
>>>>>> o SNMP NET-SNMP 5.5
>>>>>>
>>>>>> o RRDtool 1.3.8
>>>>>>
>>>>>>
>>>>>>
>>>>>> ##### Charset info #####
>>>>>>
>>>>>>
>>>>>>
>>>>>> o PHP UTF-8
>>>>>>
>>>>>> o MySQL utf8
>>>>>>
>>>>>>
>>>>>>
>>>>>> ##### Timezones info #####
>>>>>>
>>>>>>
>>>>>>
>>>>>> o Date Wednesday, 26-Aug-15 21:43:01 UTC
>>>>>>
>>>>>> o PHP +10:00
>>>>>>
>>>>>> o MySQL +10:00
>>>>>>
>>>>>>
>>>>>>
>>>>>> See mail backend options here:
>>>>>> http://www.observium.org/wiki/Configuration_Options#Email_backend_settings
>>>>>>
>>>>>>
>>>>>>
>>>>>> My Hostname: obs.tech…..
>>>>>>
>>>>>> E-mails ENABLED globally ($config['email']['enable'])
>>>>>>
>>>>>> E-mails use PHP-builtin MAIL backend ($config['email']['backend'])
>>>>>>
>>>>>> E-mails send only to default email ($config['email']['default_only'])
>>>>>>
>>>>>> E-mails use these default recipient(s) ($config['email']['default']):
>>>>>>
>>>>>> alert-infrastructure@tech......
>>>>>>
>>>>>> E-mails send from ($config['email']['from']):
>>>>>>
>>>>>> monitoring@tech......
>>>>>>
>>>>>> E-mails send to recipients based on default recipient from config and/or
>>>>>> device sysContact.
>>>>>>
>>>>>> Try send test notification for bne-b1-lvl4-r1-apc2 to these email(s):
>>>>>>
>>>>>> alert-infrastructure@tech......
>>>>>>
>>>>>> EMAILINGNotification for bne-b1-lvl4-r1-apc2 SENT.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Config.php > Email Config
>>>>>>
>>>>>>
>>>>>>
>>>>>> // Enable alerter
>>>>>>
>>>>>> $config['poller-wrapper']['alerter'] = TRUE;
>>>>>>
>>>>>>
>>>>>>
>>>>>> $config['billing']['customer_autoadd'] = 0; # Enable Auto-add bill per
>>>>>> customer
>>>>>>
>>>>>> $config['billing']['circuit_autoadd'] = 0; # Enable Auto-add bill per
>>>>>> circuit_id
>>>>>>
>>>>>> $config['billing']['bill_autoadd'] = 0; # Enable Auto-add bill per bill_id
>>>>>>
>>>>>>
>>>>>>
>>>>>> $config['email']['sendmail_path'] = '/usr/sbin/sendmail';
>>>>>>
>>>>>>
>>>>>>
>>>>>> $config['alerts']['alerter']['default']['descr'] = "System Alert Change";
>>>>>>
>>>>>> $config['alerts']['alerter']['default']['type'] = "email";
>>>>>>
>>>>>> $config['alerts']['alerter']['default']['contact'] = "monitoring@tech......";
>>>>>>
>>>>>> $config['alerts']['alerter']['default']['enable'] = TRUE;
>>>>>>
>>>>>>
>>>>>>
>>>>>> $config['email']['default'] = "alert-infrastructure@tech...... ";
>>>>>>
>>>>>> $config['alerts']['email']['from'] = "monitoring@tech.....";
>>>>>>
>>>>>> $config['email']['default_only'] = TRUE;
>>>>>>
>>>>>>
>>>>>>
>>>>>> $config['email']['from'] = "monitoring@tech.....";
>>>>>>
>>>>>>
>>>>>>
>>>>>> $config['email']['smtp_host'] = 'mx01…….;
>>>>>>
>>>>>> $config['email']['smtp_port'] = 587;
>>>>>>
>>>>>> $config['email']['smtp_timeout'] = 10;
>>>>>>
>>>>>> $config['email']['smtp_secure'] = tls;
>>>>>>
>>>>>> $config['email']['smtp_auth'] = FALSE;
>>>>>>
>>>>>>
>>>>>>
>>>>>> Contact List configuration:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Any pointers are appreciated.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>>
>>>>>>
>>>>>> Cameron
>>>>>>
>>>>>> _______________________________________________
>>>>>> observium mailing list
>>>>>> observium@observium.org
>>>>>> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
>>>>>
>>>>> _______________________________________________
>>>>> observium mailing list
>>>>> observium@observium.org
>>>>> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
>>>>
>>>> _______________________________________________
>>>> observium mailing list
>>>> observium@observium.org
>>>> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
>>>
>>> _______________________________________________
>>> observium mailing list
>>> observium@observium.org
>>> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
>>>
>>
>> _______________________________________________
>> observium mailing list
>> observium@observium.org
>> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
> _______________________________________________
> observium mailing list
> observium@observium.org
> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
> _______________________________________________
> observium mailing list
> observium@observium.org
> http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium_______________________________________________
observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
