Dear Adam,

 

Would you like me to find out for certain ?

 

Best regards.

 

 

 

 

Youssef BENGELLOUN - ZAHR - Consultant Expert
Prodware France
T : +33 979 999 000 - F : +33 988 814 001 - ybzahr@prodware.fr

Web : prodware.fr

         

 

De : observium <observium-bounces@observium.org> au nom de Adam Armstrong <adama@memetic.org>
Répondre à : Observium <observium@observium.org>
Date : jeudi 31 août 2017 à 10:52
À : Observium <observium@observium.org>
Objet : Re: [Observium] Juniper SRX system uptime value

 

By default these devices seem incapable of reporting an uptime this large because of the counter size limitations.

This isn't uncommon, and we have many workarounds for different platforms. As far as I'm aware JunOS doesn't expose an oid capable of reporting uptime this long.

Adam.

Sent from BlueMail

On 31 Aug 2017, at 10:42, Youssef BENGELLOUN - ZAHR <ybzahr@prodware.fr> wrote:

Dear Observium community,

Is there any resolution regarding this issue ?

Best regards.



Le 18/07/2017 09:10, « Youssef BENGELLOUN - ZAHR » <ybzahr@prodware.fr> a écrit :

    Dear Observium community,

    Is there any resolution regarding this issue ?

    Best regards.



    Le 29/06/2017 21:40, « Youssef BENGELLOUN - ZAHR » <ybzahr@prodware.fr> a écrit :

        Dear Tom,

        We are over the 6 hours period grace. But thanks for the advice ;-)

        Best regards.



        Le 29/06/2017 21:33, « observium au nom de Tom Laermans » <observium-bounces@observium.org au nom de tom.laermans@powersource.cx> a écrit :

            Hi Youssef,

            For guaranteed response time within 6 hours, please check with Adam for
            a support contract (I assume this will be a five figure per month deal).

            Tom

            On 29/06/2017 21:01, Youssef BENGELLOUN - ZAHR wrote:
 Dear Mike,

What is your opinion on this ?

Best regards.



Le 29/06/2017 13:45, « Youssef BENGELLOUN - ZAHR » <ybzahr@prodware.fr> a écrit :

      Here is it :


      root@prod-monit-002:/opt/observium# ./poller.php -d -m system -h fw01-par01
      DEBUG!

      CMD[/usr/bin/svn info /opt/observium]

      CMD EXITCODE[0]
     CMD RUNTIME[0.0134s]
      STDOUT[
      Path: .
      Working Copy Root Path: /opt/observium
      URL: http://svn.observium.org/svn/observium/branches/stable
      Relative URL: ^/observium/branches/stable
      Repository Root: http://svn.observium.org/svn
      Repository UUID: 61d68cd4-352d-0410-923a-c4978735b2b8
      Revision: 8603
      Node Kind: directory
      Schedule: normal
      Last Changed Author: adama
      Last Changed Rev: 8580
      Last Changed Date: 2017-05-28 14:05:18 +0200 (Sun, 28 May 2017)

      ]

      SQL[SELECT `attrib_value` FROM `observium_attribs` WHERE `attrib_type` = 'dbSchema';]
      SQL RUNTIME[0.00034308s]

      SQL[SELECT @@SESSION.sql_mode;]
      SQL RUNTIME[0.00013804s]

      SQL[SELECT `attrib_value` FROM `observium_attribs` WHERE `attrib_type` = 'current_rev';]
      SQL RUNTIME[0.00027895s]

      SQL[SELECT * FROM `config`]
      SQL RUNTIME[0.00027394s]

      CMD[/bin/hostname -f]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0019s]
      STDOUT[
      observium.dom.prodware.net
      ]

        ___   _                              _
       / _ \ | |__   ___   ___  _ __ __   __(_) _   _  _ __ ___
      | | | || '_ \ / __| / _ \| '__|\ \ / /| || | | || '_ ` _ \
      | |_| || |_) |\__ \|  __/| |    \ V / | || |_| || | | | | |
       \___/ |_.__/ |___/ \___||_|     \_/  |_| \__,_||_| |_| |_|
                                 Observium Professional 17.5.8580
                                         http://www.observium.org


      SQL[SELECT `attrib_value` FROM `observium_attribs` WHERE `attrib_type` = 'latest_ver';]
      SQL RUNTIME[0.00048208s]

      SQL[SELECT `attrib_value` FROM `observium_attribs` WHERE `attrib_type` = 'latest_rev';]
      SQL RUNTIME[0.00015903s]

      SQL[SELECT `attrib_value` FROM `observium_attribs` WHERE `attrib_type` = 'latest_rev_date';]
      SQL RUNTIME[0.00013900s]

      CMD[/opt/observium/scripts/distro]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0087s]
      STDOUT[
      Linux|4.4.0-81-generic|amd64|Ubuntu|16.04|vmware
      ]

      CMD[/usr/bin/env python --version 2>&1]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0016s]
      STDOUT[
      Python 2.7.12
      ]

      SQL[SELECT version();]
      SQL RUNTIME[0.00021601s]

      CMD[/usr/bin/snmpget --version 2>&1]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0032s]
      STDOUT[
      NET-SNMP version: 5.7.3
      ]

      CMD[/usr/bin/rrdtool --version | head -n1]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0074s]
      STDOUT[
      RRDtool 1.5.5  Copyright by Tobias Oetiker <tobi@oetiker.ch>
      ]

      CMD[/usr/bin/env apache2 -v | awk '/Server version:/ {print $3}']

      CMD EXITCODE[0]
      CMD RUNTIME[0.0239s]
      STDOUT[
      Apache/2.4.18
      ]

      CMD[date "+%:z"]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0017s]
      STDOUT[
      +02:00
      ]

      SQL[SELECT TIMEDIFF(NOW(), UTC_TIMESTAMP);]
      SQL RUNTIME[0.00028801s]

      SQL[SELECT @@SESSION.sql_mode;]
      SQL RUNTIME[0.00012112s]

      SQL[SHOW VARIABLES LIKE 'character_set_connection']
      SQL RUNTIME[0.00192690s]

      #####  Software versions  #####

       o OS                   Linux 4.4.0-81-generic [amd64] (Ubuntu 16.04)
       o Apache               2.4.18
       o PHP                  7.0.18-0ubuntu0.16.04.1 (OPcache: DISABLED)
       o Python               2.7.12
       o MySQL                5.7.18-0ubuntu0.16.04.1 (extension: mysqli 5.0.12-dev)
       o SNMP                 NET-SNMP 5.7.3
       o RRDtool              1.5.5

      #####  Memory Limit  #####

       o PHP                  Unlimited

      #####  MySQL mode  #####

       o MySQL

      #####  Charset info  #####

       o PHP                  UTF-8
       o MySQL                utf8

      #####  Timezones info  #####

       o Date                 Thursday, 29-Jun-17 13:44:08 CEST
       o PHP                  +02:00
       o MySQL                +02:00


      SQL[SELECT * FROM `alerts_maint` WHERE `maint_start` < '1498736648' AND `maint_end` > '1498736648']
      SQL RUNTIME[0.00048304s]
      #####  Starting polling run at 2017-06-29 13:44:08  #####


      SQL[SELECT `device_id` FROM `devices` WHERE `disabled` = 0 AND `hostname` LIKE 'fw01-par01' ORDER BY `device_id` ASC]
      SQL RUNTIME[0.00038719s]

      SQL[SELECT * FROM `devices` WHERE `device_id` = '80']
      SQL RUNTIME[0.00030804s]

      SQL[SELECT * FROM `entity_attribs` WHERE `entity_type` = 'device' AND `entity_id` = '80']
      SQL RUNTIME[0.00026989s]

      SQL[SELECT * FROM `observium_processes` WHERE `process_name` = 'poller.php' AND `device_id` = '80']
      SQL RUNTIME[0.00020790s]

     CMD[/bin/ps -ww -o pid,ppid,uid,gid,tty,stat,time,lstart,args -p 1042]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0092s]
      STDOUT[
        PID  PPID   UID   GID TT       STAT     TIME                  STARTED COMMAND
       1042   802     0     0 pts/0    S+   00:00:00 Thu Jun 29 13:44:07 2017 php ./poller.php -d -m system -h fw01-par01
      ]

      CMD[/bin/ps -ww -o pid,ppid,uid,gid,tty,stat,time,lstart,args -p 802]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0087s]
      STDOUT[
        PID  PPID   UID   GID TT       STAT     TIME                  STARTED COMMAND
        802   702     0     0 pts/0    Ss   00:00:00 Thu Jun 29 13:43:25 2017 -bash
      ]

      SQL[INSERT INTO `observium_processes` (`process_pid`,`process_name`,`process_ppid`,`process_uid`,`process_command`,`process_start`,`device_id`)  VALUES ('1042','poller.php','802','0','php ./poller.php -d -m system -h fw01-par01','1498736647','80')]
      SQL RUNTIME[0.00167203s]

      SQL[SELECT * FROM `alert_tests` WHERE 1]
      SQL RUNTIME[0.00035214s]
      Cached 29 alert rules.

      SQL[SELECT * FROM  `alert_table` WHERE `device_id` = '80']
      SQL RUNTIME[0.00130606s]
      #####  fw01-par01 [80]  #####

       o OS                   junos
       o Last poll duration   61.74 seconds
       o Last Polled          2017-06-29 13:41:03
       o SNMP Version         v2c

      CMD[/usr/bin/fping -t 500 -c 1 -q 10.75.1.5 2>&1]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0033s]
      STDOUT[
      10.75.1.5 : xmt/rcv/%loss = 1/1/0%, min/avg/max = 1.60/1.60/1.60
      ]

      CMD[/usr/bin/snmpget -v2c -c *** -Pu -OQUst -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'fw01-par01':'161' sysObjectID.0 sysUpTime.0]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0061s]
      STDOUT[
      sysObjectID.0 = enterprises.2636.1.1.1.2.28
      sysUpTime.0 = 339223303
      ]
      SNMP STATUS[TRUE]
       o Device status        Device is reachable by PING (1.60ms) and SNMP (6.12ms)
      RRD /opt/observium/rrd/fw01-par01/status.rrd already exists - no need to create.

      RRD CMD[update /opt/observium/rrd/fw01-par01/status.rrd N:1]
      RRD RUNTIME[0.0011s]
      RRD STDOUT[OK u:0.00 s:0.00 r:0.04]
      RRD_STATUS[TRUE]

      RRD /opt/observium/rrd/fw01-par01/ping.rrd already exists - no need to create.

      RRD CMD[update /opt/observium/rrd/fw01-par01/ping.rrd N:1.60]
      RRD RUNTIME[0.0011s]
      RRD STDOUT[OK u:0.00 s:0.00 r:0.04]
      RRD_STATUS[TRUE]

      RRD /opt/observium/rrd/fw01-par01/ping_snmp.rrd already exists - no need to create.

      RRD CMD[update /opt/observium/rrd/fw01-par01/ping_snmp.rrd N:6.12]
      RRD RUNTIME[0.0011s]
      RRD STDOUT[OK u:0.00 s:0.00 r:0.04]
      RRD_STATUS[TRUE]


      SQL[SELECT * FROM `device_graphs` WHERE `device_id` = '80']
      SQL RUNTIME[0.00041080s]
       o Modules Enabled      system, os


      including: includes/polling/system.inc.php
      #####  Module Start: system  #####


      SQL[SELECT `attrib_value` FROM `entity_attribs` WHERE `entity_type` = 'device' AND `entity_id` = '80' AND `attrib_type` = 'sysORID']
      SQL RUNTIME[0.00021601s]

      SQL[SELECT * FROM `observium_attribs`]
      SQL RUNTIME[0.00017405s]
       o SNMPv2-MIB
      CMD[/usr/bin/snmpget -v2c -c *** -Pu -OQUs -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'fw01-par01':'161' sysUpTime.0 sysLocation.0 sysContact.0 sysName.0]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0058s]
      STDOUT[
      sysUpTime.0 = 39:6:17:13.04
      sysLocation.0 = Interxion PAR05 (PAR01)
      sysContact.0 = noc@prodware.fr
      sysName.0 = fw01-par01
      ]
      SNMP STATUS[TRUE]

      CMD[/usr/bin/snmpget -v2c -c *** -Pu -Oqv -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'fw01-par01':'161' sysDescr.0]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0056s]
      STDOUT[
      Juniper Networks, Inc. srx5600 internet router, kernel JUNOS 12.3X48-D20.4, Build date: 2015-11-04 20:40:42 UTC Copyright (c) 1996-2015 Juniper Networks, Inc.
      ]
      SNMP STATUS[TRUE]

      CMD[/usr/bin/snmpget -v2c -c *** -Pu -Oqvn -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'fw01-par01':'161' sysObjectID.0]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0055s]
      STDOUT[
      .1.3.6.1.4.1.2636.1.1.1.2.28
      ]
      SNMP STATUS[TRUE]

      CMD[/usr/bin/snmpget -v2c -c *** -Pu -Ovqn -m SNMP-FRAMEWORK-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'fw01-par01':'161' snmpEngineID.0]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0071s]
      STDOUT[
      "80 00 0A 4C 01 0A 4B 01 05 "
      ]
      SNMP STATUS[TRUE]


      CMD[/usr/bin/snmpget -v2c -c *** -Pu -Oqv -m HOST-RESOURCES-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'fw01-par01':'161' hrSystemUptime.0]

      CMD EXITCODE[0]
      CMD RUNTIME[0.0081s]
      STDOUT[
      39:10:55:27.58
      ]
      SNMP STATUS[TRUE]
      Using SNMP Agent hrSystemUptime (3408927 sec. => 39 days, 10h 55m 27s)
      RRD /opt/observium/rrd/fw01-par01/uptime.rrd already exists - no need to create.

      RRD CMD[update /opt/observium/rrd/fw01-par01/uptime.rrd N:3408927]
      RRD RUNTIME[0.0011s]
      RRD STDOUT[OK u:0.00 s:0.00 r:0.08]
      RRD_STATUS[TRUE]

       o Uptime               39 days, 10h 55m 27s


        /opt/observium/includes/polling/system.inc.php:253
      array(
        [sysUpTime]      => int(3392233)
        [hrSystemUptime] => int(3408927)
        [use]            => string(14) "hrSystemUptime"
        [uptime]         => int(3408927)
        [formatted]      => string(20) "39 days, 10h 55m 27s"
        [message]        => string(31) "Using SNMP Agent hrSystemUptime"
        [previous]       => string(7) "3408681"
        [diff]           => int(-246)
        [rebooted]       => int(0)
      )
       o sysObjectID          .1.3.6.1.4.1.2636.1.1.1.2.28
       o snmpEngineID         80000A4C010A4B0105
       o sysDescr             Juniper Networks, Inc. srx5600 internet router, kernel JUNOS 12.3X48-D20.4, Build date: 2015-11-04 20:40:42 UTC Copyright (c)
                              1996-2015Juniper Networks, Inc.
       o sysName              fw01-par01
       o Location             Interxion PAR05 (PAR01)

      SQL[SELECT * FROM `devices_locations` WHERE `device_id` = '80']
      SQL RUNTIME[0.00042701s]
       o Module time          0.0443s


      including: includes/polling/os.inc.php
      #####  Module Start: os  #####


      CMD[/usr/bin/snmpget -v2c -c *** -Pu -Oqv -m JUNIPER-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/juniper 'udp':'fw01-par01':'161' jnxBoxSerialNo.0]

      CMD EXITCODE[0]
      CMD RUNTIME[0.093s]
      STDOUT[
      JN11FD5FBAGB
      ]
      SNMP STATUS[TRUE]
      Added OS param from SNMP definition walk: 'serial' = 'JN11FD5FBAGB'
       o OS Poller            OS
       o Hardware             SRX5600
       o Version              12.3X48-D20.4
       o Features             Internet Router
       o Serial               JN11FD5FBAGB
       o Asset                <empty>

       o Module time          0.0958s

      #####  fw01-par01 [80] completed poller modules at 2017-06-29 13:44:08  #####

       o Graphs [checked]     ping, ping_snmp, uptime
       o Poller time          0.1583 seconds
      Updating fw01-par01 -
 $update_array /opt/observium/includes/polling/functions.inc.php:749
      array(
        [uptime]                => int(3408927)
        [last_polled]           => array(
                                     [0] => string(5) "NOW()"
                                   )
        [last_polled_timetaken] => double(0.1583)
        [device_state]          => string(103) "a:1:{s:15:"poller_mod_perf";a:2:{s:6:"system";d:0.044299999999999999;s:2:"os";d:0.095799999999999996;}}"
      )


      SQL[UPDATE `devices` set `uptime` ='3408927',`last_polled` =NOW(),`last_polled_timetaken` ='0.1583',`device_state` ='a:1:{s:15:\"poller_mod_perf\";a:2:{s:6:\"system\";d:0.044299999999999999;s:2:\"os\";d:0.095799999999999996;}}' WHERE `device_id` = '80']
      SQL RUNTIME[0.00172496s]
       o Updated Data         uptime, last_polled, last_polled_timetaken, device_state

 $data /opt/observium/includes/alerts.inc.php:61
      array(
        [device_status]        => string(1) "1"
        [device_status_type]   => string(0) ""
        [device_ping]          => string(4) "1.60"
        [device_snmp]          => string(4) "6.12"
        [device_uptime]        => string(7) "3408681"
        [device_rebooted]      => int(0)
        [device_duration_poll] => string(5) "61.74"
      )

      SQL[UPDATE `alert_table` set `state` ='{\"metrics\":{\"device_status\":\"1\"}}',`count` ='0',`alert_status` ='1',`last_message` ='Checks OK',`last_checked` ='1498736648',`last_ok` ='1498736648' WHERE `alert_table_id` = '20']
      SQL RUNTIME[0.00062895s]
      RRD /opt/observium/rrd/fw01-par01/alert-20.rrd already exists - no need to create.

      RRD CMD[update /opt/observium/rrd/fw01-par01/alert-20.rrd N:1:1]
      RRD RUNTIME[0.0011s]
      RRD STDOUT[OK u:0.00 s:0.00 r:0.18]
      RRD_STATUS[TRUE]


      SQL[UPDATE `alert_table` set `state` ='{\"metrics\":{\"mempool_perc\":null}}',`count` ='0',`alert_status` ='1',`last_message` ='Checks OK',`last_checked` ='1498736648',`last_ok` ='1498736648' WHERE `alert_table_id` = '2231']
      SQL RUNTIME[0.00122786s]
      RRD /opt/observium/rrd/fw01-par01/alert-2231.rrd already exists - no need to create.

      RRD CMD[update /opt/observium/rrd/fw01-par01/alert-2231.rrd N:1:1]
      RRD RUNTIME[0.0011s]
      RRD STDOUT[OK u:0.00 s:0.00 r:0.19]
      RRD_STATUS[TRUE]



      SQL[DELETE FROM `observium_processes` WHERE `process_pid` = '1042' AND `process_name` = 'poller.php' AND `device_id` = '80']
      SQL RUNTIME[0.00122595s]

      SQL[INSERT INTO `perf_times` (`type`,`doing`,`start`,`duration`,`devices`)  VALUES ('poll','fw01-par01','1498736648.066','0.270','1')]
      SQL RUNTIME[0.00156879s]
      ./poller.php: fw01-par01 - 1 devices polled in 0.270 secs
      #####  Completed polling run at 2017-06-29 13:44:08  #####

      NOTE, $config['snmp']['hide_auth'] is set to TRUE, snmp community and snmp v3 auth hidden from debug output.
       o Devices Polled       1
       o Poller Time          0.270 secs
       o Memory usage         12MB (peak: 12MB)
       o MySQL Usage          Insert[2/0.004s] Update[3/0.004s] Delete[1/0.001s] Cell[4/0.001s] Row[8/0.004s] Rows[10/0.008s] Col[0/0s] (0.022s
                              8.091%)
       o RRDTool Usage        update[6/0.007s] (0.007s 2.417%)
       o SNMP Usage           snmpget[7/0.139s] (0.139s 51.62%)




      Best regards.




      Le 29/06/2017 12:22, « observium au nom de Mike Stupalov » <observium-bounces@observium.org au nom de mike@observium.org> a écrit :

          For this device should be used snmpEngineTime.0 (46335482 sec = 536 days
          6 hours 58 min 2 sec),
          please attach full debug output for:

          ./poller.php -d -m system -h <device>

          Youssef BENGELLOUN - ZAHR wrote:
 There you go :

root@prod-monit-002:~# snmpget -v 2c -c XXX -Ot ABCD  HOST-RESOURCES-MIB::hrSystemUptime.0
HOST-RESOURCES-MIB::hrSystemUptime.0 = 340248284

root@prod-monit-002:~# snmpget -v 2c -c XXX -Ot ABCD SNMP-FRAMEWORK-MIB::snmpEngineTime.0
SNMP-FRAMEWORK-MIB::snmpEngineTime.0 = INTEGER: 46335482 seconds

Best regards.



Le 29/06/2017 11:47, « observium au nom de Mike Stupalov » <observium-bounces@observium.org au nom de mike@observium.org> a écrit :

     Show output for:

     $ snmpget -v2c -c <community> -Ot <device>
     HOST-RESOURCES-MIB::hrSystemUptime.0

     $ snmpget -v2c -c <community> -Ot <device>
     SNMP-FRAMEWORK-MIB::snmpEngineTime.0

     Michael wrote:
 That would be the uptime of the SNMP engine/daemon of the device.

They usually match on most sane devices.  500+ days, your miles may vary...

Michael



 On 29 Jun 2017, at 7:36 pm, Youssef BENGELLOUN - ZAHR <ybzahr@prodware.fr> wrote:


Dear all,

Runnig an SRX5600 cluster with version 12.3X48-D20.4. I noticed that displayed system uptime value doesn’t match :

<image001.png>

via CLI :

node0:


Current time: 2017-06-29 11:18:08 CEST
System booted: 2016-01-09 23:20:47 CET (76w4d 10:57 ago)
Protocols started: 2016-01-10 03:58:57 CET (76w4d 06:19 ago)
11:18AM  up 536 days, 10:57, 2 users, load averages: 0.59, 0.34, 0.30

node1:


Current time: 2017-06-29 11:18:08 CEST
System booted: 2016-01-09 23:20:47 CET (76w4d 10:57 ago)
11:18AM  up 536 days, 10:57, 0 users, load averages: 0.00, 0.00, 0.00

I tried re-discovering the device, no changes. This is on latest stable release 8580. I tried to run a debug, this is what I get :

CMD[[0;33m/usr/bin/snmpget -v2c -c *** -Pu -OQUst -m SNMPv2-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'fw01-par01':'161' sysObjectID.0 sysUpTime.0[0m]
[0m
CMD EXITCODE[[0;32m0[0m]
CMD RUNTIME[[0;32m0.0063s[0m][0m
STDOUT[
sysObjectID.0 = enterprises.2636.1.1.1.2.28
sysUpTime.0 = 338397151
][0m
SNMP STATUS[[0;32mTRUE[0m][0m


Anyone noticed this ?

Best regards.


Youssef BENGELLOUN - ZAHR - Consultant Expert
Prodware France
T : +33 979 999 000 - F : +33 988 814 001 - ybzahr@prodware.fr
<image713928.PNG>Web : prodware.fr

<imagedece58.JPG>  <image6916e1.JPG>  <imaged404a4.JPG>  <image975500.JPG>  <image408de1.JPG>  <image199c7c.JPG>




BENGELLOUN - ZAHR Youssef - Consultant Expert
Prodware France
T : +33 979 999 000  F : +33 988 814 001 - ybzahr@prodware.fr
Web : prodware.fr



BENGELLOUN - ZAHR Youssef - Consultant Expert
Prodware France
T : +33 979 999 000  F : +33 988 814 001 - ybzahr@prodware.fr
Web : prodware.fr



BENGELLOUN - ZAHR Youssef - Consultant Expert
Prodware France
T : +33 979 999 000  F : +33 988 814 001 - ybzahr@prodware.fr
Web : prodware.fr

 
 observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
 


observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

     --
     Mike Stupalov
     Observium Limited, http://observium.org


     observium mailing list
     observium@observium.org
     http://postman.memetic.org/cgi-bin/mailman/listinfo/observium



observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium

          --
          Mike Stupalov
          Observium Limited, http://observium.org


          observium mailing list
          observium@observium.org
          http://postman.memetic.org/cgi-bin/mailman/listinfo/observium





observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
 


            observium mailing list
            observium@observium.org
            http://postman.memetic.org/cgi-bin/mailman/listinfo/observium







observium mailing list
observium@observium.org
http://postman.memetic.org/cgi-bin/mailman/listinfo/observium