First, I just took over our install of this, wasn't involved in the initial setup of observium and I understand the piece I'm asking about is not part of observium, just uses the data it collects and I want to verify if structure has changed since it was released or if there is something incorrectly setup with our system.
I'm looking to build out graphviz maps with the data observium has gathered, specifically using: https://github.com/kedare/observium_map_generator
Going through their code, it queries the db for the links table and fails. Looking at the database I have a neighbours table, but not links. Google search I see links table discussed back in 2014, but nothing much recently.
So, did the links table go away in newer versions, or is the fact that the table is missing an issue with our setup? If it is supposed to be there and missing, I'll work on figuring that out next, if it did just go away I just want to verify that.
Thanks
Hi Eric,
I rewrote the query that script uses and it's saved to a text file. I've not gotten around to rewriting the rest of the code in php yet,since I don't have a CDP network to test on.
I can dig out the SQL when I'm not driving :) (bathroom stop, I can't write email and drive! ;))
Adam.
Sent from BlueMail
On 1 Nov 2016, 16:11, at 16:11, Eric Kollmann xnih13@gmail.com wrote:
First, I just took over our install of this, wasn't involved in the initial setup of observium and I understand the piece I'm asking about is not part of observium, just uses the data it collects and I want to verify if structure has changed since it was released or if there is something incorrectly setup with our system.
I'm looking to build out graphviz maps with the data observium has gathered, specifically using: https://github.com/kedare/observium_map_generator
Going through their code, it queries the db for the links table and fails. Looking at the database I have a neighbours table, but not links. Google search I see links table discussed back in 2014, but nothing much recently.
So, did the links table go away in newer versions, or is the fact that the table is missing an issue with our setup? If it is supposed to be there and missing, I'll work on figuring that out next, if it did just go away I just want to verify that.
Thanks
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Perfect, happy to test it out if/when you get a chance to send it on. If php stuff needs tweaked I'll see what I can come up with there.
Eric
On Tue, Nov 1, 2016 at 12:06 PM, Adam Armstrong adama@memetic.org wrote:
Hi Eric,
I rewrote the query that script uses and it's saved to a text file. I've not gotten around to rewriting the rest of the code in php yet,since I don't have a CDP network to test on.
I can dig out the SQL when I'm not driving :) (bathroom stop, I can't write email and drive! ;))
Adam.
Sent from BlueMail http://www.bluemail.me/r
On 1 Nov 2016, at 16:11, Eric Kollmann xnih13@gmail.com wrote:
First, I just took over our install of this, wasn't involved in the initial setup of observium and I understand the piece I'm asking about is not part of observium, just uses the data it collects and I want to verify if structure has changed since it was released or if there is something incorrectly setup with our system.
I'm looking to build out graphviz maps with the data observium has gathered, specifically using: https://github.com/kedare/observium_map_generator
Going through their code, it queries the db for the links table and fails. Looking at the database I have a neighbours table, but not links. Google search I see links table discussed back in 2014, but nothing much recently.
So, did the links table go away in newer versions, or is the fact that the table is missing an issue with our setup? If it is supposed to be there and missing, I'll work on figuring that out next, if it did just go away I just want to verify that.
Thanks
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Once you verified it had to be changed and indicated links table is no longer there, think I got it updated to work now:
have to actually remove the 2 lines starting with //, but basically changed links to neighbours and updated l.local_port_id to l.port_id below. Running this with 1100+ devices in observium kicks out a really large map, but got to say great product!
SELECT UPPER(d.hostname) AS local_hostname, p.port_id AS local_port_id, p.ifName AS local_port, p.ifSpeed/1000000 as local_port_speed, l.remote_port_id AS remote_port_id, rp.ifName AS remote_port, UPPER(l.remote_hostname) AS remote_hostname, d.hardware AS hardware, rd.hardware AS remote_hardware, s.ifInErrors_rate AS local_in_errors_rate, s.ifOutErrors_rate AS local_out_errors_rate, s.ifInOctets_rate*8 AS local_in_octets_rate, s.ifOutOctets_rate*8 AS local_out_octets_rate, s.ifInOctets_perc AS local_in_octets_perc, s.ifOutOctets_perc AS local_out_octets_perc FROM `devices` AS d, `devices` as rd, `ports` AS p, `ports` as rp, // `links` AS l, `neighbours` AS l, `ports-state` AS s WHERE // p.port_id = l.local_port_id AND p.port_id = l.port_id AND p.device_id = d.device_id AND rp.port_id = l.remote_port_id AND rp.device_id = rd.device_id AND p.port_id = s.port_id AND UPPER(d.hostname) = UPPER(%s); """
On Tue, Nov 1, 2016 at 12:50 PM, Eric Kollmann xnih13@gmail.com wrote:
Perfect, happy to test it out if/when you get a chance to send it on. If php stuff needs tweaked I'll see what I can come up with there.
Eric
On Tue, Nov 1, 2016 at 12:06 PM, Adam Armstrong adama@memetic.org wrote:
Hi Eric,
I rewrote the query that script uses and it's saved to a text file. I've not gotten around to rewriting the rest of the code in php yet,since I don't have a CDP network to test on.
I can dig out the SQL when I'm not driving :) (bathroom stop, I can't write email and drive! ;))
Adam.
Sent from BlueMail http://www.bluemail.me/r
On 1 Nov 2016, at 16:11, Eric Kollmann xnih13@gmail.com wrote:
First, I just took over our install of this, wasn't involved in the initial setup of observium and I understand the piece I'm asking about is not part of observium, just uses the data it collects and I want to verify if structure has changed since it was released or if there is something incorrectly setup with our system.
I'm looking to build out graphviz maps with the data observium has gathered, specifically using: https://github.com/kedare/observium_map_generator
Going through their code, it queries the db for the links table and fails. Looking at the database I have a neighbours table, but not links. Google search I see links table discussed back in 2014, but nothing much recently.
So, did the links table go away in newer versions, or is the fact that the table is missing an issue with our setup? If it is supposed to be there and missing, I'll work on figuring that out next, if it did just go away I just want to verify that.
Thanks
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Could you provide instruction show to get this to work, please?
This is the one addon I really would like to see working.
Wolfgang
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Eric Kollmann Sent: Tuesday, November 1, 2016 21:44 To: Observium Network Observation System observium@observium.org Subject: Re: [Observium] Tables in the Observium DB
Once you verified it had to be changed and indicated links table is no longer there, think I got it updated to work now:
have to actually remove the 2 lines starting with //, but basically changed links to neighbours and updated l.local_port_id to l.port_id below. Running this with 1100+ devices in observium kicks out a really large map, but got to say great product!
SELECT UPPER(d.hostname) AS local_hostname, p.port_id AS local_port_id, p.ifName AS local_port, p.ifSpeed/1000000 as local_port_speed, l.remote_port_id AS remote_port_id, rp.ifName AS remote_port, UPPER(l.remote_hostname) AS remote_hostname, d.hardware AS hardware, rd.hardware AS remote_hardware, s.ifInErrors_rate AS local_in_errors_rate, s.ifOutErrors_rate AS local_out_errors_rate, s.ifInOctets_rate*8 AS local_in_octets_rate, s.ifOutOctets_rate*8 AS local_out_octets_rate, s.ifInOctets_perc AS local_in_octets_perc, s.ifOutOctets_perc AS local_out_octets_perc FROM `devices` AS d, `devices` as rd, `ports` AS p, `ports` as rp, // `links` AS l, `neighbours` AS l, `ports-state` AS s WHERE // p.port_id = l.local_port_id AND p.port_id = l.port_id AND p.device_id = d.device_id AND rp.port_id = l.remote_port_id AND rp.device_id = rd.device_id AND p.port_id = s.port_id AND UPPER(d.hostname) = UPPER(%s); """
On Tue, Nov 1, 2016 at 12:50 PM, Eric Kollmann <xnih13@gmail.commailto:xnih13@gmail.com> wrote: Perfect, happy to test it out if/when you get a chance to send it on. If php stuff needs tweaked I'll see what I can come up with there.
Eric
On Tue, Nov 1, 2016 at 12:06 PM, Adam Armstrong <adama@memetic.orgmailto:adama@memetic.org> wrote: Hi Eric, I rewrote the query that script uses and it's saved to a text file. I've not gotten around to rewriting the rest of the code in php yet,since I don't have a CDP network to test on. I can dig out the SQL when I'm not driving :) (bathroom stop, I can't write email and drive! ;)) Adam.
Sent from BlueMailhttp://www.bluemail.me/r On 1 Nov 2016, at 16:11, Eric Kollmann <xnih13@gmail.commailto:xnih13@gmail.com> wrote: First, I just took over our install of this, wasn't involved in the initial setup of observium and I understand the piece I'm asking about is not part of observium, just uses the data it collects and I want to verify if structure has changed since it was released or if there is something incorrectly setup with our system.
I'm looking to build out graphviz maps with the data observium has gathered, specifically using: https://github.com/kedare/observium_map_generator
Going through their code, it queries the db for the links table and fails. Looking at the database I have a neighbours table, but not links. Google search I see links table discussed back in 2014, but nothing much recently.
So, did the links table go away in newer versions, or is the fact that the table is missing an issue with our setup? If it is supposed to be there and missing, I'll work on figuring that out next, if it did just go away I just want to verify that.
Thanks
________________________________
observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
If I run this query against my database I get and empty set, but I know that the device has lots of CDP entries
SELECT UPPER(d.hostname) AS local_hostname, p.port_id AS local_port_id, p.ifName AS local_port, p.ifSpeed/1000000 as local_port_speed, l.remote_port_id AS remote_port_id, rp.ifName AS remote_port, UPPER(l.remote_hostname) AS remote_hostname, d.hardware AS hardware, rd.hardware AS remote_hardware, s.ifInErrors_rate AS local_in_errors_rate, s.ifOutErrors_rate AS local_out_errors_rate, s.ifInOctets_rate*8 AS local_in_octets_rate, s.ifOutOctets_rate*8 AS local_out_octets_rate, s.ifInOctets_perc AS local_in_octets_perc, s.ifOutOctets_perc AS local_out_octets_perc FROM `devices` AS d, `devices` as rd, `ports` AS p, `ports` as rp, `neighbours` AS l, `ports-state` AS s WHERE p.port_id = l.port_id AND p.device_id = d.device_id AND rp.port_id = l.remote_port_id AND rp.device_id = rd.device_id AND p.port_id = s.port_id AND UPPER(d.hostname) = UPPER('GEALICRDC03');
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Vater, Wolfgang Sent: Monday, November 7, 2016 17:27 To: Observium Network Observation System observium@observium.org Subject: Re: [Observium] Tables in the Observium DB
This sender failed our fraud detection checks and may not be who they appear to be. Learn about spoofinghttp://aka.ms/LearnAboutSpoofing
Feedbackhttp://aka.ms/SafetyTipsFeedback
Could you provide instruction show to get this to work, please?
This is the one addon I really would like to see working.
Wolfgang
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Eric Kollmann Sent: Tuesday, November 1, 2016 21:44 To: Observium Network Observation System <observium@observium.orgmailto:observium@observium.org> Subject: Re: [Observium] Tables in the Observium DB
Once you verified it had to be changed and indicated links table is no longer there, think I got it updated to work now:
have to actually remove the 2 lines starting with //, but basically changed links to neighbours and updated l.local_port_id to l.port_id below. Running this with 1100+ devices in observium kicks out a really large map, but got to say great product!
SELECT UPPER(d.hostname) AS local_hostname, p.port_id AS local_port_id, p.ifName AS local_port, p.ifSpeed/1000000 as local_port_speed, l.remote_port_id AS remote_port_id, rp.ifName AS remote_port, UPPER(l.remote_hostname) AS remote_hostname, d.hardware AS hardware, rd.hardware AS remote_hardware, s.ifInErrors_rate AS local_in_errors_rate, s.ifOutErrors_rate AS local_out_errors_rate, s.ifInOctets_rate*8 AS local_in_octets_rate, s.ifOutOctets_rate*8 AS local_out_octets_rate, s.ifInOctets_perc AS local_in_octets_perc, s.ifOutOctets_perc AS local_out_octets_perc FROM `devices` AS d, `devices` as rd, `ports` AS p, `ports` as rp, // `links` AS l, `neighbours` AS l, `ports-state` AS s WHERE // p.port_id = l.local_port_id AND p.port_id = l.port_id AND p.device_id = d.device_id AND rp.port_id = l.remote_port_id AND rp.device_id = rd.device_id AND p.port_id = s.port_id AND UPPER(d.hostname) = UPPER(%s); """
On Tue, Nov 1, 2016 at 12:50 PM, Eric Kollmann <xnih13@gmail.commailto:xnih13@gmail.com> wrote: Perfect, happy to test it out if/when you get a chance to send it on. If php stuff needs tweaked I'll see what I can come up with there.
Eric
On Tue, Nov 1, 2016 at 12:06 PM, Adam Armstrong <adama@memetic.orgmailto:adama@memetic.org> wrote: Hi Eric, I rewrote the query that script uses and it's saved to a text file. I've not gotten around to rewriting the rest of the code in php yet,since I don't have a CDP network to test on. I can dig out the SQL when I'm not driving :) (bathroom stop, I can't write email and drive! ;)) Adam.
Sent from BlueMailhttp://www.bluemail.me/r On 1 Nov 2016, at 16:11, Eric Kollmann <xnih13@gmail.commailto:xnih13@gmail.com> wrote: First, I just took over our install of this, wasn't involved in the initial setup of observium and I understand the piece I'm asking about is not part of observium, just uses the data it collects and I want to verify if structure has changed since it was released or if there is something incorrectly setup with our system.
I'm looking to build out graphviz maps with the data observium has gathered, specifically using: https://github.com/kedare/observium_map_generator
Going through their code, it queries the db for the links table and fails. Looking at the database I have a neighbours table, but not links. Google search I see links table discussed back in 2014, but nothing much recently.
So, did the links table go away in newer versions, or is the fact that the table is missing an issue with our setup? If it is supposed to be there and missing, I'll work on figuring that out next, if it did just go away I just want to verify that.
Thanks
________________________________
observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Here is what I did
1. Modify the script, the 2 changes discussed already and one other change to write out to a .dot file, instead of a .png (notes at bottom). Also, this is just looking at cdp and whatever else is in the neighbors table, it is not looking or building a graph based on OSPF which I believe there is a table for as well, but I have not had time to investigate. 2. Per the documentation, I created a graph directory under html, so in my case /opt/observium/html/graph 3. Dropped the map.php and the viz.js file in that directory (I ended up grabbing it from here: https://github.com/mdaines/viz.js/releases, unknown if there are better locations, but developer on that seems to have stopped it). 4. Modified map.php to point at the .dot file that I generated, we'll say for example: myservers.mydomain.org_map.dot (It is CHANGEME or something like that in the original map.php). 5. Then ran the following command: sudo python /opt/observium/graph.py myservers.mydomain.org (where myservers.mydomain.org is a full DNS entry that exists in your setup for what you want to be the main node so to say).
6. Visit your server/new page: http://%5Bobservium server]/graph/map.php
#5 will generate a .dot output file that will be displayed when you visit the URL in #6. By default the script actually puts out .png files, which depending on your size, may be fine, but with 1100+ objects in ours, it was too hard to zoom in on.
So for changes for #1:
This will write out to .dot format instead of .png """graph.write_png('%s_map.png' % (this_device))""" graph.write_dot('%s_map.dot' % (this_device))
As for your issue, I believe it is based on this part: UPPER*(*d*.*hostname*) = *UPPER*(*'GEALICRDC03'*)*;
I had to use FQDN as noted in #5 above.
Eric
On Mon, Nov 7, 2016 at 9:27 AM, Vater, Wolfgang < wolfgang.vater@de.unisys.com> wrote:
Could you provide instruction show to get this to work, please?
This is the one addon I really would like to see working.
Wolfgang
*From:* observium [mailto:observium-bounces@observium.org] *On Behalf Of *Eric Kollmann *Sent:* Tuesday, November 1, 2016 21:44 *To:* Observium Network Observation System observium@observium.org *Subject:* Re: [Observium] Tables in the Observium DB
Once you verified it had to be changed and indicated links table is no longer there, think I got it updated to work now:
have to actually remove the 2 lines starting with //, but basically changed links to neighbours and updated l.local_port_id to l.port_id below. Running this with 1100+ devices in observium kicks out a really large map, but got to say great product!
SELECT
UPPER(d.hostname) AS local_hostname,
p.port_id AS local_port_id,
p.ifName AS local_port,
p.ifSpeed/1000000 as local_port_speed,
l.remote_port_id AS remote_port_id,
rp.ifName AS remote_port,
UPPER(l.remote_hostname) AS remote_hostname,
d.hardware AS hardware,
rd.hardware AS remote_hardware,
s.ifInErrors_rate AS local_in_errors_rate,
s.ifOutErrors_rate AS local_out_errors_rate,
s.ifInOctets_rate*8 AS local_in_octets_rate,
s.ifOutOctets_rate*8 AS local_out_octets_rate,
s.ifInOctets_perc AS local_in_octets_perc,
s.ifOutOctets_perc AS local_out_octets_perc
FROM
`devices` AS d,
`devices` as rd,
`ports` AS p,
`ports` as rp,
// `links` AS l,
`neighbours` AS l,
`ports-state` AS s
WHERE
// p.port_id = l.local_port_id AND
p.port_id = l.port_id AND
p.device_id = d.device_id AND
rp.port_id = l.remote_port_id AND
rp.device_id = rd.device_id AND
p.port_id = s.port_id AND
UPPER(d.hostname) = UPPER(%s);
"""
On Tue, Nov 1, 2016 at 12:50 PM, Eric Kollmann xnih13@gmail.com wrote:
Perfect, happy to test it out if/when you get a chance to send it on. If php stuff needs tweaked I'll see what I can come up with there.
Eric
On Tue, Nov 1, 2016 at 12:06 PM, Adam Armstrong adama@memetic.org wrote:
Hi Eric,
I rewrote the query that script uses and it's saved to a text file. I've not gotten around to rewriting the rest of the code in php yet,since I don't have a CDP network to test on.
I can dig out the SQL when I'm not driving :) (bathroom stop, I can't write email and drive! ;))
Adam.
Sent from BlueMail http://www.bluemail.me/r
On 1 Nov 2016, at 16:11, Eric Kollmann xnih13@gmail.com wrote:
First, I just took over our install of this, wasn't involved in the initial setup of observium and I understand the piece I'm asking about is not part of observium, just uses the data it collects and I want to verify if structure has changed since it was released or if there is something incorrectly setup with our system.
I'm looking to build out graphviz maps with the data observium has gathered, specifically using:
https://github.com/kedare/observium_map_generator
Going through their code, it queries the db for the links table and fails. Looking at the database I have a neighbours table, but not links. Google search I see links table discussed back in 2014, but nothing much recently.
So, did the links table go away in newer versions, or is the fact that the table is missing an issue with our setup? If it is supposed to be there and missing, I'll work on figuring that out next, if it did just go away I just want to verify that.
Thanks
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Great, I did everything like you did, just named the directory under html different
I also found a wrong link in graph.py, search for “example” (without quotes).
But my sql squery does not work at all. Even if I completely remove the hostname (remove AND UPPER(d.hostname) like UPPER('GEALICRXI02')) ) is still get an empty table.
Query looks like this then:
SELECT UPPER(d.hostname) AS local_hostname, p.port_id AS local_port_id, p.ifName AS local_port, p.ifSpeed/1000000 as local_port_speed, l.remote_port_id AS remote_port_id, rp.ifName AS remote_port, UPPER(l.remote_hostname) AS remote_hostname, d.hardware AS hardware, rd.hardware AS remote_hardware, s.ifInErrors_rate AS local_in_errors_rate, s.ifOutErrors_rate AS local_out_errors_rate, s.ifInOctets_rate*8 AS local_in_octets_rate, s.ifOutOctets_rate*8 AS local_out_octets_rate, s.ifInOctets_perc AS local_in_octets_perc, s.ifOutOctets_perc AS local_out_octets_perc FROM `devices` AS d, `devices` as rd, `ports` AS p, `ports` as rp, `neighbours` AS l, `ports-state` AS s WHERE p.port_id = l.port_id AND p.device_id = d.device_id AND rp.port_id = l.remote_port_id AND rp.device_id = rd.device_id AND p.port_id = s.port_id;
I need to look closer into the query. It seems I got everything else right, but the query. If there is no data returned then the map won’t show anything ☺
Thanks again for your detailed steps.
Wolfgang
From: observium [mailto:observium-bounces@observium.org] On Behalf Of Eric Kollmann Sent: Monday, November 7, 2016 18:06 To: Observium Network Observation System observium@observium.org Subject: Re: [Observium] Tables in the Observium DB
Here is what I did
1. Modify the script, the 2 changes discussed already and one other change to write out to a .dot file, instead of a .png (notes at bottom). Also, this is just looking at cdp and whatever else is in the neighbors table, it is not looking or building a graph based on OSPF which I believe there is a table for as well, but I have not had time to investigate. 2. Per the documentation, I created a graph directory under html, so in my case /opt/observium/html/graph 3. Dropped the map.php and the viz.js file in that directory (I ended up grabbing it from here: https://github.com/mdaines/viz.js/releases, unknown if there are better locations, but developer on that seems to have stopped it). 4. Modified map.php to point at the .dot file that I generated, we'll say for example: myservers.mydomain.org_map.dot (It is CHANGEME or something like that in the original map.php). 5. Then ran the following command: sudo python /opt/observium/graph.py myservers.mydomain.orghttp://myservers.mydomain.org (where myservers.mydomain.orghttp://myservers.mydomain.org is a full DNS entry that exists in your setup for what you want to be the main node so to say). 6. Visit your server/new page: http://%5Bobservium server]/graph/map.php
#5 will generate a .dot output file that will be displayed when you visit the URL in #6. By default the script actually puts out .png files, which depending on your size, may be fine, but with 1100+ objects in ours, it was too hard to zoom in on.
So for changes for #1:
This will write out to .dot format instead of .png """graph.write_png('%s_map.png' % (this_device))""" graph.write_dot('%s_map.dot' % (this_device))
As for your issue, I believe it is based on this part: UPPER(d.hostname) = UPPER('GEALICRDC03');
I had to use FQDN as noted in #5 above.
Eric
On Mon, Nov 7, 2016 at 9:27 AM, Vater, Wolfgang <wolfgang.vater@de.unisys.commailto:wolfgang.vater@de.unisys.com> wrote: Could you provide instruction show to get this to work, please?
This is the one addon I really would like to see working.
Wolfgang
From: observium [mailto:observium-bounces@observium.orgmailto:observium-bounces@observium.org] On Behalf Of Eric Kollmann Sent: Tuesday, November 1, 2016 21:44 To: Observium Network Observation System <observium@observium.orgmailto:observium@observium.org> Subject: Re: [Observium] Tables in the Observium DB
Once you verified it had to be changed and indicated links table is no longer there, think I got it updated to work now:
have to actually remove the 2 lines starting with //, but basically changed links to neighbours and updated l.local_port_id to l.port_id below. Running this with 1100+ devices in observium kicks out a really large map, but got to say great product!
SELECT UPPER(d.hostname) AS local_hostname, p.port_id AS local_port_id, p.ifName AS local_port, p.ifSpeed/1000000 as local_port_speed, l.remote_port_id AS remote_port_id, rp.ifName AS remote_port, UPPER(l.remote_hostname) AS remote_hostname, d.hardware AS hardware, rd.hardware AS remote_hardware, s.ifInErrors_rate AS local_in_errors_rate, s.ifOutErrors_rate AS local_out_errors_rate, s.ifInOctets_rate*8 AS local_in_octets_rate, s.ifOutOctets_rate*8 AS local_out_octets_rate, s.ifInOctets_perc AS local_in_octets_perc, s.ifOutOctets_perc AS local_out_octets_perc FROM `devices` AS d, `devices` as rd, `ports` AS p, `ports` as rp, // `links` AS l, `neighbours` AS l, `ports-state` AS s WHERE // p.port_id = l.local_port_id AND p.port_id = l.port_id AND p.device_id = d.device_id AND rp.port_id = l.remote_port_id AND rp.device_id = rd.device_id AND p.port_id = s.port_id AND UPPER(d.hostname) = UPPER(%s); """
On Tue, Nov 1, 2016 at 12:50 PM, Eric Kollmann <xnih13@gmail.commailto:xnih13@gmail.com> wrote: Perfect, happy to test it out if/when you get a chance to send it on. If php stuff needs tweaked I'll see what I can come up with there.
Eric
On Tue, Nov 1, 2016 at 12:06 PM, Adam Armstrong <adama@memetic.orgmailto:adama@memetic.org> wrote: Hi Eric, I rewrote the query that script uses and it's saved to a text file. I've not gotten around to rewriting the rest of the code in php yet,since I don't have a CDP network to test on. I can dig out the SQL when I'm not driving :) (bathroom stop, I can't write email and drive! ;)) Adam.
Sent from BlueMailhttp://www.bluemail.me/r On 1 Nov 2016, at 16:11, Eric Kollmann <xnih13@gmail.commailto:xnih13@gmail.com> wrote: First, I just took over our install of this, wasn't involved in the initial setup of observium and I understand the piece I'm asking about is not part of observium, just uses the data it collects and I want to verify if structure has changed since it was released or if there is something incorrectly setup with our system.
I'm looking to build out graphviz maps with the data observium has gathered, specifically using: https://github.com/kedare/observium_map_generator
Going through their code, it queries the db for the links table and fails. Looking at the database I have a neighbours table, but not links. Google search I see links table discussed back in 2014, but nothing much recently.
So, did the links table go away in newer versions, or is the fact that the table is missing an issue with our setup? If it is supposed to be there and missing, I'll work on figuring that out next, if it did just go away I just want to verify that.
Thanks
________________________________
observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
So revisiting this a year later, 2 questions/issues:
1. While this worked a year ago, sometime in the past year it stopped. I had tested it back then, but never actually rolled into into any ongoing state.
I had to update cursor.execute(query_template, this_device) to: cursor.execute(query_template, { this_device })
I didn't dig into what changed on the python side, but found that it wasn't feeding into cursor correctly with the old way, at least on my system.
So in case anyone finds this old thread and runs into a truple error, hopefully this helps.
2. A friend is trying the same code that works in mine but not on his. We found his observium db doesn't have the table `ports-state`. Curious if this has gone away in newer versions or if it only gets created in certain circumstances. We are a large number of versions behind and my system was initially built 3 or so years ago, where his is a new build in the past 6-12 months.
Comparing table list I found tables in my setup that were not in his: alert_table-state bgpPeers-state bgpPeers_cbgp-state mac_accounting-state mempools-state oids_assoc ports-state processors-state pseudowires-state sensors-state slas-state status-state storage-state ucd_diskio-state
Tables in his observium db not in mine: lb_pool_members lb_snatpools oids_entries
Are the tables based on specific devices it finds in the scans or have these just changed in newer builds?
If they did change, did the information collected in ports-state migrate into one of the existing tables such as ports, or go away?
Thanks
On Tue, Nov 1, 2016 at 2:43 PM, Eric Kollmann xnih13@gmail.com wrote:
Once you verified it had to be changed and indicated links table is no longer there, think I got it updated to work now:
have to actually remove the 2 lines starting with //, but basically changed links to neighbours and updated l.local_port_id to l.port_id below. Running this with 1100+ devices in observium kicks out a really large map, but got to say great product!
SELECT UPPER(d.hostname) AS local_hostname, p.port_id AS local_port_id, p.ifName AS local_port, p.ifSpeed/1000000 as local_port_speed, l.remote_port_id AS remote_port_id, rp.ifName AS remote_port, UPPER(l.remote_hostname) AS remote_hostname, d.hardware AS hardware, rd.hardware AS remote_hardware, s.ifInErrors_rate AS local_in_errors_rate, s.ifOutErrors_rate AS local_out_errors_rate, s.ifInOctets_rate*8 AS local_in_octets_rate, s.ifOutOctets_rate*8 AS local_out_octets_rate, s.ifInOctets_perc AS local_in_octets_perc, s.ifOutOctets_perc AS local_out_octets_perc FROM `devices` AS d, `devices` as rd, `ports` AS p, `ports` as rp, // `links` AS l, `neighbours` AS l, `ports-state` AS s WHERE // p.port_id = l.local_port_id AND p.port_id = l.port_id AND p.device_id = d.device_id AND rp.port_id = l.remote_port_id AND rp.device_id = rd.device_id AND p.port_id = s.port_id AND UPPER(d.hostname) = UPPER(%s); """
On Tue, Nov 1, 2016 at 12:50 PM, Eric Kollmann xnih13@gmail.com wrote:
Perfect, happy to test it out if/when you get a chance to send it on. If php stuff needs tweaked I'll see what I can come up with there.
Eric
On Tue, Nov 1, 2016 at 12:06 PM, Adam Armstrong adama@memetic.org wrote:
Hi Eric,
I rewrote the query that script uses and it's saved to a text file. I've not gotten around to rewriting the rest of the code in php yet,since I don't have a CDP network to test on.
I can dig out the SQL when I'm not driving :) (bathroom stop, I can't write email and drive! ;))
Adam.
Sent from BlueMail http://www.bluemail.me/r
On 1 Nov 2016, at 16:11, Eric Kollmann xnih13@gmail.com wrote:
First, I just took over our install of this, wasn't involved in the initial setup of observium and I understand the piece I'm asking about is not part of observium, just uses the data it collects and I want to verify if structure has changed since it was released or if there is something incorrectly setup with our system.
I'm looking to build out graphviz maps with the data observium has gathered, specifically using: https://github.com/kedare/observium_map_generator
Going through their code, it queries the db for the links table and fails. Looking at the database I have a neighbours table, but not links. Google search I see links table discussed back in 2014, but nothing much recently.
So, did the links table go away in newer versions, or is the fact that the table is missing an issue with our setup? If it is supposed to be there and missing, I'll work on figuring that out next, if it did just go away I just want to verify that.
Thanks
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
The majority of the -state tables were removed some time ago. We still have some for little-used tables, but they're not important.
I think the ports-state table was removed a year and a half ago.
adam.
Adam Armstrong CEO & Lead Architect Observium Limited On 2017-10-13 22:33:11, Eric Kollmann xnih13@gmail.com wrote: So revisiting this a year later, 2 questions/issues:
1. While this worked a year ago, sometime in the past year it stopped. I had tested it back then, but never actually rolled into into any ongoing state.
I had to update cursor.execute(query_template, this_device)
to: cursor.execute(query_template, { this_device })
I didn't dig into what changed on the python side, but found that it wasn't feeding into cursor correctly with the old way, at least on my system.
So in case anyone finds this old thread and runs into a truple error, hopefully this helps.
2. A friend is trying the same code that works in mine but not on his. We found his observium db doesn't have the table `ports-state`. Curious if this has gone away in newer versions or if it only gets created in certain circumstances. We are a large number of versions behind and my system was initially built 3 or so years ago, where his is a new build in the past 6-12 months.
Comparing table list I found tables in my setup that were not in his: alert_table-state bgpPeers-state bgpPeers_cbgp-state mac_accounting-state mempools-state oids_assoc ports-state processors-state pseudowires-state sensors-state slas-state status-state storage-state ucd_diskio-state
Tables in his observium db not in mine: lb_pool_members lb_snatpools oids_entries
Are the tables based on specific devices it finds in the scans or have these just changed in newer builds?
If they did change, did the information collected in ports-state migrate into one of the existing tables such as ports, or go away?
Thanks
On Tue, Nov 1, 2016 at 2:43 PM, Eric Kollmann <xnih13@gmail.com [mailto:xnih13@gmail.com]> wrote:
Once you verified it had to be changed and indicated links table is no longer there, think I got it updated to work now:
have to actually remove the 2 lines starting with //, but basically changed links to neighbours and updated l.local_port_id to l.port_id below. Running this with 1100+ devices in observium kicks out a really large map, but got to say great product!
SELECT UPPER(d.hostname) AS local_hostname, p.port_id AS local_port_id, p.ifName AS local_port, p.ifSpeed/1000000 as local_port_speed, l.remote_port_id AS remote_port_id, rp.ifName AS remote_port, UPPER(l.remote_hostname) AS remote_hostname, d.hardware AS hardware, rd.hardware AS remote_hardware, s.ifInErrors_rate AS local_in_errors_rate, s.ifOutErrors_rate AS local_out_errors_rate, s.ifInOctets_rate*8 AS local_in_octets_rate, s.ifOutOctets_rate*8 AS local_out_octets_rate, s.ifInOctets_perc AS local_in_octets_perc, s.ifOutOctets_perc AS local_out_octets_perc FROM `devices` AS d, `devices` as rd, `ports` AS p, `ports` as rp, // `links` AS l, `neighbours` AS l, `ports-state` AS s WHERE // p.port_id = l.local_port_id AND
p.port_id = l.port_id AND p.device_id = d.device_id AND rp.port_id = l.remote_port_id AND rp.device_id = rd.device_id AND p.port_id = s.port_id AND UPPER(d.hostname) = UPPER(%s); """
On Tue, Nov 1, 2016 at 12:50 PM, Eric Kollmann <xnih13@gmail.com [mailto:xnih13@gmail.com]> wrote:
Perfect, happy to test it out if/when you get a chance to send it on. If php stuff needs tweaked I'll see what I can come up with there.
Eric
On Tue, Nov 1, 2016 at 12:06 PM, Adam Armstrong <adama@memetic.org [mailto:adama@memetic.org]> wrote:
Hi Eric,
I rewrote the query that script uses and it's saved to a text file. I've not gotten around to rewriting the rest of the code in php yet,since I don't have a CDP network to test on.
I can dig out the SQL when I'm not driving :) (bathroom stop, I can't write email and drive! ;))
Adam.
Sent from BlueMail [http://www.bluemail.me/r]
On 1 Nov 2016, at 16:11, Eric Kollmann <xnih13@gmail.com [mailto:xnih13@gmail.com]> wrote: First, I just took over our install of this, wasn't involved in the initial setup of observium and I understand the piece I'm asking about is not part of observium, just uses the data it collects and I want to verify if structure has changed since it was released or if there is something incorrectly setup with our system.
I'm looking to build out graphviz maps with the data observium has gathered, specifically using: https://github.com/kedare/observium_map_generator [https://github.com/kedare/observium_map_generator]
Going through their code, it queries the db for the links table and fails. Looking at the database I have a neighbours table, but not links. Google search I see links table discussed back in 2014, but nothing much recently.
So, did the links table go away in newer versions, or is the fact that the table is missing an issue with our setup? If it is supposed to be there and missing, I'll work on figuring that out next, if it did just go away I just want to verify that.
Thanks
observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
_______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
_______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
It's a pain that I don't have a network with a discovery protocol running myself. I should maybe try to fake a load of links to clean this stuff up at my end.
Thanks, adam.
Adam Armstrong CEO & Lead Architect Observium Limited On 2017-10-13 22:36:22, Adam Armstrong adama@observium.org wrote: The majority of the -state tables were removed some time ago. We still have some for little-used tables, but they're not important.
I think the ports-state table was removed a year and a half ago.
adam.
Adam Armstrong CEO & Lead Architect Observium Limited On 2017-10-13 22:33:11, Eric Kollmann xnih13@gmail.com wrote: So revisiting this a year later, 2 questions/issues:
1. While this worked a year ago, sometime in the past year it stopped. I had tested it back then, but never actually rolled into into any ongoing state.
I had to update cursor.execute(query_template, this_device)
to: cursor.execute(query_template, { this_device })
I didn't dig into what changed on the python side, but found that it wasn't feeding into cursor correctly with the old way, at least on my system.
So in case anyone finds this old thread and runs into a truple error, hopefully this helps.
2. A friend is trying the same code that works in mine but not on his. We found his observium db doesn't have the table `ports-state`. Curious if this has gone away in newer versions or if it only gets created in certain circumstances. We are a large number of versions behind and my system was initially built 3 or so years ago, where his is a new build in the past 6-12 months.
Comparing table list I found tables in my setup that were not in his: alert_table-state bgpPeers-state bgpPeers_cbgp-state mac_accounting-state mempools-state oids_assoc ports-state processors-state pseudowires-state sensors-state slas-state status-state storage-state ucd_diskio-state
Tables in his observium db not in mine: lb_pool_members lb_snatpools oids_entries
Are the tables based on specific devices it finds in the scans or have these just changed in newer builds?
If they did change, did the information collected in ports-state migrate into one of the existing tables such as ports, or go away?
Thanks
On Tue, Nov 1, 2016 at 2:43 PM, Eric Kollmann <xnih13@gmail.com [mailto:xnih13@gmail.com]> wrote:
Once you verified it had to be changed and indicated links table is no longer there, think I got it updated to work now:
have to actually remove the 2 lines starting with //, but basically changed links to neighbours and updated l.local_port_id to l.port_id below. Running this with 1100+ devices in observium kicks out a really large map, but got to say great product!
SELECT UPPER(d.hostname) AS local_hostname, p.port_id AS local_port_id, p.ifName AS local_port, p.ifSpeed/1000000 as local_port_speed, l.remote_port_id AS remote_port_id, rp.ifName AS remote_port, UPPER(l.remote_hostname) AS remote_hostname, d.hardware AS hardware, rd.hardware AS remote_hardware, s.ifInErrors_rate AS local_in_errors_rate, s.ifOutErrors_rate AS local_out_errors_rate, s.ifInOctets_rate*8 AS local_in_octets_rate, s.ifOutOctets_rate*8 AS local_out_octets_rate, s.ifInOctets_perc AS local_in_octets_perc, s.ifOutOctets_perc AS local_out_octets_perc FROM `devices` AS d, `devices` as rd, `ports` AS p, `ports` as rp, // `links` AS l, `neighbours` AS l, `ports-state` AS s WHERE // p.port_id = l.local_port_id AND
p.port_id = l.port_id AND p.device_id = d.device_id AND rp.port_id = l.remote_port_id AND rp.device_id = rd.device_id AND p.port_id = s.port_id AND UPPER(d.hostname) = UPPER(%s); """
On Tue, Nov 1, 2016 at 12:50 PM, Eric Kollmann <xnih13@gmail.com [mailto:xnih13@gmail.com]> wrote:
Perfect, happy to test it out if/when you get a chance to send it on. If php stuff needs tweaked I'll see what I can come up with there.
Eric
On Tue, Nov 1, 2016 at 12:06 PM, Adam Armstrong <adama@memetic.org [mailto:adama@memetic.org]> wrote:
Hi Eric,
I rewrote the query that script uses and it's saved to a text file. I've not gotten around to rewriting the rest of the code in php yet,since I don't have a CDP network to test on.
I can dig out the SQL when I'm not driving :) (bathroom stop, I can't write email and drive! ;))
Adam.
Sent from BlueMail [http://www.bluemail.me/r]
On 1 Nov 2016, at 16:11, Eric Kollmann <xnih13@gmail.com [mailto:xnih13@gmail.com]> wrote: First, I just took over our install of this, wasn't involved in the initial setup of observium and I understand the piece I'm asking about is not part of observium, just uses the data it collects and I want to verify if structure has changed since it was released or if there is something incorrectly setup with our system.
I'm looking to build out graphviz maps with the data observium has gathered, specifically using: https://github.com/kedare/observium_map_generator [https://github.com/kedare/observium_map_generator]
Going through their code, it queries the db for the links table and fails. Looking at the database I have a neighbours table, but not links. Google search I see links table discussed back in 2014, but nothing much recently.
So, did the links table go away in newer versions, or is the fact that the table is missing an issue with our setup? If it is supposed to be there and missing, I'll work on figuring that out next, if it did just go away I just want to verify that.
Thanks
observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
_______________________________________________ observium mailing list observium@observium.org [mailto:observium@observium.org] http://postman.memetic.org/cgi-bin/mailman/listinfo/observium [http://postman.memetic.org/cgi-bin/mailman/listinfo/observium]
_______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Ok, thanks, knew being behind may have been a part of the problem with the tables missing. Working on getting someone to build a new system on our end that is NOT on RHEL and then get the new version installed and start playing with this code again to map things out.
Eric
On Fri, Oct 13, 2017 at 3:36 PM, Adam Armstrong adama@observium.org wrote:
The majority of the -state tables were removed some time ago. We still have some for little-used tables, but they're not important.
I think the ports-state table was removed a year and a half ago.
adam.
Adam Armstrong CEO & Lead Architect Observium Limited
On 2017-10-13 22:33:11, Eric Kollmann xnih13@gmail.com wrote: So revisiting this a year later, 2 questions/issues:
While this worked a year ago, sometime in the past year it stopped. I had tested it back then, but never actually rolled into into any ongoing state.
I had to update cursor.execute(query_template, this_device) to: cursor.execute(query_template, { this_device })
I didn't dig into what changed on the python side, but found that it wasn't feeding into cursor correctly with the old way, at least on my system.
So in case anyone finds this old thread and runs into a truple error, hopefully this helps.
A friend is trying the same code that works in mine but not on his. We found his observium db doesn't have the table `ports-state`. Curious if this has gone away in newer versions or if it only gets created in certain circumstances. We are a large number of versions behind and my system was initially built 3 or so years ago, where his is a new build in the past 6-12 months.
Comparing table list I found tables in my setup that were not in his: alert_table-state bgpPeers-state bgpPeers_cbgp-state mac_accounting-state mempools-state oids_assoc ports-state processors-state pseudowires-state sensors-state slas-state status-state storage-state ucd_diskio-state
Tables in his observium db not in mine: lb_pool_members lb_snatpools oids_entries
Are the tables based on specific devices it finds in the scans or have these just changed in newer builds?
If they did change, did the information collected in ports-state migrate into one of the existing tables such as ports, or go away?
Thanks
On Tue, Nov 1, 2016 at 2:43 PM, Eric Kollmann xnih13@gmail.com wrote:
Once you verified it had to be changed and indicated links table is no longer there, think I got it updated to work now:
have to actually remove the 2 lines starting with //, but basically changed links to neighbours and updated l.local_port_id to l.port_id below. Running this with 1100+ devices in observium kicks out a really large map, but got to say great product!
SELECT UPPER(d.hostname) AS local_hostname, p.port_id AS local_port_id, p.ifName AS local_port, p.ifSpeed/1000000 as local_port_speed, l.remote_port_id AS remote_port_id, rp.ifName AS remote_port, UPPER(l.remote_hostname) AS remote_hostname, d.hardware AS hardware, rd.hardware AS remote_hardware, s.ifInErrors_rate AS local_in_errors_rate, s.ifOutErrors_rate AS local_out_errors_rate, s.ifInOctets_rate*8 AS local_in_octets_rate, s.ifOutOctets_rate*8 AS local_out_octets_rate, s.ifInOctets_perc AS local_in_octets_perc, s.ifOutOctets_perc AS local_out_octets_perc FROM `devices` AS d, `devices` as rd, `ports` AS p, `ports` as rp, // `links` AS l, `neighbours` AS l, `ports-state` AS s WHERE // p.port_id = l.local_port_id AND p.port_id = l.port_id AND p.device_id = d.device_id AND rp.port_id = l.remote_port_id AND rp.device_id = rd.device_id AND p.port_id = s.port_id AND UPPER(d.hostname) = UPPER(%s); """
On Tue, Nov 1, 2016 at 12:50 PM, Eric Kollmann xnih13@gmail.com wrote:
Perfect, happy to test it out if/when you get a chance to send it on. If php stuff needs tweaked I'll see what I can come up with there.
Eric
On Tue, Nov 1, 2016 at 12:06 PM, Adam Armstrong adama@memetic.org wrote:
Hi Eric,
I rewrote the query that script uses and it's saved to a text file. I've not gotten around to rewriting the rest of the code in php yet,since I don't have a CDP network to test on.
I can dig out the SQL when I'm not driving :) (bathroom stop, I can't write email and drive! ;))
Adam.
Sent from BlueMail http://www.bluemail.me/r
On 1 Nov 2016, at 16:11, Eric Kollmann xnih13@gmail.com wrote:
First, I just took over our install of this, wasn't involved in the initial setup of observium and I understand the piece I'm asking about is not part of observium, just uses the data it collects and I want to verify if structure has changed since it was released or if there is something incorrectly setup with our system.
I'm looking to build out graphviz maps with the data observium has gathered, specifically using: https://github.com/kedare/observium_map_generator
Going through their code, it queries the db for the links table and fails. Looking at the database I have a neighbours table, but not links. Google search I see links table discussed back in 2014, but nothing much recently.
So, did the links table go away in newer versions, or is the fact that the table is missing an issue with our setup? If it is supposed to be there and missing, I'll work on figuring that out next, if it did just go away I just want to verify that.
Thanks
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/ cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
participants (4)
-
Adam Armstrong
-
Adam Armstrong
-
Eric Kollmann
-
Vater, Wolfgang