Had a little whoopsie and have had to restore from backup. Installed observium, Pro edition. made sure that the observium version matched on our backup copy, copied the db, rrd files . it works, but I have no graphs. Just "error drawing graph"
Ideas where I should look?
Start by clicking on a graph and then the "RRD Command"-button at the top right of the graph. It will display the RRD error message which probably gives some clues.
Den mån 14 dec. 2020 kl 19:19 skrev Mark Rowlands via observium < observium@observium.org>:
Had a little whoopsie and have had to restore from backup.
Installed observium, Pro edition. made sure that the observium version matched on our backup copy, copied the db, rrd files . it works, but I have no graphs. Just “error drawing graph”
Ideas where I should look? _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
On Dec 14, 2020, at 11:29 , Markus Klock via observium observium@observium.org wrote:
Start by clicking on a graph and then the "RRD Command"-button at the top right of the graph. It will display the RRD error message which probably gives some clues.
And since I just did this myself last night …
1) check the permissions in the RRD directory 2) Recall if you enabled rrdcached because if you did and forgot to reinstall it the command will fail (don’t forget to reconfigure it too per docs) 3) Login to the system, cd over to the RRD directory, and attempt to run the command you find in Markus’s comment above. That will tell you exactly what is going on.
Thank you all for the tips, back up and running and only minor data loss!.
-----Original Message----- From: observium observium-bounces@observium.org On Behalf Of Joey Stanford via observium Sent: den 14 december 2020 22:23 To: Observium observium@observium.org Cc: Joey Stanford nv0n@rmham.org Subject: Re: [Observium] no graphs after restore
On Dec 14, 2020, at 11:29 , Markus Klock via observium observium@observium.org wrote:
Start by clicking on a graph and then the "RRD Command"-button at the top right of the graph. It will display the RRD error message which probably gives some clues.
And since I just did this myself last night …
1) check the permissions in the RRD directory 2) Recall if you enabled rrdcached because if you did and forgot to reinstall it the command will fail (don’t forget to reconfigure it too per docs) 3) Login to the system, cd over to the RRD directory, and attempt to run the command you find in Markus’s comment above. That will tell you exactly what is going on. _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hhm Not so fast, I have an issue with rrdcached.
RRD STDOUT[ERROR: rrdcached@unix:/var/run/rrdcached.sock: Cannot read/write /opt/observium/rrd/tapp01-mq.riksbyggen.intra/perf-pollermodule-probes.rrd: Permission denied]
ls of an rrd. -rw-r--r-- 1 root www-data 117272 Dec 16 14:48 /opt/observium/rrd/tapp01-mq.riksbyggen.intra/perf-pollermodule-probes.rrd
ls -al /observium drwxrwsr-x 4 observium www-data 4096 Dec 16 14:48 rrd
rrdcached runs however as observium, is this why it cannot update the rrd?
observi+ 21821 0.0 0.0 510208 3572 ? Ssl 14:17 0:00 /usr/bin/rrdcached -B -F -R -w 300 -z 0 -t 4 -b
This is as vanilla an install as possible. Fresh Ubuntu 20.04 LTS Observium Professional installed from the script. Smokeping installed rrdcached installed and instructions from https://docs.observium.org/rrdcached/ followed
what should the rights on those rrds look like?
Thank you all for the tips, back up and running and only minor data loss!.
-----Original Message----- From: observium observium-bounces@observium.org On Behalf Of Joey Stanford via observium Sent: den 14 december 2020 22:23 To: Observium observium@observium.org Cc: Joey Stanford nv0n@rmham.org Subject: Re: [Observium] no graphs after restore
On Dec 14, 2020, at 11:29 , Markus Klock via observium observium@observium.org wrote:
Start by clicking on a graph and then the "RRD Command"-button at the top right of the graph. It will display the RRD error message which probably gives some clues.
And since I just did this myself last night …
1) check the permissions in the RRD directory 2) Recall if you enabled rrdcached because if you did and forgot to reinstall it the command will fail (don’t forget to reconfigure it too per docs) 3) Login to the system, cd over to the RRD directory, and attempt to run the command you find in Markus’s comment above. That will tell you exactly what is going on. ________________________________________THis is _______ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Shouldnt your RRD-files also be owned by user observium? Perhaps you did copy them from backup with sudo as the rrd-files themselves are owned by root now.
Den ons 16 dec. 2020 kl 15:21 skrev Mark Rowlands via observium < observium@observium.org>:
Hhm Not so fast, I have an issue with rrdcached.
RRD STDOUT[ERROR: rrdcached@unix:/var/run/rrdcached.sock: Cannot read/write /opt/observium/rrd/tapp01-mq.riksbyggen.intra/perf-pollermodule-probes.rrd: Permission denied]
ls of an rrd. -rw-r--r-- 1 root www-data 117272 Dec 16 14:48 /opt/observium/rrd/tapp01-mq.riksbyggen.intra/perf-pollermodule-probes.rrd
ls -al /observium drwxrwsr-x 4 observium www-data 4096 Dec 16 14:48 rrd
rrdcached runs however as observium, is this why it cannot update the rrd?
observi+ 21821 0.0 0.0 510208 3572 ? Ssl 14:17 0:00 /usr/bin/rrdcached -B -F -R -w 300 -z 0 -t 4 -b
This is as vanilla an install as possible. Fresh Ubuntu 20.04 LTS Observium Professional installed from the script. Smokeping installed rrdcached installed and instructions from https://docs.observium.org/rrdcached/ followed
what should the rights on those rrds look like?
Thank you all for the tips, back up and running and only minor data loss!.
-----Original Message----- From: observium observium-bounces@observium.org On Behalf Of Joey Stanford via observium Sent: den 14 december 2020 22:23 To: Observium observium@observium.org Cc: Joey Stanford nv0n@rmham.org Subject: Re: [Observium] no graphs after restore
On Dec 14, 2020, at 11:29 , Markus Klock via observium <
observium@observium.org> wrote:
Start by clicking on a graph and then the "RRD Command"-button at the
top right of the graph.
It will display the RRD error message which probably gives some clues.
And since I just did this myself last night …
- check the permissions in the RRD directory
- Recall if you enabled rrdcached because if you did and forgot to
reinstall it the command will fail (don’t forget to reconfigure it too per docs) 3) Login to the system, cd over to the RRD directory, and attempt to run the command you find in Markus’s comment above. That will tell you exactly what is going on. ________________________________________THis is _______ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
This is a default install I created as a test to find out exactly what rights the files should have. All the rrds are created by Observium not restored/copied
From: observium observium-bounces@observium.org On Behalf Of Markus Klock via observium Sent: den 16 december 2020 15:28 To: Observium observium@observium.org Cc: Markus Klock markus@best-practice.se Subject: Re: [Observium] no graphs after restore
Shouldnt your RRD-files also be owned by user observium? Perhaps you did copy them from backup with sudo as the rrd-files themselves are owned by root now.
Den ons 16 dec. 2020 kl 15:21 skrev Mark Rowlands via observium <observium@observium.orgmailto:observium@observium.org>: Hhm Not so fast, I have an issue with rrdcached.
RRD STDOUT[ERROR: rrdcached@unix:/var/run/rrdcached.sockmailto:rrdcached@unix:/var/run/rrdcached.sock: Cannot read/write /opt/observium/rrd/tapp01-mq.riksbyggen.intra/perf-pollermodule-probes.rrd: Permission denied]
ls of an rrd. -rw-r--r-- 1 root www-data 117272 Dec 16 14:48 /opt/observium/rrd/tapp01-mq.riksbyggen.intra/perf-pollermodule-probes.rrd
ls -al /observium drwxrwsr-x 4 observium www-data 4096 Dec 16 14:48 rrd
rrdcached runs however as observium, is this why it cannot update the rrd?
observi+ 21821 0.0 0.0 510208 3572 ? Ssl 14:17 0:00 /usr/bin/rrdcached -B -F -R -w 300 -z 0 -t 4 -b
This is as vanilla an install as possible. Fresh Ubuntu 20.04 LTS Observium Professional installed from the script. Smokeping installed rrdcached installed and instructions from https://docs.observium.org/rrdcached/ followed
what should the rights on those rrds look like?
Thank you all for the tips, back up and running and only minor data loss!.
-----Original Message----- From: observium <observium-bounces@observium.orgmailto:observium-bounces@observium.org> On Behalf Of Joey Stanford via observium Sent: den 14 december 2020 22:23 To: Observium <observium@observium.orgmailto:observium@observium.org> Cc: Joey Stanford <nv0n@rmham.orgmailto:nv0n@rmham.org> Subject: Re: [Observium] no graphs after restore
On Dec 14, 2020, at 11:29 , Markus Klock via observium <observium@observium.orgmailto:observium@observium.org> wrote:
Start by clicking on a graph and then the "RRD Command"-button at the top right of the graph. It will display the RRD error message which probably gives some clues.
And since I just did this myself last night …
1) check the permissions in the RRD directory 2) Recall if you enabled rrdcached because if you did and forgot to reinstall it the command will fail (don’t forget to reconfigure it too per docs) 3) Login to the system, cd over to the RRD directory, and attempt to run the command you find in Markus’s comment above. That will tell you exactly what is going on. ________________________________________THis is _______ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
If you’re using rrdcached, the permissions need to be read/writeable by the rrdcached process.
It’s probably sufficient to:
chown -R observium.observium /opt/observium/rrd
chmod -R 755 /opt/observium/rrd
Adam.
From: observium observium-bounces@observium.org On Behalf Of Mark Rowlands via observium Sent: 16 December 2020 14:54 To: Observium observium@observium.org Cc: Mark Rowlands Mark.Rowlands@riksbyggen.se Subject: Re: [Observium] no graphs after restore
This is a default install I created as a test to find out exactly what rights the files should have. All the rrds are created by Observium not restored/copied
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Markus Klock via observium Sent: den 16 december 2020 15:28 To: Observium <observium@observium.org mailto:observium@observium.org > Cc: Markus Klock <markus@best-practice.se mailto:markus@best-practice.se > Subject: Re: [Observium] no graphs after restore
Shouldnt your RRD-files also be owned by user observium?
Perhaps you did copy them from backup with sudo as the rrd-files themselves are owned by root now.
Den ons 16 dec. 2020 kl 15:21 skrev Mark Rowlands via observium <observium@observium.org mailto:observium@observium.org >:
Hhm Not so fast, I have an issue with rrdcached.
RRD STDOUT[ERROR: rrdcached@unix:/var/run/rrdcached.sock mailto:rrdcached@unix:/var/run/rrdcached.sock : Cannot read/write /opt/observium/rrd/tapp01-mq.riksbyggen.intra/perf-pollermodule-probes.rrd: Permission denied]
ls of an rrd. -rw-r--r-- 1 root www-data 117272 Dec 16 14:48 /opt/observium/rrd/tapp01-mq.riksbyggen.intra/perf-pollermodule-probes.rrd
ls -al /observium drwxrwsr-x 4 observium www-data 4096 Dec 16 14:48 rrd
rrdcached runs however as observium, is this why it cannot update the rrd?
observi+ 21821 0.0 0.0 510208 3572 ? Ssl 14:17 0:00 /usr/bin/rrdcached -B -F -R -w 300 -z 0 -t 4 -b
This is as vanilla an install as possible. Fresh Ubuntu 20.04 LTS Observium Professional installed from the script. Smokeping installed rrdcached installed and instructions from https://docs.observium.org/rrdcached/ followed
what should the rights on those rrds look like?
Thank you all for the tips, back up and running and only minor data loss!.
-----Original Message----- From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Joey Stanford via observium Sent: den 14 december 2020 22:23 To: Observium <observium@observium.org mailto:observium@observium.org > Cc: Joey Stanford <nv0n@rmham.org mailto:nv0n@rmham.org > Subject: Re: [Observium] no graphs after restore
On Dec 14, 2020, at 11:29 , Markus Klock via observium <observium@observium.org mailto:observium@observium.org > wrote:
Start by clicking on a graph and then the "RRD Command"-button at the top right of the graph. It will display the RRD error message which probably gives some clues.
And since I just did this myself last night …
1) check the permissions in the RRD directory 2) Recall if you enabled rrdcached because if you did and forgot to reinstall it the command will fail (don’t forget to reconfigure it too per docs) 3) Login to the system, cd over to the RRD directory, and attempt to run the command you find in Markus’s comment above. That will tell you exactly what is going on. ________________________________________THis is _______ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Ok. the solution was to stop panicking and actually check everything really carefully.
1) I had made an error in /etc/default/rrdcached 2) of course running the poller manually as root screws up the permissions
Sorry for wasting you good folks time but thank you for your patience and encouragement.
From: observium observium-bounces@observium.org On Behalf Of Adam Armstrong via observium Sent: den 16 december 2020 19:49 To: 'Observium' observium@observium.org Cc: Adam Armstrong adama@observium.org Subject: Re: [Observium] no graphs after restore
If you’re using rrdcached, the permissions need to be read/writeable by the rrdcached process.
It’s probably sufficient to:
chown -R observium.observium /opt/observium/rrd chmod -R 755 /opt/observium/rrd
Adam.
From: observium <observium-bounces@observium.orgmailto:observium-bounces@observium.org> On Behalf Of Mark Rowlands via observium Sent: 16 December 2020 14:54 To: Observium <observium@observium.orgmailto:observium@observium.org> Cc: Mark Rowlands <Mark.Rowlands@riksbyggen.semailto:Mark.Rowlands@riksbyggen.se> Subject: Re: [Observium] no graphs after restore
This is a default install I created as a test to find out exactly what rights the files should have. All the rrds are created by Observium not restored/copied
From: observium <observium-bounces@observium.orgmailto:observium-bounces@observium.org> On Behalf Of Markus Klock via observium Sent: den 16 december 2020 15:28 To: Observium <observium@observium.orgmailto:observium@observium.org> Cc: Markus Klock <markus@best-practice.semailto:markus@best-practice.se> Subject: Re: [Observium] no graphs after restore
Shouldnt your RRD-files also be owned by user observium? Perhaps you did copy them from backup with sudo as the rrd-files themselves are owned by root now.
Den ons 16 dec. 2020 kl 15:21 skrev Mark Rowlands via observium <observium@observium.orgmailto:observium@observium.org>: Hhm Not so fast, I have an issue with rrdcached.
RRD STDOUT[ERROR: rrdcached@unix:/var/run/rrdcached.sockmailto:rrdcached@unix:/var/run/rrdcached.sock: Cannot read/write /opt/observium/rrd/tapp01-mq.riksbyggen.intra/perf-pollermodule-probes.rrd: Permission denied]
ls of an rrd. -rw-r--r-- 1 root www-data 117272 Dec 16 14:48 /opt/observium/rrd/tapp01-mq.riksbyggen.intra/perf-pollermodule-probes.rrd
ls -al /observium drwxrwsr-x 4 observium www-data 4096 Dec 16 14:48 rrd
rrdcached runs however as observium, is this why it cannot update the rrd?
observi+ 21821 0.0 0.0 510208 3572 ? Ssl 14:17 0:00 /usr/bin/rrdcached -B -F -R -w 300 -z 0 -t 4 -b
This is as vanilla an install as possible. Fresh Ubuntu 20.04 LTS Observium Professional installed from the script. Smokeping installed rrdcached installed and instructions from https://docs.observium.org/rrdcached/ followed
what should the rights on those rrds look like?
Thank you all for the tips, back up and running and only minor data loss!.
-----Original Message----- From: observium <observium-bounces@observium.orgmailto:observium-bounces@observium.org> On Behalf Of Joey Stanford via observium Sent: den 14 december 2020 22:23 To: Observium <observium@observium.orgmailto:observium@observium.org> Cc: Joey Stanford <nv0n@rmham.orgmailto:nv0n@rmham.org> Subject: Re: [Observium] no graphs after restore
On Dec 14, 2020, at 11:29 , Markus Klock via observium <observium@observium.orgmailto:observium@observium.org> wrote:
Start by clicking on a graph and then the "RRD Command"-button at the top right of the graph. It will display the RRD error message which probably gives some clues.
And since I just did this myself last night …
1) check the permissions in the RRD directory 2) Recall if you enabled rrdcached because if you did and forgot to reinstall it the command will fail (don’t forget to reconfigure it too per docs) 3) Login to the system, cd over to the RRD directory, and attempt to run the command you find in Markus’s comment above. That will tell you exactly what is going on. ________________________________________THis is _______ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.orgmailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
I sometimes get caught out by running poller as root on a system where poller runs as Observium, and wondering why nothing gets updated.
Adam.
From: Mark Rowlands Mark.Rowlands@riksbyggen.se Sent: 16 December 2020 19:30 To: Observium observium@observium.org Cc: Adam Armstrong adama@observium.org Subject: RE: [Observium] no graphs after restore
Ok. the solution was to stop panicking and actually check everything really carefully.
1) I had made an error in /etc/default/rrdcached
2) of course running the poller manually as root screws up the permissions
Sorry for wasting you good folks time but thank you for your patience and encouragement.
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Adam Armstrong via observium Sent: den 16 december 2020 19:49 To: 'Observium' <observium@observium.org mailto:observium@observium.org > Cc: Adam Armstrong <adama@observium.org mailto:adama@observium.org > Subject: Re: [Observium] no graphs after restore
If you’re using rrdcached, the permissions need to be read/writeable by the rrdcached process.
It’s probably sufficient to:
chown -R observium.observium /opt/observium/rrd
chmod -R 755 /opt/observium/rrd
Adam.
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Mark Rowlands via observium Sent: 16 December 2020 14:54 To: Observium <observium@observium.org mailto:observium@observium.org > Cc: Mark Rowlands <Mark.Rowlands@riksbyggen.se mailto:Mark.Rowlands@riksbyggen.se > Subject: Re: [Observium] no graphs after restore
This is a default install I created as a test to find out exactly what rights the files should have. All the rrds are created by Observium not restored/copied
From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Markus Klock via observium Sent: den 16 december 2020 15:28 To: Observium <observium@observium.org mailto:observium@observium.org > Cc: Markus Klock <markus@best-practice.se mailto:markus@best-practice.se > Subject: Re: [Observium] no graphs after restore
Shouldnt your RRD-files also be owned by user observium?
Perhaps you did copy them from backup with sudo as the rrd-files themselves are owned by root now.
Den ons 16 dec. 2020 kl 15:21 skrev Mark Rowlands via observium <observium@observium.org mailto:observium@observium.org >:
Hhm Not so fast, I have an issue with rrdcached.
RRD STDOUT[ERROR: rrdcached@unix:/var/run/rrdcached.sock mailto:rrdcached@unix:/var/run/rrdcached.sock : Cannot read/write /opt/observium/rrd/tapp01-mq.riksbyggen.intra/perf-pollermodule-probes.rrd: Permission denied]
ls of an rrd. -rw-r--r-- 1 root www-data 117272 Dec 16 14:48 /opt/observium/rrd/tapp01-mq.riksbyggen.intra/perf-pollermodule-probes.rrd
ls -al /observium drwxrwsr-x 4 observium www-data 4096 Dec 16 14:48 rrd
rrdcached runs however as observium, is this why it cannot update the rrd?
observi+ 21821 0.0 0.0 510208 3572 ? Ssl 14:17 0:00 /usr/bin/rrdcached -B -F -R -w 300 -z 0 -t 4 -b
This is as vanilla an install as possible. Fresh Ubuntu 20.04 LTS Observium Professional installed from the script. Smokeping installed rrdcached installed and instructions from https://docs.observium.org/rrdcached/ followed
what should the rights on those rrds look like?
Thank you all for the tips, back up and running and only minor data loss!.
-----Original Message----- From: observium <observium-bounces@observium.org mailto:observium-bounces@observium.org > On Behalf Of Joey Stanford via observium Sent: den 14 december 2020 22:23 To: Observium <observium@observium.org mailto:observium@observium.org > Cc: Joey Stanford <nv0n@rmham.org mailto:nv0n@rmham.org > Subject: Re: [Observium] no graphs after restore
On Dec 14, 2020, at 11:29 , Markus Klock via observium <observium@observium.org mailto:observium@observium.org > wrote:
Start by clicking on a graph and then the "RRD Command"-button at the top right of the graph. It will display the RRD error message which probably gives some clues.
And since I just did this myself last night …
1) check the permissions in the RRD directory 2) Recall if you enabled rrdcached because if you did and forgot to reinstall it the command will fail (don’t forget to reconfigure it too per docs) 3) Login to the system, cd over to the RRD directory, and attempt to run the command you find in Markus’s comment above. That will tell you exactly what is going on. ________________________________________THis is _______ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium _______________________________________________ observium mailing list observium@observium.org mailto:observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
On Dec 16, 2020, at 07:21 , Mark Rowlands via observium observium@observium.org wrote:
ls -al /observium drwxrwsr-x 4 observium www-data 4096 Dec 16 14:48 rrd
This is correct but look inside the rrd folder and ensure the files are the same.
participants (4)
-
Adam Armstrong -
Joey Stanford -
Mark Rowlands -
Markus Klock