# scripts/removespikes.php --rrdfile=/opt/observium/rrd/my-host/port-512.rrd NOTE: Using RRDtool Version 1.4.8 NOTE: Creating XML file '/tmp/port-512.dump.965062977' from '/opt/observium/rrd/my-host/port-512.rrd' NOTE: Searching for Spikes in XML file '/tmp/port-512.dump.965062977' NOTE: Backing Up '/opt/observium/rrd/my-host/port-512.rrd' to '/tmp/port-512.rrd.965062977' NOTE: Re-Importing '/tmp/port-512.dump.965062977' to '/opt/observium/rrd/my-host/port-512.rrd'
# scripts/removespikes.php --rrdfile=/opt/observium/rrd/my-host/port-515.rrd NOTE: Using RRDtool Version 1.4.8 NOTE: Creating XML file '/tmp/port-515.dump.248569933' from '/opt/observium/rrd/my-host/port-515.rrd' NOTE: Searching for Spikes in XML file '/tmp/port-515.dump.248569933' NOTE: Backing Up '/opt/observium/rrd/my-host/port-515.rrd' to '/tmp/port-515.rrd' NOTE: Re-Importing '/tmp/port-515.dump.248569933' to '/opt/observium/rrd/my-host/port-515.rrd'
At this point, verifying the graphs, it removed a lot the spike... but not totally. So I ran the script again.
# scripts/removespikes.php --rrdfile=/opt/observium/rrd/my-host/port-515.rrd NOTE: Using RRDtool Version 1.4.8 NOTE: Creating XML file '/tmp/port-515.dump.953792327' from '/opt/observium/rrd/my-host/port-515.rrd' NOTE: Searching for Spikes in XML file '/tmp/port-515.dump.953792327' NOTE: Backing Up '/opt/observium/rrd/my-host/port-515.rrd' to '/tmp/port-515.rrd.953792327' NOTE: Re-Importing '/tmp/port-515.dump.953792327' to '/opt/observium/rrd/my-host/port-515.rrd'
# scripts/removespikes.php --rrdfile=/opt/observium/rrd/my-host/port-512.rrd NOTE: Using RRDtool Version 1.4.8 NOTE: Creating XML file '/tmp/port-512.dump.357012197' from '/opt/observium/rrd/my-host/port-512.rrd' NOTE: Searching for Spikes in XML file '/tmp/port-512.dump.357012197' NOTE: Backing Up '/opt/observium/rrd/my-host/port-512.rrd' to '/tmp/port-512.rrd.357012197' NOTE: Re-Importing '/tmp/port-512.dump.357012197' to '/opt/observium/rrd/my-host/port-512.rrd'
Now it's much better, no spikes! Indeed, the php script works great.
Thank you.
2016-01-21 16:28 GMT-02:00 Adam Armstrong adama@memetic.org:
That script only acts on rrd files, not on the database.
It's a bit old and derpy too. There's a PHP script which works better.
adam.
On 21/01/2016 18:23:07, Eduardo Schoedler listas@esds.com.br wrote: I tested a lot the script to remove spikes, but I can't understand how it works.
# scripts/removespikes.pl -l 0.06 -v /tmp/port-512.rrd Running in VERBOSE mode Limit for bin-based chopping set to 0.06 No peaks found.!
# scripts/removespikes.pl -l 6 -v /tmp/port-512.rrd Running in VERBOSE mode Limit for bin-based chopping set to 6 No peaks found.!
# scripts/removespikes.pl -l 60 -v /tmp/port-512.rrd Running in VERBOSE mode Limit for bin-based chopping set to 60 No peaks found.!
# scripts/removespikes.pl -l 0.06 -t 100000000 -v /tmp/port-512.rrd Running in VERBOSE mode Limit for bin-based chopping set to 0.06 Max Value set to 1e+08, disabling bin-based chopping No peaks found.!
# scripts/removespikes.pl -l 10 -t 100000000 -v /tmp/port-512.rrd Running in VERBOSE mode Limit for bin-based chopping set to 10 Max Value set to 1e+08, disabling bin-based chopping No peaks found.!
# scripts/removespikes.pl -l 20 -t 100000000 -v /tmp/port-512.rrd Running in VERBOSE mode Limit for bin-based chopping set to 20 Max Value set to 1e+08, disabling bin-based chopping No peaks found.!
2016-01-21 16:09 GMT-02:00 Adam Armstrong adama@memetic.org:
These spikes shouldn't really happen, but sometimes it's hard to weed them out.
There might be some code we can use from the ports poller to try to filter these out in future, if i get a chance to take a look.
adam.
On 21/01/2016 17:23:14, Eduardo Schoedler listas@esds.com.br wrote: Adam,
Yes, there is a large value in the table:
mysql> SELECT * FROM bill_data WHERE bill_id = 10 AND timestamp BETWEEN '2016-01-20 16:00:00' AND '2016-01-20 18:00:00';
+---------+---------------------+--------+---------------+-----------+---------------+ | bill_id | timestamp | period | delta | in_delta | out_delta |
+---------+---------------------+--------+---------------+-----------+---------------+ | 10 | 2016-01-20 16:00:00 | 300 | 420089822 | 353359170 | 66730652 | | 10 | 2016-01-20 16:05:00 | 300 | 494265345 | 361777810 | 132487535 | | 10 | 2016-01-20 16:10:00 | 300 | 528145357 | 372315740 | 155829617 | | 10 | 2016-01-20 16:15:00 | 300 | 346064669 | 260674031 | 85390638 | | 10 | 2016-01-20 16:20:00 | 300 | 373899963 | 286552744 | 87347219 | | 10 | 2016-01-20 16:25:00 | 300 | 406708364 | 310406142 | 96302222 | | 10 | 2016-01-20 16:30:00 | 300 | 353585558 | 286064759 | 67520799 | | 10 | 2016-01-20 16:35:00 | 300 | 376104759 | 293666507 | 82438252 | | 10 | 2016-01-20 16:40:00 | 300 | 466089158 | 328439085 | 137650073 | *| 10 | 2016-01-20 16:45:00 | 300 | 1231515051765 | 264605699 | 1231250446066 |* | 10 | 2016-01-20 16:50:00 | 300 | 321868504 | 244902936 | 76965568 | | 10 | 2016-01-20 16:55:00 | 300 | 396602378 | 295232411 | 101369967 | | 10 | 2016-01-20 17:00:00 | 300 | 435646403 | 302257751 | 133388652 | | 10 | 2016-01-20 17:05:00 | 300 | 574915176 | 406955741 | 167959435 | | 10 | 2016-01-20 17:10:00 | 300 | 420821268 | 281749562 | 139071706 | | 10 | 2016-01-20 17:15:00 | 300 | 497632123 | 364700162 | 132931961 | | 10 | 2016-01-20 17:20:00 | 300 | 491631241 | 374396028 | 117235213 | | 10 | 2016-01-20 17:25:00 | 300 | 436369775 | 338883451 | 97486324 | | 10 | 2016-01-20 17:30:00 | 300 | 495699422 | 385720549 | 109978873 | | 10 | 2016-01-20 17:35:00 | 300 | 456040027 | 366380731 | 89659296 | | 10 | 2016-01-20 17:40:00 | 300 | 343876841 | 253132670 | 90744171 | | 10 | 2016-01-20 17:45:00 | 300 | 288716438 | 208696427 | 80020011 | | 10 | 2016-01-20 17:50:00 | 300 | 249196310 | 200195401 | 49000909 | | 10 | 2016-01-20 17:55:00 | 300 | 277595458 | 191080322 | 86515136 | | 10 | 2016-01-20 18:00:00 | 300 | 349455603 | 260510508 | 88945095 |
+---------+---------------------+--------+---------------+-----------+---------------+ 25 rows in set (0.00 sec)
Thank you!
2016-01-21 3:59 GMT-02:00 Adam Armstrong adama@memetic.org:
To remove that, go in to the database (phpmyadmin is useful!), look at the bill_data table and delete values which are too large.
http://alpha.memetic.org/~adama/snaps/www.observium.org___localhost___observ...
You shouldn't really get many spikes though!
adam.
On 21/01/2016 05:39:34, Eduardo Schoedler listas@esds.com.br wrote: Perfect, I'll do that.
For the record, here is the "accurate graph":
[image: Imagem inline 1]
The spike is there too :))
2016-01-21 3:35 GMT-02:00 Adam Armstrong adama@memetic.org:
Hmm. It actually seems that that graph is returning the billing-data 95th, rather than the rrdtool 95th. That's fine, then.
The "quick graphs" are generated from the rrd files of the ports included in the bill. the "accurate graphs" are generated from the billing data held in the database.
You should take the value from the "bill summary" box on the billing page though.
adam.
On 21/01/2016 05:31:03, Eduardo Schoedler listas@esds.com.br wrote: Sorry... 95th percentile.
2016-01-21 3:30 GMT-02:00 Eduardo Schoedler listas@esds.com.br:
Adam,
We do 97th percentile billing using this graph, with the value of the legend. Do you have sure this spike is not going to interfere in the calculations?
Thank you.
2016-01-21 3:19 GMT-02:00 Adam Armstrong adama@memetic.org:
Just ignore them. If they're not present in the non-rrd billing data, they're irrelevant.
To remove ones which don't disappear with default settings, you have to spend half of your life arsing around with the removespikes.pl settings, which is a pain.
adam.
On 21/01/2016 05:08:52, Eduardo Schoedler listas@esds.com.br wrote: I ran the script for all rrds, but the spikes are still there. How can I remove?
Thanks.
2016-01-21 3:03 GMT-02:00 Adam Armstrong adama@memetic.org:
> That graph actually isn't billing data, it's just rrd. > > The "detailed" graph is built from billing data. > > adam. > > On 21/01/2016 04:35:58, Eduardo Schoedler listas@esds.com.br > wrote: > Hi, > > Forget it, I found something in the site... :) > > for i in rrd/*/*.rrd; do scripts/removespikes.pl $i; done; > > Thanks. > > > > 2016-01-20 17:13 GMT-02:00 Eduardo Schoedler listas@esds.com.br: > >> Hi, >> >> Is there a way to remove spikes from the billing? >> >> [image: Imagem inline 1] >> >> Thank you. >> >> Regards, >> >> -- >> Eduardo Schoedler >> > > > > -- > Eduardo Schoedler > _______________________________________________ observium mailing > list observium@observium.org > http://postman.memetic.org/cgi-bin/mailman/listinfo/observium > > > _______________________________________________ > observium mailing list > observium@observium.org > http://postman.memetic.org/cgi-bin/mailman/listinfo/observium > >
-- Eduardo Schoedler _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
-- Eduardo Schoedler
-- Eduardo Schoedler _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
-- Eduardo Schoedler _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
-- Eduardo Schoedler _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
-- Eduardo Schoedler _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium