Hi.
I found Observium after reading a post on Reddit (if you're out there, whoever made the post, you have my eternal thanks!), and I've got it setup and running pretty much OK, but I've run into an issue with one specific Windows host.
Observium accesses and sets it up OK except for one disk volume - the drive is approximately 15 Tb in size, but Observium is only reporting it as being 8Tb, and I can't figure out why.
Can anyone suggest what I may have configured incorrectly? I initially used SNMP V1 to discover/configure this host - could that be the issue, perhaps?
Thanks for any input, and a huge thanks to the guys (and gals?) who write this stuff, so that those of us who can't code our way out of a wet paper bag can still satisify the demands of manglement!
DaZZa
I'd try to identify what makes this host unique if you have other Windows boxes being monitored. Do you have a partition size of 8tb and a full disk size of 15tb? Is there RAID involved?
SNMPv1 is only the means of identifying the host, it actually reads the data through MIBs. And if you're quite happy with the end product, you should think to donate, it's what keeps this project going!
-----Original Message----- From: observium [mailto:observium-bounces@observium.org] On Behalf Of DaZZa Sent: Wednesday, July 3, 2013 10:51 PM To: observium@observium.org Subject: [Observium] Windows server issue with disk space
Hi.
I found Observium after reading a post on Reddit (if you're out there, whoever made the post, you have my eternal thanks!), and I've got it setup and running pretty much OK, but I've run into an issue with one specific Windows host.
Observium accesses and sets it up OK except for one disk volume - the drive is approximately 15 Tb in size, but Observium is only reporting it as being 8Tb, and I can't figure out why.
Can anyone suggest what I may have configured incorrectly? I initially used SNMP V1 to discover/configure this host - could that be the issue, perhaps?
Thanks for any input, and a huge thanks to the guys (and gals?) who write this stuff, so that those of us who can't code our way out of a wet paper bag can still satisify the demands of manglement!
DaZZa _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
On 4 July 2013 12:59, Michael Sweikata sweikatam1@nku.edu wrote:
I'd try to identify what makes this host unique if you have other Windows boxes being monitored. Do you have a partition size of 8tb and a full disk size of 15tb? Is there RAID involved?
There is RAID involved - the disk space is on an external disk enclosure, and managed by an hardware RAID in the server. It's the only server I have where this type of disk arrangement is used.
The disk size is actually closer to 18 Tb - there are two partitions on the disk (dynamic volumes, Windows calls them), one of around 15 Tb and the other around 3 Tb. The 3Tb volume is reporting its size corectly
SNMPv1 is only the means of identifying the host, it actually reads the data through MIBs.
Yes, but SNMP V2 allows for 64 bit counters, whereas SNMP V1 doesn't - I was wondering if this volume was maybe replying with a greater sector count than SNMP V1 could deal with. I wasn't aware Windows supported SNMP V2, or I would have used that. I may delete the host and re-create it with SNMP V2 and see if that makes a difference.
Thanks.
DaZZa
Hi,
You can change to v2c in the device settings, no need to delete and re-add.
This may be a Windows problem, do you or anyone else have other machines with >16TB of diskspace on Windows?
For example, until Net-SNMP 5.7 there was no way to correctly monitor/graph/detect disk sizes over 16TB on Linux either. 5.7 added support for larger counters, which we now use.
There may be a similar limitation on Windows. (but I guess it is possible that this requires v2c and you're not using it)
Tom
On 4/07/2013 5:11, DaZZa wrote:
On 4 July 2013 12:59, Michael Sweikata sweikatam1@nku.edu wrote:
I'd try to identify what makes this host unique if you have other Windows boxes being monitored. Do you have a partition size of 8tb and a full disk size of 15tb? Is there RAID involved?
There is RAID involved - the disk space is on an external disk enclosure, and managed by an hardware RAID in the server. It's the only server I have where this type of disk arrangement is used.
The disk size is actually closer to 18 Tb - there are two partitions on the disk (dynamic volumes, Windows calls them), one of around 15 Tb and the other around 3 Tb. The 3Tb volume is reporting its size corectly
SNMPv1 is only the means of identifying the host, it actually reads the data through MIBs.
Yes, but SNMP V2 allows for 64 bit counters, whereas SNMP V1 doesn't - I was wondering if this volume was maybe replying with a greater sector count than SNMP V1 could deal with. I wasn't aware Windows supported SNMP V2, or I would have used that. I may delete the host and re-create it with SNMP V2 and see if that makes a difference.
Thanks.
DaZZa _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
On 4 July 2013 17:28, Tom Laermans tom.laermans@powersource.cx wrote:
Hi,
You can change to v2c in the device settings, no need to delete and re-add.
Yeah, I found that, but I wasn't 100% sure it would eliminate the problem, so I went down the "delete and recreate" path. It's not like there was historical data I needed anyway. :-)
This may be a Windows problem, do you or anyone else have other machines with >16TB of diskspace on Windows?
I don't - while my VMWare cluster has that much raw storage, it's broken into smaller chunks for compatibility reasons. This server is stand-alone.
For example, until Net-SNMP 5.7 there was no way to correctly monitor/graph/detect disk sizes over 16TB on Linux either. 5.7 added support for larger counters, which we now use.
There may be a similar limitation on Windows. (but I guess it is possible that this requires v2c and you're not using it)
Well, I'm now running the server with v2c, and it still doesn't read the volume size correctly.
It's not a major issue - I'll just set Observium to ignore the volume, since I have other monitors on it (users who will scream "IT!!!!" in a loud voice if it runs out of space), and the important volumes (the OS and system disks) are picked up OK.
Thanks.
DaZZa
Space looks OK to me, but I don't have large volumes (1.5TB is the largest) on my Windows machines.
How are your other tools monitoring this? Can you perhaps post a debug poller log? (strip out the community) - I wonder if Windows is reporting it wrong or if we're doing it wrong.
Tom
On 4/07/2013 23:40, DaZZa wrote:
On 4 July 2013 17:28, Tom Laermans tom.laermans@powersource.cx wrote:
Hi,
You can change to v2c in the device settings, no need to delete and re-add.
Yeah, I found that, but I wasn't 100% sure it would eliminate the problem, so I went down the "delete and recreate" path. It's not like there was historical data I needed anyway. :-)
This may be a Windows problem, do you or anyone else have other machines with >16TB of diskspace on Windows?
I don't - while my VMWare cluster has that much raw storage, it's broken into smaller chunks for compatibility reasons. This server is stand-alone.
For example, until Net-SNMP 5.7 there was no way to correctly monitor/graph/detect disk sizes over 16TB on Linux either. 5.7 added support for larger counters, which we now use.
There may be a similar limitation on Windows. (but I guess it is possible that this requires v2c and you're not using it)
Well, I'm now running the server with v2c, and it still doesn't read the volume size correctly.
It's not a major issue - I'll just set Observium to ignore the volume, since I have other monitors on it (users who will scream "IT!!!!" in a loud voice if it runs out of space), and the important volumes (the OS and system disks) are picked up OK.
Thanks.
DaZZa _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hi Tom.
I'll get a dump from SNMPwalk as well as a poller debug when I get back to work on Monday.
Thanks
DaZZa - apologies for the top-post On 07/07/2013 8:00 AM, "Tom Laermans" tom.laermans@powersource.cx wrote:
Space looks OK to me, but I don't have large volumes (1.5TB is the largest) on my Windows machines.
How are your other tools monitoring this? Can you perhaps post a debug poller log? (strip out the community) - I wonder if Windows is reporting it wrong or if we're doing it wrong.
Tom
On 4/07/2013 23:40, DaZZa wrote:
On 4 July 2013 17:28, Tom Laermans tom.laermans@powersource.cx wrote:
Hi,
You can change to v2c in the device settings, no need to delete and re-add.
Yeah, I found that, but I wasn't 100% sure it would eliminate the problem, so I went down the "delete and recreate" path. It's not like there was historical data I needed anyway. :-)
This may be a Windows problem, do you or anyone else have other machines
with >16TB of diskspace on Windows?
I don't - while my VMWare cluster has that much raw storage, it's broken into smaller chunks for compatibility reasons. This server is stand-alone.
For example, until Net-SNMP 5.7 there was no way to correctly
monitor/graph/detect disk sizes over 16TB on Linux either. 5.7 added support for larger counters, which we now use.
There may be a similar limitation on Windows. (but I guess it is possible that this requires v2c and you're not using it)
Well, I'm now running the server with v2c, and it still doesn't read the volume size correctly.
It's not a major issue - I'll just set Observium to ignore the volume, since I have other monitors on it (users who will scream "IT!!!!" in a loud voice if it runs out of space), and the important volumes (the OS and system disks) are picked up OK.
Thanks.
DaZZa ______________________________**_________________ observium mailing list observium@observium.org http://postman.memetic.org/**cgi-bin/mailman/listinfo/**observiumhttp://postman.memetic.org/cgi-bin/mailman/listinfo/observium
______________________________**_________________ observium mailing list observium@observium.org http://postman.memetic.org/**cgi-bin/mailman/listinfo/**observiumhttp://postman.memetic.org/cgi-bin/mailman/listinfo/observium
On 7 July 2013 08:00, Tom Laermans tom.laermans@powersource.cx wrote:
Space looks OK to me, but I don't have large volumes (1.5TB is the largest) on my Windows machines.
Interestingly, it *seems* to be SNMP reporting an invalid valud.
From the debug output, I get this [snipped for brevity]
Array ( [hrstorage] => Array ( [2] => Array ( [hrStorageIndex] => 2 [hrStorageType] => hrStorageFixedDisk [hrStorageDescr] => D:\ Label:Data Serial Number 56547c7f [hrStorageAllocationUnits] => 4096 [hrStorageSize] => -17524993 [hrStorageUsed] => 1328911526 [hrStorageAllocationFailures] => 0
I sure as hell don't have -17 gigabytes of storage on this volume. And the used space isn't coming back right either.
I can send you the entire poller dump, if you like. It's just a bit large to paste into an email as text.
How are your other tools monitoring this? Can you perhaps post a debug poller log? (strip out the community) - I wonder if Windows is reporting it wrong or if we're doing it wrong.
NAGIOS seems to read it correctly, and it's only using SNMP V1
Output from Nagios' status shows
D:\ Drive Space OK 07-08-2013 07:41:59 131d 0h 26m 21s 1/3 d: - total: 16317.15 Gb - used: 5069.40 Gb (31%) - free 11247.75 Gb (69%)
Interestingly, Observium does report another volume with negative space, vis-a-vis
[5] => Array ( [hrStorageIndex] => 5 [hrStorageType] => hrStorageFixedDisk [hrStorageDescr] => G:\ Label:MPEGS Serial Number 28ba4922 [hrStorageAllocationUnits] => 1024 [hrStorageSize] => -1870052353 [hrStorageUsed] => 1885592566 [hrStorageAllocationFailures] => 0 )
and despite my earlier assertion that this one was reporting OK, it's wrong too - the graphs/status shows this volume as being 3.74 Tb, whereas it's really only 2.25 Tb.
As I said, I'm willing to email you the output of the poller debug (I just dumped it to a file) and the output of a raw snmpwalk as well.
Cheers.
DaZZa
Hi,
If you could just check (or send) the snmpwalk output for the hr-storage part of the poller... if Nagios is seeing it correctly, it must be OK in SNMP - maybe PHP is doing some unsigned vs signed fuckuppery making this value negative instead of large positive? I'm just wondering how net-snmp sees that data.
Tom
On 7/07/2013 23:57, DaZZa wrote:
On 7 July 2013 08:00, Tom Laermans tom.laermans@powersource.cx wrote:
Space looks OK to me, but I don't have large volumes (1.5TB is the largest) on my Windows machines.
Interestingly, it *seems* to be SNMP reporting an invalid valud.
From the debug output, I get this [snipped for brevity]
Array ( [hrstorage] => Array ( [2] => Array ( [hrStorageIndex] => 2 [hrStorageType] => hrStorageFixedDisk [hrStorageDescr] => D:\ Label:Data Serial Number 56547c7f [hrStorageAllocationUnits] => 4096 [hrStorageSize] => -17524993 [hrStorageUsed] => 1328911526 [hrStorageAllocationFailures] => 0
I sure as hell don't have -17 gigabytes of storage on this volume. And the used space isn't coming back right either.
I can send you the entire poller dump, if you like. It's just a bit large to paste into an email as text.
How are your other tools monitoring this? Can you perhaps post a debug poller log? (strip out the community) - I wonder if Windows is reporting it wrong or if we're doing it wrong.
NAGIOS seems to read it correctly, and it's only using SNMP V1
Output from Nagios' status shows
D:\ Drive Space
OK 07-08-2013 07:41:59 131d 0h 26m 21s 1/3 d: - total: 16317.15 Gb - used: 5069.40 Gb (31%) - free 11247.75 Gb (69%)
Interestingly, Observium does report another volume with negative space, vis-a-vis
[5] => Array ( [hrStorageIndex] => 5 [hrStorageType] => hrStorageFixedDisk [hrStorageDescr] => G:\ Label:MPEGS Serial Number 28ba4922 [hrStorageAllocationUnits] => 1024 [hrStorageSize] => -1870052353 [hrStorageUsed] => 1885592566 [hrStorageAllocationFailures] => 0 )and despite my earlier assertion that this one was reporting OK, it's wrong too - the graphs/status shows this volume as being 3.74 Tb, whereas it's really only 2.25 Tb.
As I said, I'm willing to email you the output of the poller debug (I just dumped it to a file) and the output of a raw snmpwalk as well.
Cheers.
DaZZa _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
On 8 July 2013 08:02, Tom Laermans tom.laermans@powersource.cx wrote:
If you could just check (or send) the snmpwalk output for the hr-storage part of the poller... if Nagios is seeing it correctly, it must be OK in SNMP - maybe PHP is doing some unsigned vs signed fuckuppery making this value negative instead of large positive? I'm just wondering how net-snmp sees that data.
Interestingly, snmpwalk on my Observium server throws an error at the end of its run which I've never seen before, but which has got to be related to Windows (the Nagios server throws the same error with the same version of snmpwalk, so I know it's not the server)
Error in packet. Reason: (genError) A general failure occured Failed object: SNMPv2-SMI::mib-2.67.2.1.1.1.13
That's just after a general snmpwalk - weird, but I don't think it's what is causing the problem, since my Nagios host gets the same error (different machine, same version of snmpwalk)
Output of hr-storage from my poller debug as follows
/usr/bin/snmpbulkwalk -v2c -c <community> -OQUs -m HOST-RESOURCES-MIB:HOST-RESOURCES-TYPES -M /opt/observium/mibs udp:<host>:161 hrStorageEntry hrStorageIndex.1 = 1 hrStorageIndex.2 = 2 hrStorageIndex.3 = 3 hrStorageIndex.4 = 4 hrStorageIndex.5 = 5 hrStorageIndex.6 = 6 hrStorageIndex.7 = 7 hrStorageIndex.8 = 8 hrStorageType.1 = hrStorageFixedDisk hrStorageType.2 = hrStorageFixedDisk hrStorageType.3 = hrStorageCompactDisc hrStorageType.4 = hrStorageFixedDisk hrStorageType.5 = hrStorageFixedDisk hrStorageType.6 = hrStorageCompactDisc hrStorageType.7 = hrStorageVirtualMemory hrStorageType.8 = hrStorageRam hrStorageDescr.1 = C:\ Label: Serial Number 2cd320c9 hrStorageDescr.2 = D:\ Label:Data Serial Number 56547c7f hrStorageDescr.3 = E:\ hrStorageDescr.4 = F:\ Label:My Book Serial Number 1843759f hrStorageDescr.5 = G:\ Label:MPEGS Serial Number 28ba4922 hrStorageDescr.6 = H:\ hrStorageDescr.7 = Virtual Memory hrStorageDescr.8 = Physical Memory hrStorageAllocationUnits.1 = 4096 hrStorageAllocationUnits.2 = 4096 hrStorageAllocationUnits.3 = 0 hrStorageAllocationUnits.4 = 4096 hrStorageAllocationUnits.5 = 1024 hrStorageAllocationUnits.6 = 0 hrStorageAllocationUnits.7 = 65536 hrStorageAllocationUnits.8 = 65536 hrStorageSize.1 = 35832982 hrStorageSize.2 = -17524993 hrStorageSize.3 = 0 hrStorageSize.4 = 488370175 hrStorageSize.5 = -1870052353 hrStorageSize.6 = 0 hrStorageSize.7 = 392860 hrStorageSize.8 = 196444 hrStorageUsed.1 = 17982773 hrStorageUsed.2 = 1328911526 hrStorageUsed.3 = 0 hrStorageUsed.4 = 430612466 hrStorageUsed.5 = 1885592566 hrStorageUsed.6 = 0 hrStorageUsed.7 = 119440 hrStorageUsed.8 = 127275 hrStorageAllocationFailures.1 = 0 hrStorageAllocationFailures.2 = 0 hrStorageAllocationFailures.3 = 0 hrStorageAllocationFailures.4 = 0 hrStorageAllocationFailures.5 = 0 hrStorageAllocationFailures.6 = 0 hrStorageAllocationFailures.7 = 0 hrStorageAllocationFailures.8 = 0
Let me know if you need anything else
Cheers.
DaZZa
I've got a windows host (SBS 2008) that isn't reporting disk space for any of it's drives - none are over 1tb. Working fine in Cacti however. (And as to the server 2012 part of this thread - it looks like disk space may be gone out of server 2012's SNMP anyway)
Just FYI, if you plan to use Windows 2012 in future, there no SNMP service anymore. And current SNMP service barely improved since windows 2000.
On 04.07.2013 6:50, DaZZa wrote:
Hi.
I found Observium after reading a post on Reddit (if you're out there, whoever made the post, you have my eternal thanks!), and I've got it setup and running pretty much OK, but I've run into an issue with one specific Windows host.
Observium accesses and sets it up OK except for one disk volume - the drive is approximately 15 Tb in size, but Observium is only reporting it as being 8Tb, and I can't figure out why.
Can anyone suggest what I may have configured incorrectly? I initially used SNMP V1 to discover/configure this host - could that be the issue, perhaps?
Thanks for any input, and a huge thanks to the guys (and gals?) who write this stuff, so that those of us who can't code our way out of a wet paper bag can still satisify the demands of manglement!
DaZZa _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Hi All
SNMP can be found under "add features" of Server 2012
Daniel
-----Ursprüngliche Nachricht----- Von: observium im Auftrag von Nikolay Shopik Gesendet: Do 04.07.2013 10:43 An: Observium Network Observation System Betreff: Re: [Observium] Windows server issue with disk space
Just FYI, if you plan to use Windows 2012 in future, there no SNMP service anymore. And current SNMP service barely improved since windows 2000.
On 04.07.2013 6:50, DaZZa wrote:
Hi.
I found Observium after reading a post on Reddit (if you're out there, whoever made the post, you have my eternal thanks!), and I've got it setup and running pretty much OK, but I've run into an issue with one specific Windows host.
Observium accesses and sets it up OK except for one disk volume - the drive is approximately 15 Tb in size, but Observium is only reporting it as being 8Tb, and I can't figure out why.
Can anyone suggest what I may have configured incorrectly? I initially used SNMP V1 to discover/configure this host - could that be the issue, perhaps?
Thanks for any input, and a huge thanks to the guys (and gals?) who write this stuff, so that those of us who can't code our way out of a wet paper bag can still satisify the demands of manglement!
DaZZa _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
_______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
So it actually just deprecated but not completely removed.
http://technet.microsoft.com/en-us/library/hh831568.aspx
I though they get rid of it completly
On 04.07.2013 13:18, Daniel Fankhauser wrote:
Hi All
SNMP can be found under "add features" of Server 2012
Daniel
-----Ursprüngliche Nachricht----- Von: observium im Auftrag von Nikolay Shopik Gesendet: Do 04.07.2013 10:43 An: Observium Network Observation System Betreff: Re: [Observium] Windows server issue with disk space
Just FYI, if you plan to use Windows 2012 in future, there no SNMP service anymore. And current SNMP service barely improved since windows 2000.
On 04.07.2013 6:50, DaZZa wrote:
Hi.
I found Observium after reading a post on Reddit (if you're out there, whoever made the post, you have my eternal thanks!), and I've got it setup and running pretty much OK, but I've run into an issue with one specific Windows host.
Observium accesses and sets it up OK except for one disk volume - the drive is approximately 15 Tb in size, but Observium is only reporting it as being 8Tb, and I can't figure out why.
Can anyone suggest what I may have configured incorrectly? I initially used SNMP V1 to discover/configure this host - could that be the issue, perhaps?
Thanks for any input, and a huge thanks to the guys (and gals?) who write this stuff, so that those of us who can't code our way out of a wet paper bag can still satisify the demands of manglement!
DaZZa _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
On 4 July 2013 18:43, Nikolay Shopik shopik@inblock.ru wrote:
Just FYI, if you plan to use Windows 2012 in future, there no SNMP service anymore. And current SNMP service barely improved since windows 2000.
There is - it's pretty much like what happened to the telnet client in windows 2008 - if you want it, you have to add it as a "feature" after server install.
SNMP on Windows still sucks, though. :-)
DaZZa
Telnet "gone" back in windows vista timeframe btw ;)
On 05.07.2013, at 1:45, DaZZa dazzagibbs@gmail.com wrote:
On 4 July 2013 18:43, Nikolay Shopik shopik@inblock.ru wrote:
Just FYI, if you plan to use Windows 2012 in future, there no SNMP service anymore. And current SNMP service barely improved since windows 2000.
There is - it's pretty much like what happened to the telnet client in windows 2008 - if you want it, you have to add it as a "feature" after server install.
SNMP on Windows still sucks, though. :-)
DaZZa _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
Yes, that's Windows 2008.
I'll take a look at the Win2008 I have, and see what their diskspace is reporting.
Tom
On 6/07/2013 17:02, Nikolay Shopik wrote:
Telnet "gone" back in windows vista timeframe btw ;)
On 05.07.2013, at 1:45, DaZZa dazzagibbs@gmail.com wrote:
On 4 July 2013 18:43, Nikolay Shopik shopik@inblock.ru wrote:
Just FYI, if you plan to use Windows 2012 in future, there no SNMP service anymore. And current SNMP service barely improved since windows 2000.
There is - it's pretty much like what happened to the telnet client in windows 2008 - if you want it, you have to add it as a "feature" after server install.
SNMP on Windows still sucks, though. :-)
DaZZa _______________________________________________ observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
observium mailing list observium@observium.org http://postman.memetic.org/cgi-bin/mailman/listinfo/observium
participants (6)
-
Brian -
Daniel Fankhauser -
DaZZa -
Michael Sweikata -
Nikolay Shopik -
Tom Laermans