-
Notifications
You must be signed in to change notification settings - Fork 8
/
README
419 lines (382 loc) · 24.8 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
Theia is a NetApp filer monitoring tool written in Python that can send alerts to Nagios via passive checks and
write csv files for Zenoss (or other tool) consumption to process and display performance metrics. Generally speaking,
Theia must run as root (this will change in the near future0 primarily because it needs privileged access to the filers
in order to display all available metrics.
INSTALLATION
* git clone [email protected]:ning/Theia.git
* cp Theia/etc/theia.cfg /etc/
* cp Theia/bin/theia /usr/local/bin
* chmod +x /usr/local/bin/theia
* cp Theia/etc/xnsingleton.xml <filer>:/<root_volume>/etc/stats/preset/
The destinations paths are examples: you can install the theia binary wherever is convenient; the configuration file is
entirely arbitrary (it defaults to /etc/theia.cfg, but you can use the --config option to specify another location).
There is some amount of hardcoded configuration options that currently require tweaking the code to use. They affect
the Nagios functionality; if you don't plan on using these, no changes are required. The send_nsca(0 function uses
hardcoded locations for the send_nsca binary and configuration file as well as the target Nagios host to submit test
results to. Adjust the definition of the cmd variable to your needs:
cmd = "/usr/local/bin/send_nsca -H nagios -c /usr/local/etc/nagios/send_nsca.conf"
CONFIGURATION
Overview
Configuring Theia is a relatively simple task, as it uses a simple format to define metrics to watch and alert on. This
format is based on ONTAP's metric organization, where stats counters are grouped by what object they provide data for.
Object can have zero or more instances. We extended that model to be prefixed by a filer's hostname. Thus, any given
counter (metric) can be denoted as follows (in Theia):
filer:object:instance:value
See "Storage system information and the stats command" section in the NetApp's ONTAP Systems Administration Guide.
In theia.cfg, entries are as follows:
filer:object:instance:counter:critical:warning:op
where <filer> denotes the filer's hostname as shown by the "hostname" command, object, instance and counters are part
of the ONTAP stats framework (with some exceptions), and critical and warning denote values that would trigger a
critical or warning alert respectively when compared via the op to the current value, where op is one of '<', '>', '='
or '-' (don't compare). Wildcards ('*") are allowed for filer names and instances and _must_ be defined before entries
for a specific filer. Partial wild-carding is not supported (ie, 'processor1*' is not supported).
The provided theia.cfg contains a working configuration file with sample metrics and thresholds.
The Difference Between <filer> and <hostname>
In Theia's world, a hostname is the FQDN of a filer system (in the context of DNS name resolution) whereas a filer
name is the non-qualified name of the filer as returned when running the hostname command on the filer. In the CLI,
Theia can use either one: when called with a FQDN, Theia will remove the domain part of the name and use the resulting
string as the filer name. This split is done at the first '.' (if any) found. Thus, netapp.example.com is a hostname
that results in a netapp filer name. The configuration file always uses the filer name.
Stats Preset Configuration
Theia uses the stats command in singleton mode in case other stats probes are running, and maps it to a stats preset
assumed to be named xnsingleton.xml, located in the filer's root volume at etc/stats/presets/xnsingleton.xml. This
preset defines the set of metrics we're interested in collecting (as otherwise stats returns all metrics). A sample
file we use across our filers is provided as well.
From Theia's point of view, you can collect and alert on whatever counter you wish. The sysstat counters, however, are
not configurable (see below).
Sysstat
Theia also captures data via the systat command (data that is not available through the stats command). This data is
injected into the object:instance:value tree as part of the system and nfsv3 objects:
xnmetrics = {
'APB' : "system:system:avg_processor_busy",
'CIFSO' : None,
'CPU' : "system:system:cpu_busy",
'CPtime' : "system:system:cp_time",
'CacheAge' : "system:system:cache_age",
'CacheHit' : "system:system:cache_hit",
'DiskRead' : "system:system:disk_data_read",
'DiskUtil' : "system:system:disk_busy",
'DiskWrite' : "system:system:disk_data_written",
'HTTPO' : None,
'NFSO' : "nfsv3:nfs:nfsv3_ops",
'NFSRL' : "nfsv3:nfs:nfsv3_read_latency",
'NFSRO' : "nfsv3:nfs:nfsv3_read_ops",
'NFSWL' : "nfsv3:nfs:nfsv3_write_latency",
'NFSWO' : "nfsv3:nfs:nfsv3_write_ops",
'NetIn' : "system:system:net_data_recv",
'NetOut' : "system:system:net_data_sent",
'TPB' : "system:system:total_processor_busy",
'TapeRead' : None,
'TapeWrite' : None,
'TotalO' : "system:system:total_ops",
'cpu0' : "processor:processor0:processor_busy",
'cpu1' : "processor:processor1:processor_busy",
'cpu2' : "processor:processor2:processor_busy",
'cpu3' : "processor:processor3:processor_busy"
}
This approach is nice in that when it comes to evaluate counters, the code is homogeneous (if not necessarily pretty).
But it also hardcodes the number of CPU's available in your filers. A fix for this is upcoming.
SYNTAX
theia [-h|--help] [--nagios] [--debug] [--datafile=<datafile>] [--config=<configfile>] --sshkeyfile=<file> <filer>
where:
--datafile: file containing sample sysstat and stats output (useful for development and debugging)
--config: location of configuration file
--sshkeyfile: location of SSH keys used to access the filer without entring a password (passwordless or agent)
--nagios: enable threshold evaluation and nagios passive test submission
--pickle: location of pickle file for metrics collected (system, volumes and aggregates)
--debug: debug output
RUNNING
Theia is intended to be run from cron at a frequency no higher than 5 mimutes. Before you run Theia for the first time,
you must connect with the appropriate SSH key and verify the host you're connecting to (i.e., theia expects that the
filer is already present in ~/.ssh/known_hosts).
ONTAP SUPPORT
Theia has been tested with ONTAP version 7.3 and 8.0 on FAZ60x0A filers and the ONTAP Simulator. It is known to not work
on small filers (2000 series).
SAMPLE RUN (CLI)
This is a sample run against the ONTAP Simulator running version 8:
root@adminhost:~#theia --sshkeyfile=/root/.ssh/id_rsa.nap --debug subzero
xtheia: debug: {'datafile': None, 'nop': False, 'sshkeyfile': '/root/.ssh/id_rsa.nap', 'debug': True, 'pickle': None, 'nagios': False, 'config': '/etc/theia.cfg'}
xtheia: debug: filer: subzero (subzero)
xtheia: debug: reading /etc/theia.cfg configuration file
xtheia: debug: line: *:system:system:avg_processor_busy:90:95:>
xtheia: debug: line: *:system:system:total_processor_busy:350:370:>
xtheia: debug: line: *:processor:*:processor_busy:90:95:>
xtheia: debug: line: *:nfsv3:nfs:nfsv3_ops:40000:50000:>
xtheia: debug: line: *:nfsv3:nfs:nfsv3_read_latency:20:15:>
xtheia: debug: line: *:nfsv3:nfs:nfsv3_write_latency:7:15:>
xtheia: debug: line: *:system:system:cp_time:90:100:>
xtheia: debug: line: *:system:system:cache_hit:10:20:<
xtheia: debug: line: *:system:system:disk_busy:87:95:-
xtheia: debug: line: *:volume:*:df_i_capacity:82:87:>
xtheia: debug: line: *:volume:*:df_k_capacity:98:99:>
xtheia: debug: line: *:volume:root:df_k_capacity:50:60:>
xtheia: debug: thres: [system:system:avg_processor_busy] => w:90 c:95 op:>
xtheia: debug: thres: [system:system:total_processor_busy] => w:350 c:370 op:>
xtheia: debug: thres: [processor:*:processor_busy] => w:90 c:95 op:>
xtheia: debug: thres: [nfsv3:nfs:nfsv3_ops] => w:40000 c:50000 op:>
xtheia: debug: thres: [nfsv3:nfs:nfsv3_read_latency] => w:20 c:15 op:>
xtheia: debug: thres: [nfsv3:nfs:nfsv3_write_latency] => w:7 c:15 op:>
xtheia: debug: thres: [system:system:cp_time] => w:90 c:100 op:>
xtheia: debug: thres: [system:system:cache_hit] => w:10 c:20 op:<
xtheia: debug: thres: [volume:*:df_i_capacity] => w:82 c:87 op:>
xtheia: debug: thres: [volume:*:df_k_capacity] => w:98 c:99 op:>
xtheia: debug: thres: [volume:root:df_k_capacity] => w:50 c:60 op:>
xtheia: debug: line: [sysstat]
xtheia: debug: line: CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk OTHER FCP iSCSI FCP kB/s iSCSI kB/s
xtheia: debug: line: in out read write read write age hit time ty util in out in out
xtheia: debug: line: 14% 78 0 0 81 2271 73 1462 3332 0 0 3 100% 100% Bf 94% 3 0 0 0 0 0 0
xtheia: debug: line: 13% 77 0 0 96 2822 83 1200 3837 0 0 >60 100% 100% Bf 94% 19 0 0 0 0 0 0
xtheia: debug: line: 14% 147 0 0 148 4735 144 1801 3682 0 0 3 99% 100% Bf 90% 1 0 0 0 0 0 0
xtheia: debug: line: 12% 69 0 0 70 2367 71 1256 3877 0 0 3 100% 100% Bf 95% 1 0 0 0 0 0 0
xtheia: debug: line: 12% 69 0 0 72 2374 72 1166 3712 0 0 3 100% 100% Bf 94% 3 0 0 0 0 0 0
xtheia: debug: line: 13% 69 0 0 70 2376 71 1165 3584 0 0 3 100% 100% Bv 89% 1 0 0 0 0 0 0
xtheia: debug: line: 18% 137 0 0 139 4421 136 1334 3230 0 0 3 100% 100% Bf 91% 2 0 0 0 0 0 0
xtheia: debug: line: 13% 69 0 0 70 2692 80 761 4020 0 0 3 100% 100% Bf 93% 1 0 0 0 0 0 0
xtheia: debug: line: 13% 69 0 0 72 2374 71 1287 3591 0 0 3 100% 100% Bf 96% 3 0 0 0 0 0 0
xtheia: debug: line: 16% 137 0 0 185 4731 174 854 3869 0 0 3 100% 100% Bf 92% 48 0 0 0 0 0 0
xtheia: debug: line: [sysstat end]
xtheia: debug: stat: [system][system][cp_time][100%]
xtheia: debug: stat: [system][system][cache_hit][99%]
xtheia: debug: stat: [system][system][cache_age][8s]
xtheia: debug: stat: [system][system][disk_busy][92%]
xtheia: debug: stat: [system][system][total_ops][100/s]
xtheia: debug: line: [stats show]
xtheia: debug: line: StatisticsID: xnsingleton
xtheia: debug: line: processor:processor0:processor_busy:17%
xtheia: debug: stat: [processor][processor0][processor_busy][17%]
xtheia: debug: line: processor:processor1:processor_busy:17%
xtheia: debug: stat: [processor][processor1][processor_busy][17%]
xtheia: debug: line: processor:processor2:processor_busy:16%
xtheia: debug: stat: [processor][processor2][processor_busy][16%]
xtheia: debug: line: processor:processor3:processor_busy:2%
xtheia: debug: stat: [processor][processor3][processor_busy][2%]
xtheia: debug: line: system:system:cpu_busy:14%
xtheia: debug: stat: [system][system][cpu_busy][14%]
xtheia: debug: line: system:system:avg_processor_busy:13%
xtheia: debug: stat: [system][system][avg_processor_busy][13%]
xtheia: debug: line: system:system:total_processor_busy:53%
xtheia: debug: stat: [system][system][total_processor_busy][53%]
xtheia: debug: line: system:system:net_data_recv:2844KB/s
xtheia: debug: stat: [system][system][net_data_recv][2844KB/s]
xtheia: debug: line: system:system:net_data_sent:89KB/s
xtheia: debug: stat: [system][system][net_data_sent][89KB/s]
xtheia: debug: line: system:system:disk_data_read:1162KB/s
xtheia: debug: stat: [system][system][disk_data_read][1162KB/s]
xtheia: debug: line: system:system:disk_data_written:3490KB/s
xtheia: debug: stat: [system][system][disk_data_written][3490KB/s]
xtheia: debug: line: ifnet:e0a:send_errors:0/s
xtheia: debug: stat: [ifnet][e0a][send_errors][0/s]
xtheia: debug: line: ifnet:e0a:recv_errors:0/s
xtheia: debug: stat: [ifnet][e0a][recv_errors][0/s]
xtheia: debug: line: ifnet:e0a:recv_drop_packets:0/s
xtheia: debug: stat: [ifnet][e0a][recv_drop_packets][0/s]
xtheia: debug: line: ifnet:e0b:send_errors:0/s
xtheia: debug: stat: [ifnet][e0b][send_errors][0/s]
xtheia: debug: line: ifnet:e0b:recv_errors:80218286/s
xtheia: debug: stat: [ifnet][e0b][recv_errors][80218286/s]
xtheia: debug: line: ifnet:e0b:recv_drop_packets:0/s
xtheia: debug: stat: [ifnet][e0b][recv_drop_packets][0/s]
xtheia: debug: line: ifnet:e0c:send_errors:0/s
xtheia: debug: stat: [ifnet][e0c][send_errors][0/s]
xtheia: debug: line: ifnet:e0c:recv_errors:80218286/s
xtheia: debug: stat: [ifnet][e0c][recv_errors][80218286/s]
xtheia: debug: line: ifnet:e0c:recv_drop_packets:0/s
xtheia: debug: stat: [ifnet][e0c][recv_drop_packets][0/s]
xtheia: debug: line: ifnet:e0d:send_errors:0/s
xtheia: debug: stat: [ifnet][e0d][send_errors][0/s]
xtheia: debug: line: ifnet:e0d:recv_errors:80218286/s
xtheia: debug: stat: [ifnet][e0d][recv_errors][80218286/s]
xtheia: debug: line: ifnet:e0d:recv_drop_packets:0/s
xtheia: debug: stat: [ifnet][e0d][recv_drop_packets][0/s]
xtheia: debug: line: ifnet:losk:send_errors:0/s
xtheia: debug: stat: [ifnet][losk][send_errors][0/s]
xtheia: debug: line: ifnet:losk:recv_errors:0/s
xtheia: debug: stat: [ifnet][losk][recv_errors][0/s]
xtheia: debug: line: ifnet:losk:recv_drop_packets:0/s
xtheia: debug: stat: [ifnet][losk][recv_drop_packets][0/s]
xtheia: debug: line: nfsv3:nfs:nfsv3_ops:86/s
xtheia: debug: stat: [nfsv3][nfs][nfsv3_ops][86/s]
xtheia: debug: line: nfsv3:nfs:nfsv3_read_ops:0/s
xtheia: debug: stat: [nfsv3][nfs][nfsv3_read_ops][0/s]
xtheia: debug: line: nfsv3:nfs:nfsv3_write_ops:83/s
xtheia: debug: stat: [nfsv3][nfs][nfsv3_write_ops][83/s]
xtheia: debug: line: nfsv3:nfs:nfsv3_read_latency:0ms
xtheia: debug: stat: [nfsv3][nfs][nfsv3_read_latency][0ms]
xtheia: debug: line: nfsv3:nfs:nfsv3_write_latency:632.25ms
xtheia: debug: stat: [nfsv3][nfs][nfsv3_write_latency][632.25ms]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.null:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.null][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.getattr:36
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.getattr][36]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.setattr:57
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.setattr][57]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.lookup:18
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.lookup][18]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.access:12
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.access][12]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.readlink:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.readlink][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.read:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.read][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.write:4468
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.write][4468]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.create:8
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.create][8]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.mkdir:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.mkdir][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.symlink:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.symlink][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.mknod:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.mknod][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.remove:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.remove][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.rmdir:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.rmdir][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.rename:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.rename][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.link:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.link][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.readdir:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.readdir][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.readdirplus:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.readdirplus][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.fsstat:7
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.fsstat][7]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.fsinfo:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.fsinfo][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.pathconf:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.pathconf][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_count.commit:0
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_count.commit][0]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.null:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.null][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.getattr:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.getattr][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.setattr:1%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.setattr][1%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.lookup:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.lookup][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.access:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.access][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.readlink:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.readlink][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.read:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.read][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.write:97%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.write][97%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.create:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.create][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.mkdir:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.mkdir][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.symlink:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.symlink][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.mknod:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.mknod][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.remove:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.remove][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.rmdir:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.rmdir][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.rename:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.rename][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.link:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.link][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.readdir:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.readdir][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.readdirplus:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.readdirplus][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.fsstat:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.fsstat][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.fsinfo:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.fsinfo][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.pathconf:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.pathconf][0%]
xtheia: debug: line: nfsv3:nfs:nfsv3_op_percent.commit:0%
xtheia: debug: stat: [nfsv3][nfs][nfsv3_op_percent.commit][0%]
xtheia: debug: line: aggregate:aggr0:cp_reads:44/s
xtheia: debug: stat: [aggregate][aggr0][cp_reads][44/s]
xtheia: debug: line: aggregate:aggr0:user_reads:2/s
xtheia: debug: stat: [aggregate][aggr0][user_reads][2/s]
xtheia: debug: line: aggregate:aggr0:user_writes:115/s
xtheia: debug: stat: [aggregate][aggr0][user_writes][115/s]
xtheia: debug: line: aggregate:aggr0:wv_fsinfo_blks_total:5632000
xtheia: debug: stat: [aggregate][aggr0][wv_fsinfo_blks_total][5632000]
xtheia: debug: line: aggregate:aggr0:wv_fsinfo_blks_reserve:563200
xtheia: debug: stat: [aggregate][aggr0][wv_fsinfo_blks_reserve][563200]
xtheia: debug: line: aggregate:aggr0:wv_fsinfo_blks_used:331447
xtheia: debug: stat: [aggregate][aggr0][wv_fsinfo_blks_used][331447]
xtheia: debug: line: aggregate:aggr0:wvblk_snap_reserve:0
xtheia: debug: stat: [aggregate][aggr0][wvblk_snap_reserve][0]
xtheia: debug: line: volume:root:avg_latency:13742.26us
xtheia: debug: stat: [volume][root][avg_latency][13742.26us]
xtheia: debug: line: volume:root:write_data:2186b/s
xtheia: debug: stat: [volume][root][write_data][2186b/s]
xtheia: debug: line: volume:root:read_data:12418b/s
xtheia: debug: stat: [volume][root][read_data][12418b/s]
xtheia: debug: line: volume:root:write_latency:95289.73us
xtheia: debug: stat: [volume][root][write_latency][95289.73us]
xtheia: debug: line: volume:root:nfs_read_data:12361b/s
xtheia: debug: stat: [volume][root][nfs_read_data][12361b/s]
xtheia: debug: line: volume:root:nfs_read_latency:43.72us
xtheia: debug: stat: [volume][root][nfs_read_latency][43.72us]
xtheia: debug: line: volume:root:nfs_read_ops:1/s
xtheia: debug: stat: [volume][root][nfs_read_ops][1/s]
xtheia: debug: line: volume:root:nfs_write_data:241b/s
xtheia: debug: stat: [volume][root][nfs_write_data][241b/s]
xtheia: debug: line: volume:root:nfs_write_latency:83920.08us
xtheia: debug: stat: [volume][root][nfs_write_latency][83920.08us]
xtheia: debug: line: volume:root:nfs_write_ops:1/s
xtheia: debug: stat: [volume][root][nfs_write_ops][1/s]
xtheia: debug: line: volume:root:wv_fsinfo_blks_total:291280
xtheia: debug: stat: [volume][root][wv_fsinfo_blks_total][291280]
xtheia: debug: line: volume:root:wv_fsinfo_blks_reserve:29136
xtheia: debug: stat: [volume][root][wv_fsinfo_blks_reserve][29136]
xtheia: debug: line: volume:root:wv_fsinfo_blks_used:52694
xtheia: debug: stat: [volume][root][wv_fsinfo_blks_used][52694]
xtheia: debug: line: volume:data:avg_latency:581745.24us
xtheia: debug: stat: [volume][data][avg_latency][581745.24us]
xtheia: debug: line: volume:data:write_data:2731574b/s
xtheia: debug: stat: [volume][data][write_data][2731574b/s]
xtheia: debug: line: volume:data:read_data:0b/s
xtheia: debug: stat: [volume][data][read_data][0b/s]
xtheia: debug: line: volume:data:write_latency:631328.37us
xtheia: debug: stat: [volume][data][write_latency][631328.37us]
xtheia: debug: line: volume:data:nfs_read_data:0b/s
xtheia: debug: stat: [volume][data][nfs_read_data][0b/s]
xtheia: debug: line: volume:data:nfs_read_latency:0us
xtheia: debug: stat: [volume][data][nfs_read_latency][0us]
xtheia: debug: line: volume:data:nfs_read_ops:0/s
xtheia: debug: stat: [volume][data][nfs_read_ops][0/s]
xtheia: debug: line: volume:data:nfs_write_data:2731574b/s
xtheia: debug: stat: [volume][data][nfs_write_data][2731574b/s]
xtheia: debug: line: volume:data:nfs_write_latency:631328.37us
xtheia: debug: stat: [volume][data][nfs_write_latency][631328.37us]
xtheia: debug: line: volume:data:nfs_write_ops:83/s
xtheia: debug: stat: [volume][data][nfs_write_ops][83/s]
xtheia: debug: line: volume:data:wv_fsinfo_blks_total:4369072
xtheia: debug: stat: [volume][data][wv_fsinfo_blks_total][4369072]
xtheia: debug: line: volume:data:wv_fsinfo_blks_reserve:436912
xtheia: debug: stat: [volume][data][wv_fsinfo_blks_reserve][436912]
xtheia: debug: line: volume:data:wv_fsinfo_blks_used:224300
xtheia: debug: stat: [volume][data][wv_fsinfo_blks_used][224300]
xtheia: debug: line: [stats start]
xtheia: debug: line: Stats identifier 'xnsingleton' in use, resetting.
xtheia: debug: line: [df -k]
xtheia: debug: line: Filesystem total used avail capacity Mounted on
xtheia: debug: line: /vol/root/ 996148KB 158380KB 837768KB 16% /vol/root/
xtheia: debug: stat: [volume][root][df_k_capacity][16%]
xtheia: debug: line: /vol/root/.snapshot 52428KB 81132KB 0KB 155% /vol/root/.snapshot
xtheia: debug: line: /vol/data/ 14942208KB 908188KB 14034020KB 6% /vol/data/
xtheia: debug: stat: [volume][data][df_k_capacity][6%]
xtheia: debug: line: /vol/data/.snapshot 786432KB 0KB 786432KB 0% /vol/data/.snapshot
xtheia: debug: line: [df -i]
xtheia: debug: line: Filesystem iused ifree %iused Mounted on
xtheia: debug: line: /vol/root/ 4932 26190 16% /vol/root/
xtheia: debug: stat: [volume][root][df_i_capacity][16%]
xtheia: debug: line: /vol/data/ 1610 465320 0% /vol/data/
xtheia: debug: stat: [volume][data][df_i_capacity][0%]
xtheia: debug: line: [end]
This run does not perform threshold evaluation (since the --nagios flag was not provided).
FUTURES
* Remove all traces of hardcoded "stuff"
* Rework the pickle code to be less picky about what it outputs
* Provide Zenoss integration tool (use pickled data)