ONTAP performance monitoring & perfstat

Start broad, preserve a meaningful sample window, then descend through client, LIF, volume and disk evidence. One busy counter is a clue—not a diagnosis.

The monitoring toolkit

Tool	Use	Boundary
`statistics show`	Read a completed sample for selected objects, instances and counters.	Advanced privilege; counter availability varies.
`statistics start/stop`	Collect a named background sample, including concurrent samples.	Scope narrowly: broad collection can affect the system.
`sysstat`	Node-shell real-time summary of CPU, protocol ops, disk and network throughput.	Broad triage, not workload attribution.
`perfstat`	Support-oriented, full-system configuration and performance capture.	Use the package and invocation supplied for your case/version.
Active IQ / Unified Manager	History, alerting, trends and capacity context.	Depends on telemetry and product configuration; see AutoSupport, EMS & Active IQ.

cluster::> set -privilege advanced
cluster::*> system node run -node node1 sysstat -x 1
 CPU  NFS  CIFS  Total  Net kB/s  Disk kB/s  Tape kB/s  Cache
 31%  820   210   1510      48420      39180          0    94%
 46% 1205   308   2194      70110      62240          0    91%

Read CPU as processor utilization; protocol columns as operation rates; Total as the combined operation rate; network and disk columns as throughput; and cache as the reported cache-hit percentage. Exact columns/options differ by platform and release—run sysstat -h in the node shell and check your version.

Counter manager: objects, instances, counters

An object is the measured class (system, volume, lun, aggregate, disk); an instance is one member; a counter is one metric. Catalog before scripting because names and object scope can change.

cluster::*> statistics catalog object show -object volume
Object: volume
Description: These counters report activity at the volume layer.
Privilege: advanced

cluster::*> statistics catalog counter show -object system -counter cpu_busy
Object  Counter   Description                         Unit
system  cpu_busy  Percentage of elapsed time CPU busy percent

Live and background samples

The sample identifier belongs to a collection. The 9.19.1 reference uses a pipe between multiple counters; some displayed examples elsewhere use commas, so confirm with active help. A defensible explicit workflow is:

cluster::*> statistics start -object volume -counter read_data|write_data -sample-id 10
Statistics collection is being started for Sample-id: 10
cluster::*> statistics stop -sample-id 10
Statistics collection is being stopped for Sample-id: 10
cluster::*> statistics show -object volume -counter read_data,write_data -sample-id 10
Instance        read_data   write_data
svm1:db_data    812MB       244MB
svm1:archive     18MB         7MB

cluster::*> statistics start -object system -counter avg_processor_busy|cpu_busy -sample-id peak15 -duration 15
Statistics collection is being started for Sample-id: peak15
cluster::*> statistics show -object system -counter cpu_busy -sample-id peak15
Instance  cpu_busy
node1     73%
node2     29%

Latency isolation: client → LIF → volume → disk

Record client latency and time boundaries; rule out host queueing and path failover.
Check the serving LIF and port for errors, saturation or an unexpected home/node path.
Rank volumes during the same interval, then correlate IOPS, throughput and latency.
Inspect the disks backing the implicated aggregate. Do not infer disk pressure from volume latency alone.

cluster::*> statistics start -object volume -counter avg_latency|total_ops -sample-id incident42 -duration 5
cluster::*> statistics show -object volume -counter avg_latency,total_ops -sample-id incident42
Instance       avg_latency  total_ops
svm1:db_data   8.7ms        18420
svm1:logs      1.2ms         6310

cluster::*> statistics start -object disk -sample-id disk42 -duration 5
cluster::*> statistics show -object disk -sample-id disk42
Instance       disk_busy  read_ops  write_ops
1.0.3          91%        544       328
1.0.4          48%        276       151

Counter units and instance spelling are authoritative in statistics catalog counter show. “Client latency” includes more than storage; align timestamps before blaming ONTAP.

Hotspot signatures

Signature	Evidence to seek	Next move
CPU-bound	Sustained high node CPU while disk/network headroom remains; ops rise with CPU.	Find expensive protocol/workload mix; inspect QoS and node placement.
Network-bound	Port throughput near its practical ceiling, errors/drops or uneven LIF placement; disks not busy.	Verify link, MTU end to end, flow distribution and LIF home ports.
Disk-bound	High disk busy/latency across the workload aggregate while CPU/network have headroom.	Find hot disks/RAID groups, workload skew, capacity tiering or failing media.
Single hot volume	One volume dominates ops/bytes/latency; peers are healthy.	Correlate clients and QoS; consider placement or workload remediation.
False correlation	A high instantaneous value without aligned client impact.	Collect a longer, named sample and compare a known-good period.

Historical baselines

Capture the same counters, scope and interval during normal business peaks, batch windows and backup periods. Store timestamp, ONTAP version, workload event, object/instance filters and raw output off-cluster. Compare percentiles and shapes, not only averages; after upgrades or hardware/layout changes, establish a new baseline.

cluster::*> statistics start -object system|volume|aggregate -sample-id baseline_aug -duration 15
Statistics collection is being started for Sample-id: baseline_aug
cluster::*> statistics show -sample-id baseline_aug
Object     Instance       Counter             Value
system     node1          cpu_busy            36%
volume     svm1:db_data   avg_latency         1.4ms
aggregate  node1_aggr1    total_transfers     9221/s

perfstat collection runbook

perfstat is a NetApp diagnostic collection utility, not a stable ONTAP cluster-shell command. NetApp support normally supplies or points to the appropriate package and requests a duration/sample count matched to the incident. Do not invent a universal perfstat -v command: -v semantics and supported options depend on the tool build. Run the downloaded binary’s -h/--help, follow the case instructions, and check your version.

Record cluster, case, incident time/time zone and affected clients.
Place the approved tool on a secure admin host with management reachability and sufficient free space.
Review help and support’s requested frequency/duration; avoid an unnecessarily short window.
Start before reproduction, annotate the incident, stop after recovery, verify the archive, then transfer it through the approved support channel.

admin-host$ ./perfstat --help
Usage: perfstat [options] <controller>
# Option names vary by perfstat release; use the package supplied for the case.
admin-host$ sha256sum perfstat capture_case_2001234567.tgz
8ed8...c1a2  perfstat
4a91...52bf  capture_case_2001234567.tgz

Common gotchas

Wrong counter name: catalog it on the installed release; aliases seen in a case note may not exist.
Too-short samples: one second can show a spike but miss queue buildup and workload phases.
Collection scope: the docs warn that broad statistics start scope can materially affect performance.
Rate versus cumulative value: verify the counter description and unit before comparing.
i2p versus system counters: internal processing-stage counters answer different questions than end-to-end system or volume counters. Do not add them together or treat one layer as client latency.
Clock mismatch: synchronize and document time zones; see NTP and name services.

Official ONTAP sources

Examples are realistic illustrations, not captured from your system. Validate counters, units and privileges against the selected ONTAP release.