ONTAP HA pairs: takeover, giveback, and recovery

An HA pair is two controllers that mirror in-flight write intent and can serve each other’s storage. Takeover transfers the unavailable node’s storage service to its partner; giveback returns it after recovery. The mechanism protects a controller failure—not every shared-storage or site-wide failure.

Two ONTAP nodes mirroring NVRAM and controlling partner storage during takeover

Architecture, failover domains, and terminology

Each node normally owns aggregates and serves their volumes. Client writes are logged locally and mirrored to the partner’s NVRAM/NVMEM before acknowledgement; see the NVRAM write-path guide. The HA interconnect carries mirror and heartbeat traffic. During takeover, the survivor imports the partner-owned aggregates and replays protected writes as required.

SFO (storage failover) is current clustered ONTAP terminology for controller-pair takeover/giveback.
CF/CFO (controller or cluster failover) appears in older 7-Mode and some low-level/root-aggregate wording. It is not the same as cluster quorum or a MetroCluster site switchover.
The failover domain is the HA partner pair. A four-node cluster contains two separate HA pairs; node 3 cannot arbitrarily take over node 1’s disks.

Takeover can follow panic, power loss, missed heartbeat/hardware-assisted notification, boot/halt operations, or an operator command. MetroCluster adds site-level behavior; that workflow is intentionally outside this page—see MetroCluster fundamentals.

Read `storage failover show`

cluster1::> storage failover show
                              Takeover
Node           Partner        Possible State Description
-------------- -------------- -------- -------------------------------------
node1          node2          true     Connected to node2
node2          node1          true     Connected to node1
2 entries were displayed.

Takeover Possible: true is the minimum green light, not the whole precheck. Detailed output can expose NVRAM mismatch, interconnect error, partner booting, version mismatch, mailbox uncertainty, low memory, takeover already in progress, or log-not-synchronized states.

cluster1::> storage failover show -node node1 -instance
cluster1::> storage failover show -fields node,partner,takeover-possible,state-description
cluster1::> storage failover check-takeover -node node1

Planned takeover runbook

Confirm both nodes healthy, takeover possible, quorum healthy, no aggregate/disk path issue, and host multipathing/NAS LIF failover ready.
Record performance headroom on the survivor and pause high-load jobs.
Start a normal negotiated takeover. Watch state before touching hardware.

cluster1::> storage failover takeover -ofnode node2
Warning: A takeover will be initiated. Continue? {y|n}: y
Takeover initiated. Run "storage failover show-takeover" for progress.

cluster1::> storage failover show-takeover

A normal takeover gives the partner time to close resources and can optimize aggregate/LIF movement. The 9.19.1 command uses -bypass-optimization for the negotiated optimization path; -option immediate is more disruptive and explicitly bypasses clean shutdown behavior.

# Only after understanding why optimization cannot complete
cluster1::> storage failover takeover -ofnode node2 -bypass-optimization true

# Emergency/advanced choice; risks disruption and can break NDU
cluster1::> storage failover takeover -ofnode node2 -option immediate

There is no standalone 9.19.1 storage failover override command. “Override” means a documented takeover option or a giveback -override-vetoes parameter. Never substitute a guessed command.

Giveback and automatic giveback

After the repaired node boots to “waiting for giveback,” confirm it is stable and its HA links, disks, keys and network ports are healthy. Then return storage:

cluster1::> storage failover giveback -ofnode node2
Warning: Initiating giveback. Continue? {y|n}: y
Giveback initiated.

cluster1::> storage failover show-giveback
Partner     Aggregate       Giveback Status
----------- --------------- -------------------------------
node2       node2_aggr1     completed

A subsystem can veto giveback because an operation makes it unsafe or disruptive. Read the message and EMS, correct the cause, then retry. Override only when NetApp’s corrective procedure says the specific veto is safe to bypass.

cluster1::> event log show -time >30m -severity *
cluster1::> storage failover giveback -ofnode node2 -override-vetoes true
# Advanced/risky: do not use for missing encryption keys or unknown disk state.

cluster1::> storage failover modify -node node1 -auto-giveback true
cluster1::> storage failover show -node node1 -fields auto-giveback
node   auto-giveback
------ -------------
node1  true

Automatic giveback defaults and panic-specific behavior vary by condition and version. ONTAP also has -auto-giveback-after-panic; inspect storage failover modify ?. During automated nondisruptive upgrades, the upgrade orchestrator controls takeover/giveback and can ignore a manually preset auto-giveback preference.

Unplanned takeover and root-cause analysis

If the partner loses power or panics, the survivor can take over automatically when failover is enabled and safe. First stabilize service; do not repeatedly boot, halt or give back a flapping node.

cluster1::> storage failover show
Node    Partner Takeover Possible State Description
------- ------- ----------------- ----------------------------------
node1   node2   false             In takeover, partner stopped

cluster1::> system node show -fields health,eligibility,uptime
cluster1::> event log show -time >2h -node node1,node2

Operators often search for storage failover show-history, but the ONTAP 9.19.1 SFO family does not document that command. Reconstruct the timeline from EMS (event log show), storage failover show -instance, takeover/giveback progress, SP/BMC event logs, panic/core records, and AutoSupport. Preserve timestamps and timezone before clearing anything.

cluster1::> storage failover hwassist stats show
cluster1::> storage failover progress-table show
cluster1::> system node coredump show
cluster1::> system node autosupport invoke -node * -type all -message "HA-RCA"

Maintenance mode and upgrades

For hardware procedures, take over normally, verify the impaired node is at LOADER, then boot maintenance mode only when the platform procedure requires it:

LOADER-A> boot_ontap maintenance

*> disk show -a
*> ha-config show
*> halt

Maintenance mode bypasses normal clustered safeguards and has a different command set. Never use it as a generic fix. Follow the exact model replacement guide and exit with halt before a normal boot_ontap. For software, prefer automated nondisruptive upgrade and its validation; see the upgrade runbook.

Performance impact and when not to take over

The survivor now runs both controllers’ workloads with one controller’s CPU, cache, ports and paths. Expect reduced headroom and possibly higher latency, especially if it was already busy. LIF migration and SAN ALUA/ANA path changes can add a brief client pause; validate multipathing with the SAN guide.

Do not force takeover when NVRAM logs are unsynchronized, both nodes or shared shelves have unstable power, disk ownership/path state is unknown, the survivor lacks capacity, versions are incompatible, encryption keys are unavailable, or ONTAP says data may be at risk. Escalate with AutoSupport. An HA command cannot repair a common-mode failure.

Troubleshooting matrix

Symptom	Evidence	Safe next step
Takeover not possible	`storage failover show -instance` reason	Correct interconnect, NVRAM, version or partner state; rerun check
Giveback hangs	`show-giveback` names aggregate/subsystem	Resolve veto; override only with documented risk acceptance
Partner “waiting for giveback”	Console plus SFO state	Validate keys/disks/ports, then normal giveback
NVRAM mismatch / log unsync	State description and EMS	Stop; collect AutoSupport and contact support
MLAG/vPC network anomaly	LIFs up but clients lose paths	Check switch peer-link, VLAN and port-channel state; do not bounce both nodes
Mismatched partner states	Each console reports different phase	Preserve logs, inspect HA links/mailboxes, avoid force loops
Performance collapses in takeover	Survivor CPU/latency saturated	Pause backup/scan jobs; restore partner only when safe
Auto-giveback does not start	setting, timer, panic-specific field, veto	Check modify fields and EMS; use controlled manual giveback
Aggregate remains with survivor	partial giveback output	Check aggregate/key health and subsystem veto

# Evidence bundle before escalation
storage failover show -instance
storage failover show-takeover
storage failover show-giveback
event log show -time >2h
system health alert show
network interface show -fields home-node,curr-node,status-oper