NVRAM/NVMEM deep-dive: how ONTAP makes writes durable

What NVRAM/NVMEM actually is

Physically it's a few gigabytes of very fast, non-volatile storage on the node — historically a battery-backed DRAM card, today persistent memory (NVMEM) on the system board of AFF/FAS platforms. Functionally it is a write log: a record of writes that have been acknowledged to clients but not yet written to disk. It is not a cache in the read sense, and it is not where data permanently lives — data stays there only for the few seconds between acknowledgment and the next consistency point (CP).

• NVRAM: classic battery-backed RAM card (older systems, e.g. FAS2xxx/3xxx of earlier generations).
• NVMEM: the modern persistent-memory implementation on AFF A-series, FAS A-series, and current platforms. Same job, no battery — the media itself is non-volatile.
• Size: typically a few GB per node — enough to hold the write backlog between CPs (which fire every few seconds or when the log fills).

Both serve the identical logical role, so the rest of this page says "NVRAM" and means either.

The write path, start to finish

Follow a single client write (NFS/SMB/block) through the system:

1. Client sends the write. The node's protocol engine accepts the data into memory buffers.
2. Log to NVRAM: the write (data + metadata describing where it will go) is appended to the local NVRAM log.
3. Mirror to partner: the log entry is copied over the HA interconnect to the partner node's NVRAM. This mirror is the key to lossless takeover.
4. Acknowledge: only after the entry is durable in both NVRAMs does ONTAP reply to the client. This is the durability point: once the client gets the ack, the write will survive any single-node failure.
5. Background flush: at the next CP, WAFL writes the data blocks to disk and the log entries are retired (space in NVRAM is freed).

Two properties fall straight out of this design:

• Low latency regardless of disk speed: the client-visible latency is the NVRAM write + the interconnect mirror — microseconds to low milliseconds — not the disk write. Spinning-disk arrays can still deliver fast small-write latency.
• Write coalescing: because the data sits in NVRAM until the CP, ONTAP accumulates many small client writes and flushes them to disk as large sequential groups — the WAFL batching described on the WAFL page.

Why the mirror makes takeover lossless

If node A dies, its data disks are owned by partner B (in an HA pair). B has a copy of every acknowledged write that hasn't hit disk yet — the NVRAM mirror. So B can:

1. Take over A's storage (storage failover / takeover),
2. Replay A's NVRAM log entries into the WAFL filesystems on A's disks,
3. Serve A's volumes as if nothing happened — no acknowledged write is lost, no filesystem inconsistency, no client data loss.

This is the famous "zero data loss on takeover" claim, and it rests entirely on the NVRAM mirror. If the mirror were broken or the log full, ONTAP would (safely) stop acknowledging writes rather than risk acknowledging something it couldn't make durable — you'd see that as latency or protocol errors, and it's a health condition worth treating as an emergency.

Crash and power-loss behavior

Different failure modes, same guarantee — no loss of acknowledged writes:

• Node crash (OS fault): partner takes over, replays the mirrored log, serves the storage. When the failed node reboots, giveback moves storage back.
• Whole-node power loss: NVRAM/NVMEM retains the log (battery or persistent media). On reboot the node replays its own log before serving data.
• Full-site power loss (both nodes): both NVRAMs retain their logs; on restart each node replays its own. Writes that were mirrored but never acknowledged may be lost — but that's fine, the client never got an ack and will retry.
• Unplanned takeover during a long log backlog: replay just takes longer; takeover time depends partly on how much log has to be replayed.

One nuance: a node replays its own NVRAM log on reboot before serving data, and the HA partner replays the mirrored log after takeover. The filesystems themselves need no repair because WAFL metadata is only ever updated atomically at CPs (see the WAFL page). NVRAM replay restores data, not metadata structure.

Health checks and commands

NVRAM health is a "check before anything big" item: before firmware upgrades, before planned takeovers, and whenever you see unexplained write latency.

storage failover show                        # HA pair state + takeover capability
system node hardware nvram show              # NVRAM/NVMEM status per node
system node hardware battery show            # battery status (legacy NVRAM platforms)
system node run -node node1 sysconfig -a     # hardware summary incl. NVRAM (nodeshell)

What healthy looks like:

• storage failover show: partner nodes, Enabled takeover, no "not possible" flags.
• system node hardware nvram show: status OK, no degraded/mirror-loss states.
• No persistent "NVRAM battery low" or "NVRAM not mirrored" EMS events (check with event log show -severity error).

If you ever see NVRAM mirror loss during normal operation, the system will throttle or stop write acknowledgments — that's the safety valve doing its job. It's a support case, not a "wait and see" item.

Takeover/giveback: the operational side

Planned and unplanned takeover both use the same machinery — the surviving node replays the mirrored log and serves both nodes' storage:

storage failover takeover -ofnode node1      # take over node1 (planned: add -bypass-... checks as advised)
storage failover giveback -ofnode node1      # return storage once node1 is healthy
storage failover show-takeover               # what a takeover would require

Operational notes from the field:

• Planned takeover still mirrors: during a planned takeover (maintenance), ONTAP drains the target node's NVRAM log to disk first, so replay on the partner is minimal.
• Giveback is also log-based: the partner sends back any remaining dirty data so the returning node has a consistent view.
• Takeover time is bounded by replay: a node with a huge backlog takes longer to serve after takeover. Keep an eye on NVRAM utilization during heavy write storms (system node run -node node1 nvram stat in advanced/nodeshell shows utilization).
• Mediated takeover (MetroCluster): the mediator votes to break ties in split-brain scenarios — covered in the MetroCluster guide.

Quick reference

Next in the series: copy-on-write and snapshots — how the same root-flip design that protects WAFL also gives you instant snapshots.