NetApp SolidFire & Element OS: Scale-Out All-Flash Block Storage

SolidFire is NetApp's scale-out all-flash block storage platform, built around per-volume guaranteed QoS and a fully distributed data path. This guide covers the Element OS architecture (nodes, slice services, replicas, fault domains), the QoS model that made SolidFire famous, volumes/accounts/access groups, snapshots and replication, vVols and NetApp HCI, plus a real Element CLI/API operations runbook and a SolidFire-vs-ONTAP decision guide.

What SolidFire is

NetApp acquired SolidFire in December 2015, and the product line has kept its own operating system (Element OS) and management plane ever since — it was never folded into ONTAP. SolidFire is a block-only, all-flash, scale-out storage platform: every volume is striped across every node in the cluster, all nodes are equal, and each volume carries a hard performance guarantee (minimum IOPS) that the cluster scheduler enforces regardless of what its neighbors are doing.

That guarantee is the product's identity. Where ONTAP QoS caps or floors workloads on a shared AFF pair, SolidFire was designed from day one as multi-tenant storage: service providers and large virtualization estates size the cluster once, then carve volumes with contractual IOPS numbers. The other design pillar is predictable scale-out — adding a node adds capacity and IOPS to every volume in the cluster, because all volumes already span all nodes.

If you administer NetApp estates you will most often meet SolidFire in three places: as dedicated all-flash block for VDI or high-density virtualization, as the storage layer of a NetApp HCI rack, or as the backend behind a managed-service provider's multi-tenant offering. See also the ONTAP vs Pure decision guide and the hardware lineup for where SolidFire sits in NetApp's portfolio.

Scale-out architecture: nodes, slice services, replicas

Element's data path is the opposite of the active/passive HA-pair model in ONTAP HA pairs. A SolidFire cluster is a set of equal nodes (from 4 or so up to triple digits) and every volume lives on all of them:

Thin provisioning is the only provisioning model: a volume consumes space only as it is written, cluster-wide capacity is shared, and snapshots are space-efficient point-in-time views. This is why capacity math on SolidFire is about the cluster (used vs over-provisioned) rather than per-volume aggregates — see capacity planning for the ONTAP-side comparison.

Volumes, accounts and access groups

Element's access model is deliberately multi-tenant:

Access-group hygiene is the day-one discipline: one VAG per host cluster (e.g. one per vSphere cluster), all ESXi hosts' IQNs in it, and only the volumes that cluster should see. See the iSCSI on ONTAP guide for the ONTAP-side equivalent, and SAN multipathing for pathing rules that apply to both.

The QoS model: min, max, burst

Every SolidFire volume carries three QoS numbers, and this triple is the core of the product:

SettingMeaningPlanning rule
minIOPSGuaranteed floor. The cluster scheduler reserves this across all nodes; under contention, volumes below their min are served before anyone's burst or max.The sum of all minIOPS must fit inside the cluster's sustainable IOPS envelope. This is the hard capacity number.
maxIOPSCeiling the volume can sustain indefinitely.Sum of maxIOPS may oversubscribe the cluster — contention just means not everyone hits max at once.
burstIOPSShort-term ceiling used while burst credits last.Credits accumulate while the volume runs below its max and are spent during spikes (boot storms, backup windows).

Two practical consequences. First, the guarantee is enforceable only if you size honestly: if the sum of minIOPS exceeds what the cluster can deliver, the scheduler cannot honor every floor and overall latency rises — Element reports this condition, and the fix is more nodes or lower floors, not tuning. Second, QoS is per-volume, not per-application or per-pool, which is what makes multi-tenant hosting viable: tenant A's boot storm draws down its own burst credits and can never starve tenant B's guaranteed floor.

Element 11.3 and later also support named QoS policies (a reusable min/max/burst triple assigned to many volumes), which keeps service tiers consistent instead of hand-typing numbers per volume. Compare this with ONTAP's QoS policies and adaptive QoS, which are per-workload ceilings/floors on a shared AFF pair — the same idea, a different guarantee model.

Snapshots, clones and replication

Replication and snapshot retention are independent: you can replicate snapshots or only the live volume. For a DR architecture comparison with SnapMirror, MetroCluster and SVM DR, see the DR strategy guide.

vSphere, VVols and NetApp HCI

VVols. SolidFire ships a VASA provider, so vSphere can bind each virtual disk to its own Element volume (storage container = a pool of Element volumes). VVols give per-VM snapshots, clones and QoS through vCenter — the natural fit for the platform, and the reason SolidFire deployments are overwhelmingly VMware shops. Classic (non-VVol) datastores over iSCSI remain fully supported for simpler estates.

NetApp HCI. NetApp HCI combined SolidFire storage nodes with Cisco UCS-based compute nodes in one validated rack: compute and storage scale independently (add compute sleds or storage nodes as needed), and everything is deployed and managed through NetApp Hybrid Cloud Control (the management plane that replaced the original NetApp Deployment Engine), with the Element UI for storage. The compute nodes boot from the SolidFire cluster over iSCSI. If you run across retired HCI gear, see the HCI node repurposing guide — homelabbers have been reflashing those exact nodes for years.

For the ONTAP/Tools side of vSphere management (ONTAP Tools, SnapCenter), see ONTAP Tools for VMware; for the general vSphere-on-NetApp design, see vSphere on NetApp.

Management: UI, SSH CLI and the JSON-RPC API

Element is managed three ways, all backed by the same JSON-RPC API:

Basic inventory and capacity:

admin@sf-cluster:~> GetClusterInfo
admin@sf-cluster:~> GetClusterCapacity
admin@sf-cluster:~> ListActiveNodes
admin@sf-cluster:~> ListServices

Tenant and volume lifecycle (SSH CLI syntax):

admin@sf-cluster:~> CreateAccount username=tenant-a \
    initiatorSecret=CHAP-secret-1 targetSecret=CHAP-secret-2
admin@sf-cluster:~> CreateVolume name=db01 accountID=7 \
    totalSize=10737418240 enable512e=true \
    minIOPS=1000 maxIOPS=5000 burstIOPS=10000
admin@sf-cluster:~> ModifyQos volumeID=12 minIOPS=1500 maxIOPS=8000 burstIOPS=16000
admin@sf-cluster:~> ListVolumes
admin@sf-cluster:~> RemoveVolume volumeID=12

The same call over JSON-RPC:

curl -ksu admin:<password> -H 'Content-Type: application/json' \
  https://192.0.2.10/json-rpc/2.0 \
  -d '{"method":"CreateVolume","params":{
        "name":"db01","accountID":7,"totalSize":10737418240,
        "enable512e":true,
        "qos":{"minIOPS":1000,"maxIOPS":5000,"burstIOPS":10000}},
       "id":1}'

Initiator access:

admin@sf-cluster:~> ListInitiators
admin@sf-cluster:~> CreateVolumeAccessGroup name=esxi-cluster-a \
    initiators=["iqn.1998-01.com.vmware:esxi01","iqn.1998-01.com.vmware:esxi02"]
admin@sf-cluster:~> AddVolumesToVolumeAccessGroup volumeAccessGroupID=3 volumes=[12,13,14]

Snapshots, clones and replication:

admin@sf-cluster:~> CreateSnapshot volumeID=12 name=db01-hourly
admin@sf-cluster:~> CreateGroupSnapshot volumes=[12,13] name=db01-grp
admin@sf-cluster:~> CloneVolume volumeID=12 name=db01-clone snapshotID=455
admin@sf-cluster:~> ListSnapshots
admin@sf-cluster:~> StartClusterPairing
admin@sf-cluster:~> ListVolumePairings

Performance and session checks:

admin@sf-cluster:~> ListVolumeStats
admin@sf-cluster:~> ListISCSISessions
admin@sf-cluster:~> ListEvents

Method names are stable across recent Element releases, but always confirm against the API reference for your exact Element version before scripting (the API docs ship with the Element UI, and method names are versioned).

Day-2 operations runbook

Health and capacity checks

admin@sf-cluster:~> ListActiveNodes        # all nodes Active, no FailedServices
admin@sf-cluster:~> ListEvents             # errors/warnings since last review
admin@sf-cluster:~> GetClusterCapacity     # used vs overProvisioned vs maxProvisioned

Watch the over-provisioned ratio: thin volumes plus long snapshot retention can push logical usage far past physical flash. Element reports cluster utilization honestly — when used space approaches the physical ceiling, writes slow cluster-wide before they fail, so trend it monthly (see capacity planning).

Node maintenance and replacement

Because every volume spans every node, you do not "move volumes off" a node the way you drain an ONTAP aggregate. Removing a node triggers automatic slice-service migration; Element completes migration before the node is released. The safe sequence is: confirm the cluster has enough spare capacity and replicas, remove the node through the UI/API (which migrates its slice services), verify ListActiveNodes and ListServices are healthy, then power the node down. Replacing a failed node uses the same path — the cluster re-replicates from its remaining copies and rebalances.

Performance triage

  1. ListVolumeStats — find which volumes are hot: sustained IOPS at max, or latency climbing.
  2. Check QoS posture: if many volumes sit at max while others miss their min, the cluster is oversubscribed — raise minIOPS floors only within the cluster's real envelope, or add nodes.
  3. ListISCSISessions — confirm every host has the expected session count and paths (multipathing rules: see SAN multipathing); a missing path shows up as one-sided latency.
  4. Network: jumbo frames on the storage VLAN are the standard expectation; MTU mismatch is the classic "everything is slow on one host" cause.

Common gotchas

SolidFire vs ONTAP AFF: which one when

DimensionSolidFire / Element OSONTAP AFF / ASA
ProtocolsBlock only (iSCSI, FC)Unified: NFS, SMB, iSCSI, FC, NVMe-oF, S3 object
Scale modelScale-out: add nodes, every volume spans all nodesScale-up HA pairs, then clusters; volumes live on aggregates
QoSPer-volume guaranteed minIOPS enforced cluster-wideQoS policies/adaptive QoS — ceilings and floors on shared pairs
SnapshotsVolume + group snapshots, clonesSnapshots, FlexClone, SnapMirror/SnapVault ecosystem
EcosystemElement API/SDKs, VVols, CSI, OpenStack CinderSnapCenter, SnapMirror, FabricPool, BlueXP, ONTAP tools
Best fitMulti-tenant hosting, VDI at scale, guaranteed-IOPS SLAsMixed file+block estates, NAS, DR fabrics, cyber-resilience

In a NetApp-heavy estate the two are complements, not rivals: ASA gives you the ONTAP feature set in a block-focused form, SolidFire gives you enforceable per-tenant IOPS. If the requirement is "a number we can put in a contract," SolidFire's model is the stronger answer; if the requirement is "one platform for NFS, SMB, FC and S3," ONTAP wins. The on-prem vs cloud guide covers the wider platform decision.

Lifecycle and further reading

NetApp has announced lifecycle transitions for NetApp HCI hardware (end-of-sale/end-of-support milestones; exact dates change, so confirm on NetApp's official lifecycle pages before planning around it). SolidFire SF-series systems and Element OS remain supported NetApp products, with Element releases continuing to ship. As with any NetApp platform, validate exact firmware/Element/plugin versions in the NetApp Interoperability Matrix before buying or upgrading.

Related guides: iSCSI on ONTAP · SAN multipathing · ONTAP QoS deep dive · vSphere on NetApp · ONTAP vs Pure · Repurposing NetApp HCI nodes.

Part of the NetApp Cloud & Hybrid Storage Hub · Related: data protection · performance