rfc-9766

Extensions for Weak Cache Consistency in NFSv4.2's Flexible File Layout

T. Haynes, T. Myklebust
date2025-04 streamIETF areawit wgnfsv4 statusPROPOSED STANDARD pages11 canonicalhttps://www.rfc-editor.org/rfc/rfc9766 doi10.17487/RFC9766
This document specifies extensions to NFSv4.2 for improving Weak Cache Consistency (WCC). These extensions introduce mechanisms that ensure partial writes performed under a Parallel NFS (pNFS) layout remain coherent and correctly tracked. The solution addresses concurrency and data integrity concerns that may arise when multiple clients write to the same file through separate data servers. By defining additional interactions among clients, metadata servers, and data servers, this specification enhances the reliability of NFSv4 in parallel-access environments and ensures consistency across diverse deployment scenarios.

Extracted elements (20)

design-rationale §3.7

A client may present all mirrors in ff_layout_wcc4 (with an empty attribute mask for unmodified ones) or only mirrors with updated attributes; the triple (ffdsw_deviceid, ffdsw_stateid, ffdsw_fh_vers) unambiguously identifies each data server regardless of how many mirrors are included, making positional-only matching insufficient.

design-rationale §3.3

No bitmap4 of accepted attributes is included in the LAYOUT_WCC result because the attributes are entirely optional and the client has no recourse if the server ignores them; returning a bitmap would add wire overhead with no actionable value for the client.

design-rationale §2

pNFS reintroduces the WCC problem that NFSv4 had solved: because pNFS clients write directly to data servers, the metadata server cannot track file modifications without probing data servers via NFSv3 GETATTR, creating scalability and round-trip overhead. LAYOUT_WCC lets the client push WCC data it already received, eliminating many of those probes.

design-rationale §2

With client-side mirroring the client must aggregate N mirrored data files and can only report WCC data at delegation return under the existing RFC 8435 mechanism; LAYOUT_WCC removes this restriction by allowing the client to report at any time and aggregate N mirror attributes into one MDS round trip.

interoperability-note §1

LAYOUT_WCC is introduced as an NFSv4.2 extension following the RFC 8178 versioning rules; it is built on top of XDR generated from RFC 7863 and depends on types from the nfs4_prot.x file. The XDR snippets must be placed in their respective areas of that file.

interoperability-note §2

The flexible file layout type restricts clients to only three NFSv3 operations on data server filehandles (READ, WRITE, COMMIT), while the metadata server may use CREATE, GETATTR, and SETATTR on the data server to manage metadata; LAYOUT_WCC allows the client to proxy WCC data from these operations back to the MDS.

normative-requirement §3.7 REQUIRED

All three of ffdsw_deviceid, ffdsw_stateid, and ffdsw_fh_vers are required and together uniquely identify which data server attributes to update; positional correspondence alone is insufficient because a layout may have multiple data files on the same storage device.

normative-requirement §3.6 OPTIONAL

LAYOUT_WCC is OPTIONAL for both NFSv4.2 implementations and implementations of the flexible file layout type; neither is required to support or use it.

normative-requirement §3.7 MUST

The flexible file layout payload MUST have a one-to-one positional correspondence between ff_data_server4 and ff_data_server_wcc4, ff_mirror4 and ff_mirror_wcc4, and ff_layout4 and ff_layout_wcc4.

normative-requirement §3.4.2 MUST

The LAYOUT_WCC operation MUST provide all eight mapped attributes to the metadata server: size, space_used, mode, owner (uid), owner_group (gid), time_access (atime), time_modify (mtime), and time_metadata (ctime), as listed in the NFSv3-to-NFSv4.2 attribute mapping table.

normative-requirement §3.3 MAY

The metadata server MAY use LAYOUT_WCC attributes to avoid querying data servers via NFSv3 GETATTR, but MAY also choose to ignore those attributes and still issue GETATTR calls to strengthen the consistency model.

protocol-element §3.5

LAYOUT_WCC errors are a subset of existing NFSv4 errors defined in RFC 8881 Section 15.1; the operation introduces no new error codes. The MDS may ignore the entire operation or apply a partial payload upon error.

protocol-element §3

LAYOUT_WCC is a new NFSv4.2 operation (Operation 77) that allows a pNFS client to periodically report data file attributes to the metadata server, enabling the MDS to update its view of file metadata without issuing costly NFSv3 GETATTR calls to data servers.

protocol-element §3.4.1

Recommended triggers for sending LAYOUT_WCC include: client issuing GETATTR for size, space_used, change, time_access, time_metadata, or time_modify; client sending NFS4ERR_ACCESS via LAYOUTRETURN or LAYOUTERROR; and client sending SETATTR to refresh proxied times per RFC 9754.

protocol-element §3.3

The lowa_type field carries the value from the IANA 'pNFS Layout Types' registry identifying the layout type, and governs interpretation of the opaque lowa_body payload.

registry

protocol-element §3.4.2

The NFSv3-to-NFSv4.2 attribute mapping used in LAYOUT_WCC: size→size, used→space_used, mode→mode, uid→owner, gid→owner_group, atime→time_access, mtime→time_modify, ctime→time_metadata. The uid and gid are stringified into owner and owner_group.

security-consideration §5

There are no new security considerations introduced by this document beyond those already specified in RFC 8435 (Parallel NFS Flexible File Layout).

security

wire-format §3.7

The flexible file layout payload uses three nested XDR structures: ff_data_server_wcc4 (ffdsw_deviceid, ffdsw_stateid, ffdsw_fh_vers, ffdsw_attributes), ff_mirror_wcc4 (array of ff_data_server_wcc4), and ff_layout_wcc4 (array of ff_mirror_wcc4), mirroring the hierarchy of ff_layout4.

wire-format §3.1

The LAYOUT_WCC4args structure encodes three fields: lowa_stateid (stateid4), lowa_type (layouttype4), and lowa_body (variable-length opaque). The stateid and current filehandle identify the layout; lowa_type indicates how to interpret lowa_body; lowa_body carries the layout-type-specific data file attributes.

cbor

wire-format §3.2

The LAYOUT_WCC4res structure contains a single field, lowr_status (nfsstat4), indicating success or an error code. No bitmap4 of accepted attributes is returned because clients have no recourse if the server ignores attributes.