Reporting Errors in NFSv4.2 via LAYOUTRETURN
Extracted elements (19)
During MDS restart recovery, clients cannot use LAYOUTRETURN to report I/O errors because layout stateids are invalid during the grace period and clients cannot obtain new layouts during grace recovery. This left the MDS with no choice but to assume all files with write intents needed resilvering.
The MDS is permitted to allow client access during resilvering via a proxy server inserted into the layout, which copies a good mirror instance to a new instance while handling in-flight I/O. This avoids locking out clients for the entire resilvering duration while still maintaining mirror consistency.
The resilvering process is defined to require fencing the file first (RFC 8435 Section 2.2) before recording the resilvering need and releasing the write intent. This ordering ensures data consistency and prevents concurrent I/O from corrupting the resilvering operation.
The MDS has no expectation for clients to use the new anonymous-stateid functionality. If the client uses it but the MDS does not support it, the MDS MUST reply with NFS4ERR_BAD_STATEID, and the client should fall back to the old behavior of not reporting errors.
After the grace period, if the client sends an lrf_stateid in LAYOUTRETURN with a value of the anonymous stateid of all zeros, the MDS MUST respond with NFS4ERR_NO_GRACE.
During the grace period, if the client sends an lrf_stateid in LAYOUTRETURN with any value other than the anonymous stateid of all zeros, the MDS MUST respond with NFS4ERR_GRACE.
If the client reclaims the file and reports no errors, the MDS MUST NOT resilver the file. If the client reports an error on the file, then the file MUST be resilvered.
If the MDS allows client access to a file during resilvering, the client MUST have the same layout (set of mirror instances) after the resilvering as before, and any proxy server inserted into the layout MUST stay in the layout until the grace period is finished.
If the MDS detects that the layout being returned in LAYOUTRETURN does not match the current mirror instances found for the file, it MUST ignore the LAYOUTRETURN and resilver the file in question.
The MDS MUST accept the anonymous stateid of all zeros for the lrf_stateid in LAYOUTRETURN, allowing clients to report I/O errors encountered during an MDS outage without needing a valid layout stateid.
The MDS MUST NOT resilver a file if there are clients with outstanding write intents, and MUST also track the need to resilver so that if the MDS restarts during the grace period it can restart file recovery or start resilvering appropriately.
The MDS MUST resilver any files that are neither explicitly recovered with a CLAIM_PREVIOUS nor have a reported error via a LAYOUTRETURN, because the client has most likely restarted and lost any state.
The MDS MUST track outstanding write intents (which occur when a client opens a file and gets LAYOUTIOMODE4_RW), and when it restarts, it MUST track recovery of those write intents.
When the MDS builds a reply to LAYOUTRETURN with an lrf_stateid of the anonymous stateid of all zeros, it MUST NOT bump the seqid of the lorr_stateid.
When the MDS cannot determine whether client modifications were consistent, it MUST NOT assume those modifications were consistent and MUST assume mirrors are inconsistent, picking one for resilvering.
The anonymous stateid of all zeros (defined in Section 8.2.3 of RFC 8881) is repurposed as the lrf_stateid value in LAYOUTRETURN to allow clients to report I/O errors to the MDS during the grace recovery period without a valid layout stateid.
The ff_ioerr4 structure (defined in Section 9.1.1 of RFC 8435) is used within LAYOUTRETURN to carry I/O error information from the client to the MDS, enabling the MDS to identify which mirrors are corrupt and require resilvering.
There are no new security considerations beyond those in RFC 7862 (NFSv4.2). The extension does not introduce new authentication, authorization, or cryptographic requirements.
Write intent lifecycle: a write intent is created when a client opens a file and receives LAYOUTIOMODE4_RW. During MDS recovery, the MDS tracks write intents and transitions each to either 'recovered' (client reclaims with no errors), 'resilver-needed' (client reports error or grace period expires without reclaim), or 'resilvering' (once no outstanding write intents remain on the file).