Requirements for Parallel NFS (pNFS) Layout Types
updates
- rfc-5661 — Network File System (NFS) Version 4 Minor Version 1 Protocol
Extracted elements (25)
For the object layout type, security checks occur at layout allocation time rather than per I/O. Capabilities embedded in the layout enforce access control at storage devices. This design allows the client to perform subsequent I/O without per-operation layout lookups, while permission changes and revocations are handled by invalidating the capability version attribute on affected objects.
The block layout type shifts security enforcement from storage devices to the client because SAN disk arrays provide only coarse-grained access control (LUN mapping/masking at host and logical unit granularity), making NFSv4 file-based security inapplicable. RFC 5663 Section 2.1 specifies that in environments where pNFS clients cannot be trusted to enforce such policies, block layout types SHOULD NOT be used.
The SCSI layout type (RFC 8154) extends the block layout type specifically to remedy its lack of a mechanism for fencing individual clients via the control protocol (the SCSI storage protocol). This is an example of a Standards Track control protocol that meets the control communication requirements for an existing layout type, effectively making it a tightly coupled model.
This document was created because some implementers interpreted Sections 12 and 13 of RFC 5661 as applying only to the file layout type, causing confusion about metadata server and data server responsibilities for new layout types. The absence of a separate Standards Track document for the file layout type (unlike block and object layouts) created the ambiguity.
Three control protocol models exist: (1) loose coupling where the storage protocol doubles as the control protocol (block and object layouts); (2) tight coupling with a purpose-built proprietary protocol co-developed with the metadata server (file layout); and (3) a new Standards Track control protocol with documented interoperability. Each model carries different interoperability assumptions about independent development of metadata servers, storage devices, and clients.
For the block layout type, layout revocation fencing can only be at the granularity of host and logical unit. If one of a client's layouts is revoked, it will effectively revoke all of the client's layouts for files on the storage units comprising the logical volume, potentially including layouts for files in other file systems. Clients must be prepared for such broad revocations and reacquire layouts as needed.
For the object layout type, the client MUST authenticate and authorize users by issuing respective OPEN and ACCESS calls to the metadata server before performing I/O on their behalf, similar to NFSv4 data delegations. This is required because the client may be acting for multiple local users.
A pNFS implementation MUST NOT allow the violation of NFSv4.1's access controls: ACLs and file open modes (RFC 5661 Section 12.9). The layout type specification may require the metadata server to update storage devices so they can enforce security; enforcement may be delegated to the storage device (file layout) or to the client (flexible file layout).
Any disagreement between the metadata server and a storage device over attribute values such as modify time, the change attribute, and EOF position MUST be of limited duration with clear resolution means. A LAYOUTCOMMIT requires that changes in attributes resulting from operations on the storage device be reflected in the metadata server by the completion of the operation.
Clients MUST NOT perform I/O to the storage device if they do not hold layouts for the files in question, MUST NOT perform I/O outside the specified ranges in the layout segment, and MUST NOT perform I/O inconsistent with the iomode specified in the layout segments they hold.
Each new Standards Track document for a layout type MUST address how the client, metadata server, and storage devices interact to meet the control protocol requirements. A document that does not impose implementation requirements sufficient to ensure semantic requirements are met is not appropriate for publication as an RFC from the IETF stream.
For the file layout type, the stateid and principal used for I/O MUST have the same effect and be subject to the same validation on a data server as they would if the I/O were performed on the metadata server itself. Storage devices MUST make all required access checks on each READ or WRITE as determined by NFSv4.1.
For the object layout type, when permissions change, a conflicting mandatory byte-range lock is granted, or a layout is revoked and reassigned, the metadata server MUST change the capability version attribute on all objects comprising the file to invalidate any outstanding capabilities before committing to those changes.
Interactions between locking and I/O operations MUST obey existing semantic restrictions. If an I/O operation would be invalid when directed at the metadata server, it must not be allowed when performed on the storage device. For block and SCSI layouts where the storage device cannot reject the I/O, the client is responsible for enforcing this requirement.
New layout type defining documents MUST address (1) the fencing of clients after a layout is revoked and (2) the security implications of the native capabilities of the storage devices with respect to the requirements of the NFSv4.1 security model. Additionally, they must clarify how other NFSv4.1 semantic requirements such as locking are met.
The metadata server MUST allow clients to perform data I/O against it even if it has already granted the client a layout; a layout type might discourage such I/O but cannot forbid it. The metadata server MUST also be able to do storage allocation (create, delete, extend, or truncate files).
The metadata server MUST be able to fence off a client's access to the data file on a storage device. When the metadata server revokes the layout, the client's access MUST be terminated at the storage devices; the client then has an opportunity to reacquire the layout and perform security checks under the newly current access permissions.
The metadata server MUST be able to restrict access to a file on the storage devices when it revokes a layout. Revocation typically occurs when a client fails to respond to a recall or a client's lease expires; it may also enforce locking state or access permission changes that the storage device cannot directly enforce.
The metadata server MUST be able to service the client's I/O requests if the client directs them to the metadata server instead of to the storage device, including retrieving data from constituent storage devices and presenting it to the client. Even after granting a layout, clients MAY still send I/O requests to the metadata server.
A control protocol is the mechanism an implementation of a layout type uses to meet the control communication requirement between the metadata server and storage devices. It need not be a protocol in the traditional sense; in some cases the storage protocol itself serves as the control protocol (loose coupling).
A layout is the information a client uses to access file data on a storage device, including specification of the protocol (layout type) and the identity of the storage devices. The bulk of layout contents are defined in RFC 5661 as nominally opaque, but individual layout types are responsible for specifying the format of the layout data.
A layout stateid is a 128-bit quantity returned by a server that uniquely defines the layout state for a specific layout type and file (RFC 5661 Section 12.5.2). Layout stateids have different handling from other stateid types as described in RFC 5661 Section 12.5.3.
Fencing is the process by which the metadata server prevents storage devices from processing I/O from a specific client to a specific file. Recalling a layout is a graceful callback allowing the client to flush WRITEs before returning; revoking a layout is an immediate invalidation after which neither the metadata server nor storage devices will accept references to that layout.
Layout iomode grants either read-only or read/write I/O access to the client. Layout type specifications must ensure iomode is enforced; for layout types where storage devices cannot enforce it (e.g., block layout), the enforcement responsibility falls solely on the client.
The layout type specification must ensure that only data access consistent with the NFSv4.1 security model is allowed, either directly via per-access checks or indirectly by delegating checks to the client or storage device. The layout type must provide a mechanism to prevent inappropriate access arising from permissions changes between when a layout is granted and when access is performed.