Direct Data Placement Protocol (DDP) / Remote Direct Memory Access Protocol (RDMAP) Security
updated by
- rfc-7146 — Securing Block Storage Protocols over IP: RFC 3723 Requirements Update for IPsec v3
Extracted elements (30)
IPsec is specified as mandatory-to-implement rather than DTLS because the two initial RDDP adopters, iSCSI and NFSv4, are both compatible with IPsec, and RFC 3723 already mandates IPsec for iSCSI; a common IPsec profile simplifies joint deployments.
STag values should be allocated randomly and reused as infrequently as possible so that a Remote Peer cannot predict the next STag number. Monotonically increasing or fixed STag allocation strategies enable both intentional attacks and accidental off-by-one reuse.
The RDDP IPsec requirements are intentionally based on the older RFC 2401 profile (as profiled by RFC 3723) rather than RFC 4301, to allow a common IPsec profile with iSCSI. If RFC 3723 is updated to RFC 4301, that update should apply uniformly to both iSCSI and RDDP.
TLS is inappropriate for DDP/RDMAP security for two independent reasons: below RDMAP, its in-order stream-cipher decryption requirement eliminates DDP's zero-copy flow-through architecture; above RDMAP, it leaves DDP/RDMAP headers exposed to man-in-the-middle header modification.
A ULP with multiple Remote Peers that do not share Partial Mutual Trust MUST NOT grant write access to the same buffer through different STags; a buffer should be exposed to only one untrusted Remote Peer at a time.
All Non-Privileged ULP interactions with the RNIC Engine that could affect other ULPs MUST be done using the Privileged Resource Manager as a proxy. All ULP resource allocation requests for scarce resources MUST also be done using a Privileged Resource Manager.
An RNIC implementation MUST ensure that a Remote Peer is not able to access memory outside of the buffer specified when the STag was enabled for remote access, applying byte-granularity base-and-bounds checks on every incoming DDP Tagged Message.
An RNIC implementation MUST follow the IPsec requirements defined in RFC 3723, Sections 2.3 and 5, including their normative references; this makes IPsec ESP mode support normative.
An RNIC MUST ensure that a specific Stream in a specific Protection Domain cannot access an STag in a different Protection Domain.
An RNIC MUST ensure that, if an STag is limited in scope to a single Stream, no other Stream can use the STag.
An RNIC MUST ensure that network packets using the STag for a previously Advertised Buffer can no longer modify the buffer after the ULP revokes remote access rights for the specific STag.
For implementations that do not share STags between Streams, each Stream MUST either be in a separate Protection Domain or the scope of an STag MUST be limited to a single Stream, preventing cross-stream STag spoofing attacks.
If a ULP only intends a buffer to be exposed for remote write access, it MUST set the access rights to the buffer to only enable remote write access, preventing unintended RDMA Read disclosure. This requirement does not restrict zero-length RDMA Reads.
If an RDMAP/DDP Stream may be subject to impersonation or Stream hijacking attacks, it is RECOMMENDED that the Stream be authenticated, integrity protected, and protected from replay attacks; confidentiality MAY also be used when the Stream traverses a public network.
If an RNIC Engine provides the ability to share receive buffers across multiple Streams, the combination of the RNIC Engine and the Privileged Resource Manager MUST be able to detect if the Remote Peer is attempting to consume more than its fair share of resources so countermeasures can be applied.
It is not RECOMMENDED that TLS be layered on top of RDMAP or DDP: TLS beneath RDMAP destroys flow-through zero-copy because out-of-order packets cannot be decrypted until all preceding packets arrive; TLS above RDMAP leaves RDMAP/DDP headers unprotected.
Receipt of an IKE Phase 2 delete message MUST NOT be interpreted as a reason for tearing down a DDP/RDMA Stream; the Stream should remain up, and a new IKE Phase 2 SA established when additional traffic is sent, to avoid continual Stream churn.
The allocation of all scarce RNIC resources (Protection Domains, Stream Context Memory, Translation and Protection Tables, STag namespace) MUST be placed under the control of a Privileged Resource Manager to prevent DoS attacks.
The local ULP SHOULD revoke remote access rights for an STag before validating or using the buffer contents, to prevent a Remote Peer from modifying the buffer after indicating that the transfer has completed.
The Privileged Resource Manager MUST NOT assume that different Streams share Partial Mutual Trust unless there is a mechanism to ensure that the Streams do indeed share Partial Mutual Trust, such as explicit notification from the ULP that owns the Streams.
A Protection Domain (PD) is a local construct that associates Stream Context Memory, STags, and Data Buffers; the RNIC enforces that resources belonging to one PD cannot be accessed by resources in a different PD. Each untrusted Stream should have its own unique PD.
Partial Mutual Trust is a defined collection of RDMAP/DDP Streams whose endpoints agree not to perform malicious attacks against each other; any Stream outside the collection is treated as untrusted. ULPs have explicit control over collection membership.
The Asynchronous Event Queue carries RNIC-to-Privileged-Resource-Manager notifications about protocol violations, Stream state changes, local errors, and low-watermark events; it is a bounded resource that can be exhausted by a Remote or Local Peer generating excessive events.
The RDMA architectural model defines four components: RNIC Engine (protocol implementation), Privileged Resource Manager (resource allocation and control), Privileged ULP (trusted application layer), and Non-Privileged ULP (untrusted application layer), interacting through three distinct interfaces with different trust levels.
The STag has a 32-bit namespace with configurable scope: Protection Domain scope (valid on any Stream within the PD) or Single Stream scope (valid on exactly one Stream). The scope is a primary mechanism for controlling which remote peers may access an Advertised buffer.
A local Non-Privileged ULP may attack the Page Translation Table and STag mapping to gain access to memory belonging to another ULP. All PTT initialization by Non-Privileged ULPs must be proxied through the Privileged Resource Manager, which validates that the ULP has authority over the memory being exposed.
A man-in-the-middle attacker who can intercept and modify packets may perform RDMA Read/Write operations or invalidate STags on exposed Data Buffers. End-to-end integrity protection and authentication (e.g., IPsec) is required; physical network isolation is an alternative for constrained deployments.
A network-based attacker can impersonate a legal RDMAP/DDP peer by spoofing an IP address, requiring it to also guess valid STag and TO values for Tagged Messages; an invalid STag causes Stream teardown. End-to-end authentication such as IPsec is the primary countermeasure.
A Remote Peer can flood a shared Completion Queue with minimum-length DDP/RDMAP messages, causing CQ overflow or starvation for other Streams. A Privileged Resource Manager must control CQ entry allocation and the CQ must be sized relative to all queues it serves.
A Remote Peer can mount a buffer overrun attack by issuing an RDMA Write or Read Response targeting memory addresses outside the valid STag buffer bounds. The RNIC must perform byte-granularity base-and-bounds checks before allowing any STag-based access.