Internet Small Computer System Interface (iSCSI) Extensions for the Remote Direct Memory Access (RDMA) Specification

iSER does not use SCSI Data-Out PDUs for solicited write data; instead, the target uses RDMA Read to pull data directly from the initiator's advertised buffer. This eliminates the store-and-forward overhead of TCP reassembly that motivates iSER's design.

SendSE and SendInvSE messages are treated as optional optimizations, not requirements. New iSER implementations SHOULD use plain Send because SendSE/SendInvSE are not universally supported across RCaP implementations (e.g., InfiniBand does not mandate them), and using them with a non-supporting peer causes fatal RCaP connection closure.

The iSER layer is intentionally oblivious to the iSCSI session concept and maintains minimal per-connection state, to permit implementation on generic RDMA-Capable Controllers without requiring iSCSI- or iSER-specific hardware assists in the RNIC.

The target drives all solicited data transfers (RDMA Read for SCSI Write, RDMA Write for SCSI Read) rather than the initiator, because RDMA requires the Data Sink to issue Read Requests and the Data Source to initiate Writes. Only the target knows which direction each command requires, and having the target drive all transfers simplifies buffer advertisement.

An iSER-enabled node is not required to initiate RDMAExtensions key exchange if it prefers Traditional iSCSI mode. If both peers fail to negotiate RDMAExtensions='Yes' on the leading connection, some RCaP implementations (e.g., InfiniBand) may require the existing connection to be torn down and a new one established in TCP-capable mode (e.g., IPoIB).

For 'new' initiators (supporting late connection allocation) connecting to 'old' targets (not supporting iSERHelloRequired), the initiator SHOULD either allocate full connection resources before the final Login Request, or pre-allocate receive buffers for unexpected control-type PDUs, to prevent RCaP connection closure from buffer exhaustion before Full Feature Phase resources are ready.

If iSERHelloRequired is negotiated to 'Yes', the first iSER Message sent by the initiator MUST be the iSER Hello Message; if 'No', the initiator MUST NOT send an iSER Hello Message. Receipt of an iSER Hello Message when iSERHelloRequired='No' MUST be treated as an iSER protocol error.

iSER-assisted mode is defined for Normal sessions only; the RDMAExtensions key MUST NOT be negotiated for a Discovery session, which is always conducted using the transport layer as described in iSCSI.

iSER-assisted mode is session-wide: if RDMAExtensions='Yes' was negotiated on the leading connection, all subsequent connections of the same session MUST enable iSER-assisted mode without re-exchanging the RDMAExtensions key.

iSER-assisted mode MUST NOT be enabled unless negotiated on the leading connection during the LoginOperationalNegotiation stage via RDMAExtensions='Yes'. If both peers fail to negotiate this key to 'Yes', the connection MUST continue with standard iSCSI login semantics.

The iSCSI layer at the initiator MUST NOT issue SNACKs for PDUs when operating in iSER-assisted mode.

The iSCSI layer at the initiator MUST set ExpDataSN to zero in Task Management Function Requests for Task Allegiance Reassignment for read/bidirectional commands, so as to cause the target to resend all unacknowledged read data.

The iSCSI layer at the target MUST always return SCSI status in a separate SCSI Response PDU for read commands; there MUST NOT be a 'phase collapse' when concluding a SCSI Read Command in iSER-assisted mode.

The iSCSI layer MUST NOT request iSER to allocate RCaP connection resources until sufficiently far into the Login Phase. If the Login Phase includes a SecurityNegotiation stage, resource allocation MUST be deferred to the LoginOperationalNegotiation stage, after authentication is completed.

security

The iSER layer at the initiator MUST check that STag invalidation has occurred when Send with Invalidate is the expected invalidation mechanism, and MUST explicitly invalidate the STag if it has not yet been invalidated (e.g., because a plain Send was used instead). Bidirectional commands and abnormal task completions require explicit invalidation.

security

The iSER layer at the initiator MUST set IRD to 1 or higher if R2Ts are to be used on the connection, and the iSER-IRD value MUST NOT exceed the IRD value at the initiator. The initiator MAY set IRD to zero to indicate that no R2Ts will be used.

The iSER layer at the target MUST send the final Login Response PDU in native transport mode (byte stream mode for TCP) to conclude the iSCSI Login Phase, and for iWARP connections the sending of the Login Response PDU and the iSER HelloReply MUST be performed atomically.

When RDMAExtensions is negotiated to 'Yes', HeaderDigest and DataDigest MUST be negotiated to 'None' on all iSCSI/iSER connections in that session, because RCaP provides error detection at least as good as a 32-bit CRC for all iSER Messages.

security

iSCSI data-type PDUs are SCSI Data-In and R2T; they cause RDMA operations transparent to the remote iSCSI layer and are not delivered as-is to the remote iSCSI layer. All other PDUs including unsolicited SCSI Data-Out are iSCSI control-type PDUs and are carried via RDMA Send Messages with an iSER header.

iSER-assisted SCSI Read flow: initiator advertises an STag in the SCSI Command PDU; target uses one or more RDMA Writes to place data into the advertised buffer; target sends SCSI Response PDU via Send Message, optionally using Send with Invalidate for automatic STag invalidation.

iSER-assisted SCSI Write flow: initiator advertises an STag in the SCSI Command PDU for solicited data; target uses one or more RDMA Reads to pull write data from the initiator's advertised buffer; unsolicited data may be sent via Send Messages. The target drives all solicited data transfer.

iSER Hello Message is sent by the initiator as the first iSER Message (when iSERHelloRequired='Yes') declaring the iSER-IRD value. The target responds with an iSER HelloReply Message declaring iSER-ORD, which the target sets to min(target-ORD, initiator-declared iSER-IRD). Failure conditions include: iSER-IRD>0 but target-ORD=0, or non-overlapping iSER protocol versions.

Login/Text key iSERHelloRequired (Section 6.10): a boolean key negotiated by the initiator to determine if the target supports iSER Hello exchanges. If 'Yes', the iSER Hello/HelloReply exchange is mandatory before any other Full Feature Phase PDUs.

Login/Text key RDMAExtensions (Section 6.3): a boolean key negotiated to enable iSER-assisted mode. It MUST be offered in the first available Login Request/Response PDU in the LoginOperationalNegotiation stage and is a session-wide leading-connection-only key.

Login/Text keys TargetRecvDataSegmentLength (Section 6.4) and InitiatorRecvDataSegmentLength (Section 6.5) declare the maximum size of an iSER Message (RDMA Send payload) that the target and initiator, respectively, can receive. These replace the single MaxRecvDataSegmentLength semantics for iSER connections.

The iSER layer exposes nine Operational Primitives to the iSCSI layer: Send_Control, Put_Data, Get_Data, Allocate_Connection_Resources, Deallocate_Connection_Resources, Enable_Datamover, Connection_Terminate, Notice_Key_Values, and Deallocate_Task_Resources. Each invocation is qualified with a Connection_Handle.

The STag (Steering Tag) identifies an I/O Buffer unique to an RDMA-Capable Controller. The initiator Advertises the STag and Base Offset in the iSER header of the Send Message containing the SCSI Command PDU, enabling the target to perform RDMA Reads (for SCSI Writes) or RDMA Writes (for SCSI Reads) directly into/from the initiator's buffer.

If an Advertised STag is not invalidated after task completion (e.g., cached for reuse), the I/O Buffer remains exposed for RDMA Read or Write access by the remote peer outside the scope of the original iSCSI operation, risking both unauthorized data access and unexpected memory modification by the network.

security

Premature allocation of RCaP connection resources exposes the iSER target to resource exhaustion attacks via multiple connections that only progress to the resource allocation point. The countermeasure is to require initiator authentication before allocating resources, deferring allocation to the LoginOperationalNegotiation stage when a SecurityNegotiation stage is present.

security

iSER Header for iSCSI Control-Type PDU (Section 9.2, Figure 3): carries flags and opcode fields; for SCSI Command PDUs also carries the STag and Base Offset of the I/O Buffer being Advertised to the target for subsequent RDMA operations.

iSER Header Format (Figure 2, Section 9.1): a fixed-size header prepended to the iSCSI PDU payload in every RDMA Send Message. No length field is included because the RCaP layer provides the message length to the iSER layer upon delivery.

iSER Hello Message header (Section 9.3, Figure 4) and iSER HelloReply header (Section 9.4, Figure 5) carry the iSER protocol version range and RDMA queue depth parameters (iSER-IRD in Hello; iSER-ORD in HelloReply) used for connection-level RDMA resource negotiation.