The Use of RSA/SHA-1 Signatures within Encapsulating Security Payload (ESP) and Authentication Header (AH)
Extracted elements (25)
HMAC transforms cannot provide true data origin authentication in group (multicast) settings because all group members share the single HMAC key, meaning any member can forge another member's packets. RSA digital signatures provide asymmetric authentication so only the holder of the private key can sign, enabling per-sender verification even in groups.
Key lifetime recommendations depend on modulus size: 768-bit moduli should not be used beyond 1 week, and 1024-bit moduli should not be used beyond 1 year, based on contemporary estimates of attacker cryptanalysis cost (processing, memory, and cryptographic analysis methods).
RSA was chosen because signature verification is relatively efficient compared to other public key algorithms, which benefits multicast receivers that may have limited processing capabilities, and because RSA is commonly supported in hardware.
The authentication tag added by an RSA signature ranges from 62 to 256 bytes, making the scheme best suited for senders with substantial processing power and for traffic small enough that the large tag does not cause packet fragmentation.
RSA signatures cannot be used with an ESP Combined Mode algorithm that includes an explicit ICV (such as AES-GCM), because the Combined Mode algorithm adds its own ICV field, leaving no room for a separate authentication algorithm to supply the ICV.
A conforming implementation MUST support an RSA modulus size of 1024 bits.
Key management mechanisms negotiating RSA signatures MUST also include the encoding method (RSASSA-PKCS1-v1_5 or RSASSA-PSS) during policy negotiation using the Signature Encoding Algorithm SA Attribute.
Key management mechanisms negotiating RSA signatures MUST include the RSA modulus length during policy negotiation using the Authentication Key Length SA Attribute, giving constrained devices the opportunity to decline.
New values for the Signature Encoding Algorithm SA Attribute in the range 3–61439 MUST be added only through a Standards Action as defined in RFC 2434.
RSASSA-PKCS1-v1_5 MUST be supported by a conforming implementation. RSASSA-PSS is generally believed to be more secure and SHOULD be used whenever it is available.
SHA-1 MUST be used as the signature hash algorithm used by the RSA digital signature algorithm within ESP and AH.
The use of an ephemeral RSA key pair with a lifetime equal to that of the ESP or AH Security Association is RECOMMENDED, limiting private key exposure and obviating the need for revocation or validity period transmission.
Use of the RSA signature transform to obtain data origin authentication for pairwise SAs is NOT RECOMMENDED; HMAC transforms are sufficient for pairwise cases and carry lower performance overhead.
When used with a group key management system, the RSA public key SHOULD be sent as part of the key download policy. If the group has multiple senders, the public key of each sender SHOULD be sent.
Signature generation computes a SHA-1 hash of the authenticated portion of the AH or ESP packet (message M), signs the hash using the sender's RSA private key (K), and encodes the result using RSASSA-PSS or RSASSA-PKCS1-v1_5. The output S becomes the ICV field.
Signature verification recomputes the SHA-1 hash of the authenticated packet bytes, decrypts the ICV using the sender's RSA public key (n, e), validates encoding, and compares the two hashes; identical hashes indicate a valid signature.
RFC 4359 requests an assigned number in the ISAKMP 'IPSec Authentication Algorithm' registry with mnemonic 'SIG-RSA', and an assigned number in the 'IPSEC AH Transform Identifiers' registry with mnemonic 'AH_RSA'.
RFC 4359 requests two new ISAKMP SA Attributes: 'Authentication Key Length' (Variable type, carries RSA modulus size) and 'Signature Encoding Algorithm' (Basic type, values: 0=Reserved, 1=RSASSA-PKCS1-v1_5, 2=RSASSA-PSS; 3–61439 reserved via Standards Action; 61440–65535 for private use).
Man-in-the-middle attacks are defeated as long as the receiver obtains the sender's RSA public key via a trusted mechanism. An attacker cannot forge signatures without the RSA private key; the modulus size must be chosen so that the private key cannot be reconstructed before it is retired.
Non-repudiation is explicitly not claimed as a property of this transform. IP packets are ephemeral and not directly correlated to any application, so non-repudiation is inapplicable at the network level.
RSA signature verification is computationally expensive; an attacker with a valid SA and sequence number can force receivers to perform expensive signature verifications, potentially causing a denial-of-service attack. In multicast, all group members are attacked simultaneously.
SHA-1 was scheduled for phase-out in 2010 due to advances in computing power. Use of this transform after that date SHOULD be preceded by an analysis of its continued suitability, given known cryptanalytic progress against SHA-1.
The DoS attack can be mitigated by nesting RSA-authenticated ESP/AH inside an HMAC-authenticated ESP/AH; the HMAC is checked first and attackers without the HMAC key cannot force RSA verification. However, a legitimate group member with the HMAC key can still mount the attack.
For AH, the ICV equals the RSA modulus size. If the modulus is not a multiple of 32 bits (IPv4) or 64 bits (IPv6), explicit ICV Padding bits are required per AH Section 3.3.3.2.1 to produce a suitably sized ICV.
For ESP, the Integrity Check Value (ICV) is equal in size to the RSA modulus. If the modulus is not a multiple of 8 bits, the ICV MUST be prepended with 1–7 zero bits to make it a multiple of 8 bits; no implicit ESP ICV Padding is required.