Export of Delay Performance Metrics in IP Flow Information Export (IPFIX)
Extracted elements (25)
Delay metrics are aggregated within the Flow Record on the OAM transit or decapsulating node, which reduces both export bandwidth and Collector processing requirements compared to exporting each packet as an individual Flow Record with timestamp information.
For long-running Flows, the IPFIX Metering Process may miss the temporal distribution of delay (e.g., elevated delay only at the start). Operators experiencing this as a problem should configure a smaller Flow expiration timeout per RFC 5470 Section 5.1.1.
Two encoding strategies are offered: exporting pathDelayMeanDeltaMicroseconds (mean computed on the exporter) reduces Collector processing, while exporting pathDelaySumDeltaMicroseconds allows the Collector to compute mean as sum/packetDeltaCount, reducing Exporter computation and per-packet caching requirements.
unsigned64 was chosen for pathDelaySumDeltaMicroseconds (rather than unsigned32) to avoid overflow: a flow with 100 ms path delay cannot exceed 42,949 packets without wrapping an unsigned32 counter. Reduced-size encoding per RFC 7011 Section 6.2 may be applied when unsigned32 suffices.
The delay metrics defined here can be computed using IOAM (RFC 9197) via the Edge-to-Edge or Direct Exporting Option-Type, or using the Enhanced Alternate Marking Method per RFC 9947 applied to SRv6 Segment Routing Headers. Implementations must document at which forwarding-plane point the encapsulating timestamp is introduced.
This solution is primarily applicable to postcard mode On-Path Telemetry, in which OAM header transit nodes also export telemetry data. In passport mode (only the decapsulating node reports), per-hop transit delay export is not directly supported.
Implementations SHOULD refer to BCP 195 for additional security guidance when selecting the underlying protocol used to exchange the IPFIX delay information.
IPFIX collectors MUST ensure that IPFIX data originates from trusted sources. Accepting data from unauthenticated sources could lead to data spoofing, policy misapplication, or denial of service.
The maximum one-way delay SHALL be calculated using the conditional distribution of all packets with a finite value of one-way delay; the maximum is the largest FiniteDelay[j] across all finite singleton delays.
The mean one-way delay SHALL be calculated using the conditional distribution of all packets with a finite value of one-way delay, excluding undefined delays, yielding a single value expressed in microseconds.
The minimum one-way delay SHALL be calculated using the conditional distribution of all packets with a finite value of one-way delay, excluding undefined delays.
The sum of one-way delays SHALL be calculated using the conditional distribution of all packets with a finite value of one-way delay; either FiniteDelay or MaxDelay MAY be used per RFC 6049 Section 4.3.5.
The UTC Time Zone is required for the T0 and Tf runtime parameters (start and end of measurement interval), per Section 6.1 of RFC 2330.
Clock synchronization between monitored nodes is required to produce representative delay measurements. NTP (RFC 5905) is cited as a suitable synchronization mechanism. Output values are expressed as decimal64 with fraction digits = 9, providing 0.001 microsecond (1 ns) resolution.
Four new IPFIX Information Elements are defined: pathDelayMeanDeltaMicroseconds (530), pathDelayMinDeltaMicroseconds (531), pathDelayMaxDeltaMicroseconds (532), and pathDelaySumDeltaMicroseconds (533). These expose On-Path delay measured at OAM header transit and decapsulating nodes within an IPFIX Flow Record.
pathDelayMaxDeltaMicroseconds (ElementID 532) reports the highest path delay of all packets in a Flow in microseconds, encoded as unsigned32 with deltaCounter semantics.
pathDelayMeanDeltaMicroseconds (ElementID 530) reports the mean path delay of all packets in a Flow in microseconds, encoded as unsigned32 with deltaCounter semantics, between the OAM header encapsulating node and the local node.
pathDelayMinDeltaMicroseconds (ElementID 531) reports the lowest path delay of all packets in a Flow in microseconds, encoded as unsigned32 with deltaCounter semantics.
pathDelaySumDeltaMicroseconds (ElementID 533) reports the sum of path delays of all packets in a Flow in microseconds, encoded as unsigned64 with deltaCounter semantics. The larger type is required to avoid overflow for high-delay, high-packet-count flows.
The OAM domain roles are: OAM Header Encapsulating Node (inserts timestamp into OAM header), OAM Header Transit Node (measures delay between OAM timestamp and local receive time), and OAM Header Decapsulating Node (computes final delay and strips OAM header).
Four new entries (ElementIDs 530–533) are added to the IANA 'IPFIX Information Elements' registry in the 'IP Flow Information Export (IPFIX) Entities' registry group: pathDelayMeanDeltaMicroseconds, pathDelayMinDeltaMicroseconds, pathDelayMaxDeltaMicroseconds, and pathDelaySumDeltaMicroseconds.
Four new performance metrics (Identifiers 27–30) are registered in the IANA 'Performance Metrics Registry': OWDelay_HybridType1_IP_RFC9951_Seconds_Mean, _Min, _Max, and _Sum. These follow the RFC 8911 template and metric-naming convention.
Exporting delay metrics may expose sensitive business or privacy information: receivers of these metrics gain visibility equivalent to a direct observer along the data path, potentially enabling traffic analysis or targeted attacks.
The underlying protocol exchanging IPFIX delay data must guarantee integrity and confidentiality of the exported information; see the IPFIX security considerations in RFC 7011 Section 11.
pathDelayMeanDeltaMicroseconds, pathDelayMinDeltaMicroseconds, and pathDelayMaxDeltaMicroseconds are each encoded as unsigned32 (4 bytes); pathDelaySumDeltaMicroseconds is encoded as unsigned64 (8 bytes). All use deltaCounter semantics in the IPFIX Template Record.