Operation of the IP Flow Information Export (IPFIX) Protocol on IPFIX Mediators
Extracted elements (28)
Absolute timestamps (e.g., flowStartSeconds, flowStartMilliseconds) are RECOMMENDED over relative timestamps (e.g., flowStartSysUpTime, flowStartDeltaMicroseconds) because relative timestamps require constant translation across mediation operations and can introduce range problems — for example, flowStartDeltaMicroseconds requires export within 71 minutes.
Template IDs are unique only per Exporter, per Transport Session, and per Observation Domain (per RFC 7011), so there is no guarantee that the same Template Record will have the same Template ID on incoming and outgoing Transport Sessions at a Mediator. This necessitates the Template Mapping mechanism.
Template Records with identical Information Elements but different non-scope field ordering SHOULD be treated as identical by the Mediator, because the IPFIX guiding principle is that the semantic meaning of one Information Element does not depend on the value of any other. The exception is Scope field ordering, which MUST be treated as semantically significant.
The originalExporterIPv4Address and originalExporterIPv6Address Information Elements were introduced because, without them, a downstream Collector in an Exporter/Mediator/Collector chain would wrongly conclude that the IPFIX Mediator directly observed the packets that generated the Flow Records.
All specifications in this document also apply to the PSAMP (Packet Sampling) protocol, since PSAMP is based on the IPFIX protocol specifications. All RFC 7011 specifications apply to IPFIX Mediators unless explicitly specified otherwise in this document.
An IPFIX Mediator that does not modify (Options) Template Records MUST maintain a Template Mapping composed of related received and exported Template Records, including Template Record Information Elements, Template ID, Observation Domain ID, Transport Session information, and metadata scoped to the Template.
If a Template Record is not used anymore in an outgoing Transport Session, it MUST be withdrawn with an IPFIX Template Withdrawal Message on that specific outgoing Transport Session, and its entry MUST be removed from the Template Mapping.
If an incoming or outgoing Transport Session is gracefully shut down or reset, the (Options) Template Records corresponding to that Transport Session MUST be removed from the Template Mapping.
If an IPFIX Mediator receives an IPFIX Message composed of Template Withdrawals and Template Sets and forwards it, the Mediator MUST NOT modify the Set order and MUST NOT modify the IPFIX Message order.
If an IPFIX Mediator receives Options Template Records from multiple Exporters with the same Information Elements in the Scope fields but ordered differently, it MUST consider those Template Records as semantically different, because scope field ordering affects composite scope semantics.
If the IPFIX Mediator uses IPFIX Structured Data (RFC 6313) to export Original Exporter Information, the Collecting Process MUST support RFC 6313.
Intermediate Processes MUST NOT assume that received records appear in flowStartTime, flowEndTime, or observationTime order. A process MAY ignore significantly out-of-order records but SHOULD report that records were dropped.
Mediators that do not generate new Records MUST NOT omit fields from re-exported Records containing unknown Information Elements, and MUST NOT reorder any fields within such Records, as there may be ordering dependencies.
SCTP using the PR-SCTP extension (RFC 3758) MUST be implemented by all compliant IPFIX Mediator implementations. TCP and UDP MAY also be implemented.
The Observation Domain ID of any IPFIX Message containing Flow Records relevant to no particular Observation Domain, or to multiple Observation Domains, MUST have an Observation Domain ID of 0.
When an Intermediate Process exports Data Records containing different Flow Keys from those received from the Original Exporter, and the Original Exporter sent a Flow Keys Options record, the IPFIX Mediator MUST export a Flow Keys Options record defining the new set of Flow Keys.
When an IPFIX Mediator receives Flow Records containing originalExporterIPv4Address or originalExporterIPv6Address, it SHOULD NOT modify those values when composing new Flow Records. Known exceptions include anonymization and an Intermediate Correlation Process rewriting addresses across NAT.
The ignoredDataRecordTotalCount Information Element (ElementId 407, Data Type: unsigned64, Semantics: totalCounter) counts Data Records that the Intermediate Process did not process since (re-)initialization due to resource constraints; it excludes records that were examined but did not contribute to exported records.
The Intermediate Process Reliability Statistics Options Template reports statistics about an Intermediate Process using scope fields observationDomainId and intermediateProcessId, along with ignoredDataRecordTotalCount and timestamps of the first and last ignored records.
The intermediateProcessId Information Element (ElementId 406, Data Type: unsigned32, Semantics: identifier) is an identifier of an Intermediate Process that is unique per IPFIX Device, used primarily as a scope field in Options Templates. Note that process identifiers may be assigned dynamically.
The originalExporterIPv4Address Information Element (ElementId 403, Data Type: ipv4Address) carries the IPv4 address used by the Exporting Process on an Original Exporter as seen by the Collecting Process on an IPFIX Mediator, providing Original Observation Point information to downstream Collectors.
The originalExporterIPv6Address Information Element (ElementId 404, Data Type: ipv6Address) carries the IPv6 address used by the Exporting Process on an Original Exporter as seen by the Collecting Process on an IPFIX Mediator.
The originalObservationDomainId Information Element (ElementId 405, Data Type: unsigned32, Semantics: identifier) carries the Observation Domain ID reported by the Exporting Process on an Original Exporter; when cascading through multiple Mediators, this identifies the initial Observation Domain in the cascade.
The Template Mapping is a data structure maintained by an IPFIX Mediator that maps received (Options) Template Records to exported (Options) Template Records, scoped by Transport Session and Observation Domain ID on both incoming and outgoing sides, including metadata that affects Template semantics such as flowKeyIndicator.
This document adds five new Information Elements to the IANA IPFIX Information Element registry: originalExporterIPv4Address (403), originalExporterIPv6Address (404), originalObservationDomainId (405), intermediateProcessId (406), and ignoredDataRecordTotalCount (407).
IPFIX Mediators are 'men in the middle' by design, intercepting communication between Original Exporters and downstream Collectors, which breaks end-to-end confidentiality and data integrity guarantees. Trust placed in Collecting Processes is absolute, so care must be taken when exporting IPFIX Messages between processes controlled by different entities.
The exporterCertificate and collectorCertificate Information Elements cannot provide true end-to-end assertions about a chain of IPFIX Mediators because any Mediator in the chain can falsify information about upstream Transport Sessions. Chain-of-mediation integrity must be determined out of band. An Exporting Process has no in-band way to determine whether a given Collecting Process will act as a Mediator.
The IPFIX Message Header exported by an IPFIX Mediator contains Version (16-bit), Length (16-bit), Export Time (32-bit, seconds since UNIX epoch), Sequence Number (32-bit, modulo 2^32), and Observation Domain ID (32-bit). The format is identical to RFC 7011 but Export Time and Observation Domain ID may be handled differently at Mediators.