Flow Selection Techniques
Extracted elements (24)
Flow-state dependent packet selection is included in this document (despite being a packet selection technique, not an IFSP technique) because it aims at reducing resources for Flow capturing and processing, and because its configuration parameters were only briefly discussed in RFC 5475.
Hash-based Flow Filtering is preferred over packet-level hash selection when the sampling probability must be dynamically adjusted based on the number of observed Flows (e.g., in the Metering Process), while still enabling coordinated subset selection across multiple Observation Points using a shared Hash Domain of invariant Flow Key fields.
The IFSP is distinguished from the Intermediate Selection Process because it can base selection decisions on Flow Selection State information and resource variables (Flow Cache occupancy, export link capacity, timeout intervals) rather than only on the values of Flow Record fields.
An IFSP technique must select either all or none of the packets of a Complete Flow; techniques that cannot guarantee this (e.g., flow-state dependent packet selection) must be classified as packet selection techniques under RFC 5475 rather than as IFSP techniques.
The IFSP is always applied after packets are classified into Flows, in contrast to packet selection which may be applied before or after Packet Classification; an IFSP in the Metering Process before Packet Classification may only use Flow Key fields and hash values, not post-classification characteristics such as Flow size or duration.
For the flowSelectorAlgorithm Information Element, configuration parameters MUST be clearly specified; further Information Elements are needed to fully specify selection techniques and all their parameters.
If Hash-based Flow Filtering is used to select the same subset of flows at different Observation Points, the Hash Domain MUST only include parts of the Flow Record content that are invariant on the Flow path.
In order to be compliant with IPFIX, at least one of this document's Flow Filtering schemes MUST be implemented.
Systematic count-based Flow sampling MAY be applied in the Metering Process, for example by selecting every N-th new Flow based on packet arrivals.
The Information Elements specified in Section 8 MUST be exported by an Intermediate Flow Selection Process to support interpretation of measurement results, including total observed and selected flows, packets, and bytes.
Flow data exported by IPFIX Exporting Processes and collected by Collecting Processes can be sensitive for privacy reasons; privacy considerations for collected data are provided in RFC 7011.
Flow Filtering is a deterministic function on IPFIX Flow Record content with two subtypes: Property Match Filtering (selecting flows whose field values match a configured value or interval) and Hash-based Flow Filtering (selecting flows whose hash over the Hash Domain falls within the Hash Selection Range).
Flow Sampling selects flows based on Flow Record sequence or arrival times using systematic (periodic N-th flow or time-based interval) or random (n-out-of-N, uniform probabilistic, or non-uniform probabilistic) functions to obtain a representative subset for estimating population characteristics.
Flow-state Dependent Packet Selection (e.g., Sample and Hold) is a packet selection technique—not an IFSP technique—that selects packets based on current Flow Selection State; it does not guarantee Complete Flow selection and therefore cannot replace an IFSP.
The Flow-state Dependent IFSP selects or drops flows based on current Flow Selection State, including the Frequent algorithm (maintains a cache of size k-1, selects flows exceeding 1/k of observed packet stream) and Lossy Counting (identifies flows whose packet count exceeds a percentage threshold with bounded estimation error e).
The IFSP can be deployed at three locations in the IPFIX architecture: (1) in the Metering Process at the IPFIX Exporter, (2) in the Exporting Process, or (3) within an IPFIX Mediator, as illustrated in Figure 1.
The information model for IFSP reporting requires exporting: flowSelectorAlgorithm (IE 390), selected octet/packet/flow delta counts (IEs 391–393), total flows observed and selected per SelectorID (IEs 394–395), and sampling or hash configuration IEs appropriate to the active technique.
The Intermediate Flow Selection Process (IFSP) is an Intermediate Process that takes Flow Records as input and selects a subset as output, using criteria that may include Flow Selection State, resource constraints (e.g., Flow Cache occupancy, export link capacity), and counters—not just Flow Record field values.
IANA registered eleven new IPFIX Information Elements (IDs 390–400): flowSelectorAlgorithm (390, unsigned16 identifier), flowSelectedOctetDeltaCount (391, unsigned64 octets), flowSelectedPacketDeltaCount (392, unsigned64 packets), flowSelectedFlowDeltaCount (393, unsigned64 flows), selectorIDTotalFlowsObserved (394), selectorIDTotalFlowsSelected (395), samplingFlowInterval (396), samplingFlowSpacing (397), flowSamplingTimeInterval (398, microseconds), flowSamplingTimeSpacing (399, microseconds), and hashFlowDomain (400, unsigned16 identifier).
IANA registered OID decimal 8 (flowSelectorAlgorithm) in the IPFIX-SELECTOR-MIB Functions subregistry per RFC 6615, identifying the IFSP technique applied.
The flowSelectorAlgorithm registry defines nine initial technique identifiers: systematic count-based sampling (1), systematic time-based sampling (2), random n-out-of-N sampling (3), uniform probabilistic sampling (4), property match filtering (5), hash-based filtering using BOB (6), IPSX (7), or CRC (8), and flow-state dependent IFSP (9). New assignments use First Come First Served with Expert Review; proposed algorithms must be reviewed against RFC 2804 wiretapping policy.
An adversary may craft flows disproportionately selected or excluded by hash-based or time-based systematic sampling to circumvent accounting or evade attack detection; hash function security considerations from RFC 5475 Section 6.2.3 apply equally to Hash-based Flow Selection.
For random Flow sampling, an input to a block cipher (e.g., the CBC Initialization Vector) should be used as the random seed to prevent adversaries from predicting selection decisions.
IFSP configuration information must be protected by: (1) encryption against disclosure of selection parameters, (2) data integrity mechanisms (authentication) against modification, and (3) access control lists and source authentication against unauthorized configuration by malicious nodes.