Internet Application Protocol Collation Registry
Extracted elements (25)
Rather than attempting to solve the large and not fully explored problem space of searching and sorting international strings, this specification creates an abstraction framework so that application protocols can precisely identify a comparison function, and the repertoire of comparison functions can be extended in the future.
Some behavior of the initial collations (e.g., i;ascii-casemap accepting non-ASCII input) is not ideal, but compatibility with widely deployed code was judged more important than fixing the collations.
The expert reviewer is encouraged to reject 'common' use registrations if the collation would be better listed as 'limited', as it is desirable to keep the number of 'common' registrations small and of high quality to avoid an explosion of collations that would harm interoperability.
ACAP and Sieve used collations prior to this specification and do not meet all application protocol requirements in Section 5. Both allow i;octet working directly on UTF-8 data. In Sieve, 'undefined' and 'no-match' results are treated as false; in IMAP, the default collation is i;ascii-casemap.
A collation specification MUST state which of the three basic operations are supported (equality, substring, ordering) and how to perform each on any two input character strings including empty strings. Collations must be deterministic.
A protocol SHOULD NOT expose sort keys to clients, since some collations leave the sort key format up to the implementation, and current conformant implementations are known to use different formats.
Any system that automatically parses collation registration XML documents MUST account for the possibility that future revisions may add additional function types, XML attributes, values, and elements, to preserve future compatibility.
Clients and servers MUST NOT use i;octet with protocols not based on octet-strings. If the protocol permits use of collations with non-string data structures, the protocol MUST describe the default behavior for such cases.
If positional information is returned for substring match results, it SHOULD fully specify the substring(s) in the result that matches, independent of the length of the search string, such as both starting and ending offsets. Returning just the starting offset is not acceptable.
The collation identifier MUST NOT be longer than 254 characters.
The protocol MUST provide a mechanism for the client to select the collation to use with equality matching, substring matching, and ordering operations.
The protocol MUST specify whether it allows wildcards in collation identifiers. If wildcards are allowed, the protocol MUST specify how comparisons behave in the absence of explicit collation negotiation or when 'default' is requested.
A collation is a named function that takes two arbitrary-length strings as input and can perform one or more of three basic comparison operations: equality test, substring match, and ordering test.
Collation registration uses a well-formed XML document with mandatory elements: <identifier>, <title>, <operations>, <specification>, and <owner>. The <collation> root element MUST include 'scope=' (global/local/other) and 'intendedUse=' (common/limited/vendor/deprecated) attributes.
Collations can be referenced as URIs in the IANA URL space. The collation-uri form is 'http://www.iana.org/assignments/collation/{collation-id}.xml' for specific named collations, and collation-auri is an abstract name for ordering or pattern references.
The collation identifier is a single US-ASCII string that MUST NOT exceed 254 characters, formed by a collation-prefix, a semicolon, a collation-core-name, and optional collation arguments. The ABNF grammar specifies allowed characters as ALPHA, DIGIT, '-', ';', '=', and '.'.
The equality test MUST be reflexive and symmetric, and for valid input MUST be transitive. It returns 'match', 'no-match', or 'undefined'. If a collation provides substring or ordering, it MUST also provide equality, and those operations MUST be consistent with equality.
The i;ascii-casemap collation operates on octet strings and treats US-ASCII letters (octets 97-122 mapped to 65-90) case-insensitively. It provides equality, substring, and ordering. All input is valid; letters outside ASCII are not treated case-insensitively.
The i;ascii-numeric collation operates on arbitrarily-sized unsigned decimal integers stored as octet strings. Input is truncated at the first non-digit; strings not starting with a digit represent positive infinity. It supports equality and ordering only, not substring.
The i;octet collation operates on binary octet strings using unsigned byte comparison (equivalent to C's memcmp). It never returns 'undefined', supports equality, substring, and ordering, and MUST NOT be used unless explicitly allowed by the protocol.
The ordering operation MUST be reflexive, and for valid input MUST be transitive and trichotomous, returning 'less', 'equal', 'greater', or 'undefined'. When used with a '-' prefix, the result MUST be reversed.
The substring operation determines if the first string is a substring of the second using the collation's equality operation. Application protocols MAY return position information; if so, it SHOULD include both starting and ending offsets for each match.
Wildcard selection of collations uses '*' characters that match zero or more collation-chars. Wildcard characters MUST NOT be adjacent. The collation-order form uses an optional '+' or '-' prefix to specify ordering direction, where '-' inverts the ordering result.
This document creates the IANA Collation Registry, stored at http://www.iana.org/assignments/collation/, with each registration as an XML document. The registry is organized into four sections: common use, limited use, vendor-specific, and deprecated collations.
Collations will normally be used with UTF-8 strings, so the security considerations for UTF-8 (RFC 3629), stringprep (RFC 3454), and Unicode TR-36 apply and are normative to this specification. Implementers must consider Unicode security considerations including homograph attacks and normalization issues.