Lightweight Directory Access Protocol (LDAP): Internationalized String Preparation
Extracted elements (20)
The algorithms are based on the 'stringprep' (RFC 3454) approach but add two steps: transcoding to Unicode before applying stringprep steps, and insignificant character handling after. This refinement addresses X.500 matching requirements not covered by basic stringprep.
The complex substring space handling (rather than a simpler 'collapse all spaces' approach) is necessary to prevent unexpected matching behavior: simplified handling would cause (CN=foo\20*\20bar) to match 'foobar', and would cause sub-partitioning failures where subdividing a matching substring into multiple substrings would fail to match.
The X.500 substrings rule requires that insignificant space treatment apply to assertion and attribute values as whole strings, not individually to substrings. This means an assertion value matches any attribute value differing only by inclusion or removal of insignificant spaces, which drives the two-space normalization design.
The previous X.520 caseIgnoreMatch definition was insufficient for Unicode strings: different implementations could yield different results depending on whether they folded lowercase to uppercase or vice versa, and on how they defined 'space' (only U+0020, any Zs character, or any WS category character).
Each non-Unicode string value must be transcoded to Unicode as the first step. PrintableString transcodes directly; UniversalString, UTF8String, and bmpString need not be transcoded; TeletexString transcoding is left a local matter.
The combining mark table in Appendix A is normative and definitive for purposes of implementing this specification. It lists all Unicode code points with Mn, Mc, or Me mark properties and was derived from Unicode 3.2 data files.
The input string must be normalized to Unicode Form KC (compatibility composed) as described in UAX15. This is the Normalize step.
The Prohibit step rejects all Unassigned code points (Table A.1 of RFC 3454), characters that change display properties or are deprecated (Table C.8), Private Use code points (Table C.3), non-character code points (Table C.4), surrogate codes (Table C.5), and REPLACEMENT CHARACTER (U+FFFD). The step fails if any prohibited code point is present.
The six-step preparation process (Transcode, Map, Normalize, Prohibit, Check bidi, Insignificant Character Handling) SHALL be applied to each presented and attribute value in preparation for character string matching rule evaluation. Failure in any step causes the assertion to evaluate to Undefined.
Use of TeletexString is NOT RECOMMENDED because there is no standard for mapping TeletexString values to Unicode.
For insignificant space handling in case ignore and exact string matching, a space is defined as SPACE (U+0020) followed by no combining marks. For attribute or non-substring assertion values: if no non-space characters, output is exactly two SPACEs; otherwise the string starts and ends with exactly one SPACE and any inner sequence of spaces is replaced with exactly two SPACEs.
For numericString insignificant character handling, all spaces (SPACE U+0020 followed by no combining marks) are regarded as insignificant and are removed entirely. An all-space string results in an empty string.
For substring assertion values in insignificant space handling: if no non-space characters, output is exactly one SPACE. Initial substrings are modified to start with exactly one SPACE; substrings ending in spaces are modified to end with exactly one SPACE; substrings starting in spaces are modified to start with exactly one SPACE; final substrings are modified to end with exactly one SPACE.
For telephoneNumber insignificant character handling, a hyphen is defined as one of HYPHEN-MINUS (U+002D), ARMENIAN HYPHEN (U+058A), HYPHEN (U+2010), NON-BREAKING HYPHEN (U+2011), MINUS SIGN (U+2212), SMALL HYPHEN-MINUS (U+FE63), or FULLWIDTH HYPHEN-MINUS (U+FF0D) followed by no combining marks. All hyphens and spaces are considered insignificant and removed.
In the Check bidi step, bidirectional characters are ignored.
In the Map step, for case ignore, numeric, and stored prefix string matching rules, characters are case folded per B.2 of RFC 3454. ZERO WIDTH SPACE (U+200B) is mapped to nothing; all other code points with Separator property (Zs, Zl, Zp) are mapped to SPACE (U+0020).
In the Map step, SOFT HYPHEN (U+00AD), MONGOLIAN TODO SOFT HYPHEN (U+1806), COMBINING GRAPHEME JOINER (U+034F), VARIATION SELECTORs (U+180B-180D, FF00-FE0F), and OBJECT REPLACEMENT CHARACTER (U+FFFC) are mapped to nothing. Control characters like CHARACTER TABULATION, LINE FEED, CARRIAGE RETURN, and NEXT LINE are mapped to SPACE (U+0020).
Insignificant Character Handling differs by matching rule: Section 2.6.1 applies to case ignore and exact string matching (space handling), Section 2.6.2 applies to numericString matching (remove all spaces), and Section 2.6.3 applies to telephoneNumber matching (remove all hyphens and spaces).
The string preparation process operates on Unicode 3.2 as the character repertoire. Implementations may use alternative processes so long as matching rule evaluation behavior is consistent with this specification.
The stringprep (RFC 3454) security considerations generally apply to the algorithms described in this document. Imprecise character string matching has led to security vulnerabilities when used in certificate chain validation.