On Stable Storage for Items in Concise Binary Object Representation (CBOR)
Extracted elements (20)
CBOR files need a protocol-specific magic number because identifying content as merely 'CBOR' is insufficient, analogous to how APK, OpenOffice, and MSOffice files are all ZIP files yet require further identification. This document provides a way to encode a magic number into the beginning of a CBOR format file so file(1) and similar tools can identify the specific CBOR Protocol.
Tag 55800 was picked to have properties similar to 55799: its encoded form 0xd9d9f8 is not valid UTF-8 (second 0xd9 is not a valid UTF-8 continuation byte) and not valid UTF-16 (0xd9d9 is in the high-half surrogate zone with no valid following sequence), preventing false-positive format identification.
Tag values with embedded zero bytes (e.g., 0x12003456) are discouraged to avoid confusing implementations that treat the magic number as a C string. The use of four ASCII characters mnemonic to the protocol is encouraged but not required.
The CBOR Tag Wrapped format is easier to retrofit to an existing single-item CBOR format because existing parsers typically tolerate unknown tags and may simply ignore them; the Labeled CBOR Sequence is preferred when the on-wire overhead of 8 extra bytes is unacceptable or when many items are concatenated.
The constant inner tag content 'BOR' (0x42_4f_52) was chosen so that the ASCII characters 'CBOR' appear prominently in hex dumps or editor views of the file, aiding human identification. It also gives the inner tag something to tag.
The specification was motivated by the disarray in ASN.1-based systems where PEM-encoded files all use the '.pem' extension, confounding public keys, private keys, certificate requests, and S/MIME content. CBOR Protocols should not suffer the same ambiguity.
The TN(ct) formula for CoAP Content-Format tags deliberately avoids zero bytes in the 4-byte tag number representation, preventing issues with C-string-based magic number matching. No tags are defined for Content-Format numbers 65025–65535 (the experimental range).
Applications sending CBOR Tag Wrapped data over constrained networks may remove the two outer tags when the type is understood from protocol context (e.g., a CoAP Content-Format Option). A CBOR Protocol specification should specify exactly which cases call for the envelope.
When concatenating multiple Labeled CBOR Sequences, if the CBOR-Sequence-based protocol does not define semantics for or at least tolerate embedded labels, care must be taken to remove the label from all but the first part to avoid parse errors.
CBOR Protocol designers must obtain a CBOR tag for each kind of object stored in files. To be in the 4-byte range with no leading zero bytes, the tag value must be in the range 0x01000000 (16777216) to 0xFFFFFFFF (4294967295) inclusive.
The CBOR-Labeled Non-CBOR Data method (tag 55801) prepends a 12-byte CBOR header to non-CBOR data: outer tag 55801, inner protocol-specific 4-byte tag, and the constant 3-byte byte string 'BOR'. The non-CBOR payload follows directly after the 12-byte header.
The CBOR Tag Wrapped enveloping method wraps a single CBOR data item in two nested tags: an outer self-described CBOR tag (55799) and an inner protocol-specific tag. The result is a deterministic first 8 bytes serving as a magic number.
The Labeled CBOR Sequence enveloping method prepends a 12-byte label to a CBOR Sequence. The label is: outer tag 55800 (self-described CBOR Sequence), inner protocol-specific 4-byte tag, and a 3-byte CBOR byte string containing 0x42_4f_52 ('BOR').
IANA allocated tag 55800 (encoded 0xd9d9f8) for the Labeled CBOR Sequence Enveloping Method in the 'Concise Binary Object Representation (CBOR) Tags' registry, with semantics 'indicates that the file contains CBOR Sequences'.
IANA allocated tag 55801 (encoded 0xd9d9f9) for the CBOR-Labeled Non-CBOR Data Enveloping Method in the 'CBOR Tags' registry, with semantics 'indicates that the file starts with a CBOR-Labeled Non-CBOR Data label'.
IANA allocated tag numbers 1668546817 (0x63740101) through 1668612095 (0x6374ffff) for CoAP Content-Format numbers, computed by TN(ct) = 0x63740101 + (ct / 255) * 256 + ct % 255, covering content-format numbers ct < 65025.
Clearly identifying CBOR contents in files may allow malware to identify, exfiltrate, or corrupt interesting stored objects. Protective applications (that check data) cannot rely on the applications that use the data making identical format-recognition decisions, so endpoint assessment technologies must not solely rely on these labels to decide whether to inspect a file.
Depending on OS configuration, the magic number labeling might influence which default application processes a file in ways a protective application cannot predict — a check-vs-use vulnerability where the checker and the user may disagree on format identification.
In the CBOR Tag Wrapped method, the encoding begins with tag 55799 (encoded as 0xd9d9f7, 3 bytes) followed by the protocol-specific 4-byte tag head (da + 4-byte tag number), totaling 8 bytes before the CBOR data item content.
The Labeled CBOR Sequence label is exactly 12 bytes: tag 55800 encoded as 0xd9d9f8 (3 bytes), the 4-byte protocol-specific tag head (0xda + tag number), and the CBOR-encoded 3-byte byte string 'BOR' (0x43 0x42 0x4f 0x52, 4 bytes).