Metalink/HTTP: Mirrors and Hashes

ETags should be derived from a cryptographic hash of file contents rather than server-unique filesystem metadata, so that mirrored copies of the same file share the same ETag across all coordinated servers, enabling early mismatch detection by clients.

http

Metalink/HTTP uses existing HTTP standards (Web Linking RFC 5988, Instance Digests RFC 3230, ETags RFC 2616) rather than defining new header fields, in order to provide Metalink/XML equivalent functionality while remaining compatible with standard HTTP infrastructure.

http

Traditional HTTP redirects to a mirror convey only a single link with no failure handling. Metalink/HTTP instead transfers the full mirror database knowledge to the client, enabling automatic failover, multi-source parallel downloads, and hash verification without user intervention.

http

FTP mirror servers are always classified as 'normal' mirrors because they do not emit ETags or support Instance Digests in HTTP. Early file mismatch detection is therefore not available when downloading from FTP mirrors.

http

Metalink clients SHOULD close all but the fastest connection if any range request receives a complete (200) response instead of a partial (206) response, since some mirrors may not support HTTP byte ranges.

http

If Instance Digests are not provided by the Metalink server, the Link header fields pertaining to this specification MUST be ignored.

http

If Metalink clients find Link header fields from mirrors that in turn list mirrors, or from a Metalink server listing itself as a mirror, they MUST discard such Link header fields to prevent a possible infinite loop.

http, security

Metalink clients MUST limit the number of parallel connections to mirror servers, ideally based on observing aggregate throughput changes. Clients MUST NOT make more than one concurrent request to each individual mirror server.

http, congestion

Metalink clients MUST reject individual downloads from mirrors where the file size does not match the file size as reported by the Metalink server.

http

Metalink clients MUST support HTTP and SHOULD support FTP. Clients MUST support Instance Digests in HTTP by requesting and verifying cryptographic hashes. Clients SHOULD support error recovery using partial file hashes from Metalink/XML files.

http

Metalink clients MUST verify the cryptographic hash of the file once the download has completed. If the hash does not match the Instance Digest from the Metalink server, error correction via Metalink/XML partial hashes SHOULD be attempted.

http, security

Metalink clients SHOULD use preferred mirrors, if possible, and SHOULD use If-Match conditions based on the ETag from the Metalink server response to quickly detect out-of-date mirrors.

http

Metalink clients that support verifying digital signatures MUST verify digital signatures of requested files if included. Digital signatures MUST validate back to a trust anchor per RFC 3156 and RFC 5280.

http, security, crypto

Metalink servers and their associated preferred mirror servers MUST all share the same ETag policy; normal mirror servers SHOULD share the same ETag policy. Byte-for-byte identical files must have the same ETag across coordinated servers.

http

Metalink servers MUST include the cryptographic hash of a resource via Instance Digests in HTTP (RFC 3230), and MUST include SHA-256 as specified in FIPS-180-3. Other hash algorithms MAY also be included.

http, security, registry

Metalink servers MUST support Link header fields for listing mirrors and MUST support Instance Digests in HTTP. Metalink servers MUST return the same Link header fields and Instance Digests on HEAD requests as on GET requests.

http

Mirrors are specified using Link header fields (RFC 5988) with relation type 'duplicate'. Optional attributes include 'pri' (priority), 'geo' (ISO 3166-1 alpha-2 country code), 'pref' (preferred mirror flag), and 'depth' (mirror directory depth).

http

OpenPGP signatures are referenced via Link header fields with rel='describedby' and type='application/pgp-signature'. S/MIME signatures use type='application/pkcs7-mime'. Metalink clients SHOULD support both signature types.

http, security, crypto

Peer-to-Peer metainfo files (e.g., BitTorrent .torrent, Metalink/XML .meta4) are referenced via Link header fields with relation type 'describedby' and a 'type' parameter indicating the MIME type. An optional 'name' attribute identifies a specific file within the metainfo.

http

Preferred mirror servers are HTTP mirrors that MUST share the same ETag policy or MUST provide Instance Digests using the same algorithm as the Metalink server; they enable early file mismatch detection before data transfer. Normal mirrors (including FTP) do not provide these guarantees.

http

This document registers the 'duplicate' link relation type in the IANA 'Link Relation Types' registry. It refers to a resource whose available representations are byte-for-byte identical with the context IRI; it is intended for static resources only.

http, registry

Metalink clients handle URIs and IRIs; security considerations from RFC 3986 Section 7 and RFC 3987 Section 8 apply regarding parsing, normalization, and comparison of resource identifiers.

security, http

Metalinks SHOULD include digital signatures to provide authentication, message integrity, and non-repudiation. A shared ETag or Instance Digest from a mirror cannot strictly protect against a malicious actor who controls the mirror and can forge expected values.

security, crypto

Some digest algorithms in the Instance Digests registry (particularly the MD family) are cryptographically weak and vulnerable to collisions, allowing an attacker to substitute a malicious file with a matching weak hash. SHA-256 or stronger algorithms MUST be used.

security, crypto

Spoofing attacks are possible if an attacker publishes Metalinks with false mirror URIs, deceiving users into downloading malicious files or enabling distributed denial-of-service via unrelated URIs. Clients should only download from trusted sources and must implement loop prevention.

security, http

Client download flow: (1) GET or HEAD to Metalink server to receive Link/Digest/ETag headers; (2) issue range GET to preferred mirror with If-Match ETag condition; (3) optionally open parallel range requests to additional mirrors; (4) monitor aggregate throughput to gate new connections; (5) verify whole-file hash on completion; (6) if hash fails, fetch Metalink/XML for partial-hash error correction.

http

Cryptographic hashes in Instance Digest HTTP header fields are encoded in base64, while hashes in Metalink/XML files are encoded in hexadecimal. Implementors must handle both encodings correctly.

http

Mirror depth is expressed as an integer 'depth' attribute on Link header entries. 'depth=0' (default) means only the specific file is mirrored; 'depth=N' means N URI path segments closer to the host are also mirrored.

http

Mirror priority is expressed as an integer 'pri' attribute (1–999999) on Link header entries; lower values indicate higher priority. Mirrors without a 'pri' attribute are treated as having pri=999999 (lowest priority).

http