The signature covers "the whole entry in canonical serialization" — and canonical serialization is defined nowhere

Источник: dev/fera-research/docs/oc1/027-canonical-serialization-undefined.md (read-only; сгенерировано scripts/sync-docs.mjs)

The signature covers “the whole entry in canonical serialization” — and canonical serialization is defined nowhere

• kind: problem
• area: trust
• evidence:
  • ADR.md:843-845 (ADR-021, manifest hardening) — “The signature covers the whole entry in canonical serialization, not just the artifact hash — otherwise host.max_version or fallback can be substituted without breaking the signature.”
  • TECHSPEC.md:392-394 — the same sentence, verbatim.
  • “Canonical serialization” occurs exactly twice in the entire corpus, and both occurrences are this rule quoting itself. Verified by grep for canonical across all of context/*.md: the only other hits are unrelated (“canonical tree” in OQ-021, “canonicalization” of sticker alpha in RESEARCH-OQ-003, a Canonical-ABI URL in OPTIM-IDEAS-4’s bibliography).
  • So the corpus defines no canonicalization rule at all: not the serialization format for the bytes that get signed, not key ordering, not number/string escaping, not whitespace, not what happens to absent vs null vs empty fields, not float/integer representation, not whether artifacts[] order matters.
  • The manifest’s format is itself under-specified at the same level: TECHSPEC.md:362-364 — “Format: JSON (chosen over RON for external-tooling interoperability, ADR-003)”; ADR-003:79-81 — “We publish a signed manifest … as plain JSON”.
  • Three separate JSON serializations now exist in the corpus with no agreement on which is signed: the trust manifest is JSON (TECHSPEC.md:363, ADR.md:805’s trust-manifest.json), the canonical tree’s copy is RON (OQ.md:711 — “manifest.ron (copy from Lattice, canonical) + manifest.ttso”), and Lattice’s own manifests are RON (OQ.md:398-399 — “the packed copy is RON”). Note OQ.md:711 calls the RON copy “canonical” while TECHSPEC.md:363 says the signed format is JSON — the same word meaning two different things in two documents.
  • QA has one gate here, and it is written against the wrong threat: QA.md:149-151 — “Manifest signature covers the entire entry: a tampered-but-unsigned modification of any field … is rejected.” That is a field-substitution test. It cannot detect a canonicalization mismatch, because two parties can produce different bytes for identical content and each verify its own.
  • Independently reached by oc3 (bus, finding 003: “canonical serialization undefined”).
• severity: high

Суть / Substance

The corpus’s single most load-bearing security rule — the one ADR-021 gives as the reason whole-entry signing exists — is stated in terms of a byte-level property (“canonical serialization”) that the corpus never defines, in a format (JSON) whose canonicalization is famously unsettled, while a second, different serialization of the same data (RON) exists in the canonical tree and is itself labelled “canonical”.

The rule “sign the canonical bytes” only works if every signer and every verifier agree on what those bytes are. Without a stated canonicalization:

• Two independent implementations disagree. A Rust signer using serde_json (field order as declared, nulls emitted, 2 vs 2.0 for floats) and a Python verifier using json.dumps(sort_keys=True) produce different bytes for the same object. One of them fails verification, and the failure looks like tampering — a warning dialog on an untampered manifest, or (worse, with ADR-004’s warn-not-refuse policy for hashes, finding 021) an untampered manifest that is silently distrusted.
• The attack the rule was written for still works if the format shifts. If canonicalization is not pinned, an attacker who can influence the parser’s output (duplicate keys, artifacts[] reordering, unknown-field handling) can get two byte-strings that a verifier accepts for two different logical entries — the classic JSON-canonicalization desync (differential parsing), which is precisely what host.max_version/fallback substitution is meant to prevent.
• ADR-004’s warning posture compounds it. If hash mismatches warn rather than refuse (finding 021), then a canonicalization mismatch on the manifest — where the real boundary is — degrades to a warning too, rather than being the hard failure it should be.
• Three serializations, no owner: JSON signed (trust manifest), RON “canonical” copy in the runtime tree (OQ.md:711), RON inside Lattice containers (OQ.md:398-399). Which one is authoritative, and which one is signed, is not stated.

This is the third instance of my cross-cutting pattern (“surrounding machinery designed, leaf artifact missing”) — after the capability IDs (finding 020) and the missing traits (finding 016) — and it is the one with the worst failure mode, because it is in the trust root.

Варианты решения (tradeoffs)

• (A) Adopt an existing, tested standard: JCS (RFC 8785) JSON Canonicalization Scheme, or a CBOR/COSE-signed manifest instead of JSON. Removes the ambiguity by delegating it to a specification with test vectors. Costs: JCS has opinions (no null, specific number rules) that must be checked against size, features, min_version etc.; moving off JSON conflicts with ADR-003:71-76’s explicit “chosen for external-tooling interoperability”.
• (B) Pin a project-specific canonical form and ship test vectors. Keep JSON, define the rule precisely (sorted keys, UTF-8, no insignificant whitespace, explicit null/absent policy, integers only, artifacts[] order-significant, no duplicate keys), and publish a fixture set so every implementation is checkable against the same bytes.
• (C) Sign a digest of a typed structure, not the serialized text. Decode → validate into a typed struct → serialize with one designated serializer for signing. Removes parser-differential risk from the trust root at the cost of “sign what you validated” discipline (the parser must be strict, rejecting unknown fields and duplicates).
• (D) Leave it. Reject — the rule currently references a nonexistent definition.

Предлагаемое решение

(B) now, with (C)’s discipline — i.e. one designated serializer plus a strict parse — and (A) as a documented fallback if JCS’s constraints clash with the schema.

1. New ADR addendum to ADR-021 defining canonical serialization in full: the exact serializer and version, key ordering, encoding, number rules (integers only — size and version fields should not be floats), absent-vs-null policy, duplicate-key rejection, array-order policy for artifacts[]/depends_on, and the rule that unknown fields are rejected before verification (not ignored) — otherwise a field-addition attack survives the round trip.
2. Ship canonicalization test vectors as a fixture in tools/manifest-signer/ (the crate already exists in the §0 layout at TECHSPEC.md:42), so signer and verifier can be tested against identical expected bytes, in Rust and in at least one non-Rust language (the “external-tooling interoperability” that motivated JSON at ADR-003:71-76).
3. Resolve the three-serialization question explicitly: state that the signed artifact is the canonical JSON, and that OQ.md:711’s RON copy is a derived, local convenience that is either (a) re-signed in its own right or (b) verified against a digest carried in the signed entry. Right now OQ.md:711 calls the RON file “canonical” and TECHSPEC.md:363 calls JSON the signed format.
4. Add the QA gate that was missing: QA.md:149-151 currently tests field substitution only. Add — “two encoders producing logically identical entries produce byte-identical canonical forms (test-vector based)”, and “a manifest with duplicate keys or unknown fields is rejected, not coerced”.
5. Make manifest verification a hard failure even for locally present manifests (the distinction I drew in finding 021 applies here too): a manifest that fails signature or canonicalization verification must not be used with a warning — unlike an artifact hash, there is no legitimate user-modifies-my-manifest workflow, and ADR-003’s design has the user manage keys (ADR.md:82-84), not manifests.