The wire header never matched spec §3 — MAGIC, field order and size all diverge, and TODO.md's plan grows from the wrong baseline

Источник: dev/fera-research/docs/oc2/header-layout-never-matched-spec.md (read-only; сгенерировано scripts/sync-docs.mjs)

The wire header never matched spec §3 — MAGIC, field order and size all diverge, and TODO.md’s plan grows from the wrong baseline

• kind: problem
• area: protocol
• severity: high
• evidence:
  • ~/dev/fera/chroix-project/CHROIX-DISPLAY-LIST.md:71-86 — spec §3 Header: magic 'CHDL', order magic, header_size, abi_version, _pad, frame_id, op_count, op_stream, strings, glyph_runs, images, transforms, damage, damage_count = 13 fields, 52 bytes
  • ~/dev/fera/chroix-contract/src/header.rs:9 — pub const MAGIC: u32 = u32::from_le_bytes(*b"CXL\0"); (not 'CHDL')
  • ~/dev/fera/chroix-contract/src/header.rs:46-78 — code Header: 14 fields, frame_id placed before abi_version, plus an extra op_bytes: u32
  • ~/dev/fera/chroix-contract/src/header.rs:80-85 — assert!(size_of::<Header>() == 56)
  • ~/dev/fera/chroix-contract/src/header.rs:56-57 — a comment claiming the field name is pinned by “the wire-layout spec (CHROIX-DISPLAY-LIST.md §3)”
  • ~/dev/fera/chroix-contract/src/decode.rs:79-87 — reads frame_id@8, abi_version@16, op_count@20, op_bytes@24, op_stream@28
  • ~/dev/fera/chroix-contract/src/validate.rs:36-40 — reads abi@[16..18], with a layout comment: [8..16) frame_id, [16..18) abi_version
  • ~/dev/fera/chroix-contract/src/lib.rs:29-30 — stale: // 13 fields: 2*u32 + 1*u64 + 2*u16 + 8*u32 = 8 + 8 + 4 + 32 = 52
  • ~/dev/fera/chroix/TODO.md:37-42 (G2), :59-62 (D1), :94-99 (Stage 2a: “grow header 56→72, ABI 0.1→0.2”)
  • ~/dev/fera/chroix/TODO.md:7-9 — “Specs are authoritative and read-only … Never edit them”

Sum (Суть)

chroix/TODO.md records G2 as “validate_header reads the ABI version from the wrong offset” — a one-line off-by-eight bug. That framing is what caused the deeper defect to be missed. The evidence says G2 was a symptom of a layout divergence, and it has already been “fixed” in the working tree without anyone noticing the cause.

The original bug is the proof

The G2 bug was validate.rs reading the ABI version at [8..10]. Look at where the spec puts it (DISPLAY-LIST.md:73-75): magic[0..4) header_size[4..8) abi_version[8..10). [8..10] is exactly the spec’s offset. So validate_header was written from the spec, while header.rs and decode.rs were written from a different layout in which frame_id: u64 occupies [8..16). Two functions in one crate followed two different specs, and the mismatch surfaced as a garbage Version.

Fixing the offset (validate.rs:38 now reads [16..18], matching decode.rs:79) makes the crate self-consistent — and hides the real finding: the wire format has never matched the spec it claims to implement.

What actually diverges

	spec §3	code	consequence
magic	'CHDL' (0x4348444C)	"CXL\0" (0x004C_5843)	any spec-conforming reader rejects a chroix buffer, and vice versa
order	abi_version,_pad then frame_id	frame_id then abi_version,_pad	the same 8 bytes mean different things at every offset ≥8
size	52 B (13 fields)	56 B (14 fields)	—
extra field	—	op_bytes: u32 @[24..28)	occupies the offset the spec gives to op_stream

So spec op_stream is at [24..28) while code op_bytes is at [24..28); spec damage_count is at [48..52) while code damage is at [48..52). A buffer produced by this crate is not a §3 buffer, and no amount of self-consistency testing will reveal it, because both sides of every test are the code.

Two supporting symptoms of the divergence being invisible:

• header.rs:56-57 justifies naming the pad field _pad by appeal to “the wire-layout spec (CHROIX-DISPLAY-LIST.md §3)” — i.e. the spec is being consulted for details, while being contradicted in the layout itself.
• lib.rs:29-30 still carries the spec’s arithmetic (“13 fields … = 52”) directly contradicting header.rs:81-83 (“14 fields … = 56 bytes”). Two different header sizes live in one crate.

Why this matters more than G2

1. The spec is declared authoritative and read-only (TODO.md:7-9), so per the project’s own rules the code is what’s wrong — and the divergence is therefore a real defect, not a spec-update candidate. Nobody wrote it down, because each defect presents as something else.
2. TODO.md Stage 2a grows the wrong baseline. D1 and Stage 2a both assume the current header is the spec’s header minus two append-only fields, and plan “56 → 72 bytes, ABI 0.1 → 0.2” (:94-99). Appending brushes/blobs to a layout that never matched §3 produces a v0.2 header that matches §3 even less. The question “what should the header be?” has never been answered — only “what should it be plus four fields?”
3. Stage 4 will find this, but late. The planned “adversarial spec-conformance audit … offsets, endianness, append-only rules” (TODO.md:135-137) is exactly the check that catches this — and it is the last stage, after Stage 2 has rewritten the header and the encoder.

Related: the fix is uncommitted, and the second copy still has the bug

git status in chroix-contract shows 9 modified, uncommitted source files — including the validate.rs offset fix and its regression tests. Meanwhile both backends resolve chroix-contract as a git dependency pinned to #c37de46a (chroix-vello/Cargo.toml:16, chroix-headless/Cargo.toml:18), i.e. they build against the committed tree, not the working tree beside them. And the vendored copy at chroix/crates/contract is still on the old code (TODO.md G4 calls it “a diverging fork”). So the fix exists in one working tree, is invisible to both backends, and is absent from the second copy.

Options (with tradeoffs)

• A. Answer “what should the header be?” before Stage 2a. Compute the §3 layout and compare field-by-field against header.rs, then decide explicitly: conform the code to §3, or amend §3 (which needs the spec owner, since it is read-only for the implementers). Cost: one decision, minutes. Avoids growing the wrong baseline by four fields.
• B. Defer to Stage 4’s conformance audit. Cost: the header gets rewritten twice, and the ABI gets bumped twice (0.1→0.2, then again), with backends pinned to the first bump.
• C. Treat MAGIC as a naming question only. Cheap, but it misses the field-order divergence, which is the part that actually breaks a reader.

Proposed solution

A, folded into Stage 1 (before 2a). This is a decision, not a code change, and it is cheap now and expensive later:

1. Settle the layout. Either (i) conform header.rs to §3 (52 B, 'CHDL', abi_version before frame_id, drop or relocate op_bytes) — the code is the wrong side per TODO.md:7-9; or (ii) get the spec owner to amend §3 to the 56 B layout. Recommend (i), since the spec is authoritative and the extra op_bytes is derivable from op_stream’s extent anyway.
2. Fix MAGIC to 'CHDL', or record explicitly that the spec is wrong here. 'CHDL' reads as “Chroix Display List” and CXL as an abbreviation — the latter was never in the spec.
3. Delete the stale lib.rs:29-30 arithmetic — a second header size in the same crate is exactly how this stays invisible.
4. Commit the 9 modified files. Backends are pinned to a git SHA, so uncommitted contract work is unobservable to both of them. Also close the vendored copy (G4) or the fix will fork again immediately.
5. Add the offset table to the conformance audit as a first check, not the last, and make it a table-driven test: for each field, assert (name, spec_offset, code_offset, width) all four agree. That single test would have caught MAGIC, the reorder, and G2 in one shot.

Severity is high because it is a protocol correctness defect at the boundary the whole framework is organized around — and because TODO.md’s current plan will grow the divergence rather than close it, under an ABI bump that is meant to be cheap precisely because it is the first one.