xtask fork-sync: automated upstream-drift detection for the 5 long-lived forks

Источник: dev/fera-research/docs/oc4/idea-01-fork-sync-xtask.md (read-only; сгенерировано scripts/sync-docs.mjs)

xtask fork-sync: automated upstream-drift detection for the 5 long-lived forks

• kind: idea
• area: tooling
• evidence:
  • ~/dev/fera/project-design/context/ADR.md ADR-012 (fork milli → mikkaj, retargeted heed→sanakirja), ADR-026 (fork grammers → grmrs)
  • ~/dev/fera/project-design/context/OQ.md OQ-024 (“The library we planned to use for the module runtime gets forked: 1. optimize it, 2. rewrite its internals to our approach”) → fload
  • ~/dev/fera/project-design/context/NAMING-UX.md §1 (“Repos in the thefera org: fera, fload, mikkaj, grmrs, lattice, rstgcalls”)
  • ~/dev/fera/project-design/context/OQ.md OQ-021 “Resolved details” (derived signed index of module manifests — RON source of truth → Turso queryable copies)
  • ~/dev/fera/community/TOOLCHAIN.md §“Bump procedure” (prove N-1 compatibility, classify break as “N-1 compatible” vs “hard break”)
  • ~/dev/fera/community/COMMIT-STYLE.md (referenced from community/README.md: “benchmark proof for perf:”)
  • ~/dev/fera/fera/tools/manifest-signer/src/main.rs, ~/dev/fera/fera/tools/bench-scene/src/main.rs — the only two tools that exist

Суть / Thesis

The entire architecture rests on five hand-maintained forks plus a pinned compiler:

Fork	Upstream	Why	Drift pressure
fload	nylon-ring	OQ-024: remove vtables, flat pointer tables, QSBR, pin rustc so modules drop #[repr(C)]	highest — the fork rewrites the internals
mikkaj	milli (Meilisearch)	ADR-012: swap the heed/LMDB layer for sanakirja	highest — upstream’s whole storage-facing surface moves
grmrs	grammers	ADR-026: reshape for secret chats (ADR-025) + passkey login (ADR-027) + typestate (ADR-015)	high — new TL types land upstream constantly
lattice	— (standalone)	tracks nothing, but has the .lttc spec as its own compatibility surface	low
toolchain	stable/nightly	community/TOOLCHOWN.md policy	policy exists, automation does not

ADR-012 states the cost plainly: “the fork must track milli’s heed-facing API surface carefully across upstream updates. This is accepted as ongoing maintenance burden.” ADR-026 restates it: “We own protocol maintenance: tracking MTProto changes becomes our job.” ADR-021 adds a fifth axis that also has to stay pinned: modules must all be built by one rustc version.

The burden is accepted in prose but never budgeted in tooling. There is no CI job, no script, no dashboard anywhere in the plan that asks the one question that decides whether the architecture is still buildable: “has upstream moved under us?” Today the answer is found out the hard way — mid-refactor, on the author’s machine, when a nightly bump or a milli release silently breaks the heed shim.

Proposed idea: a single xtask binary (in fera, alongside the two existing tools) with a fork-sync subcommand that, on a schedule and on demand, reports per-fork upstream drift as a machine-readable artifact and opens a tracking issue with a categorized diff summary.

Variants / Options (with tradeoffs)

A. Manual “I remember to check” (status quo).

• Cost: zero tooling.
• Failure mode: exactly the expensive one — a mid-refactor break, discovered by surprise, with no bisect point and no recorded classification.

B. “Just rebase and let CI tell me.”

• cargo update / git rebase upstream → CI red → fix.
• Cheapest to build (no analysis, just a cron rebase PR).
• Tradeoff: breaks OQ-024’s rustc-pinning guarantee. CI rebase on a nightly would produce a module set built by a different rustc than the host expects, and the “drop #[repr(C)]” property (OQ-024 (d)) is exactly the assumption that a rustc mismatch silently invalidates. Must be constrained to a matrix, not a blind rebase.
• Also: a red CI on a 300-file rebase tells you that you drifted, not where, and not whether the semantic contract (ADR-012’s storage semantics) survived.

C. xtask fork-sync — three independent detectors, one report (proposed).

1. Release/tag watcher (trivially cheap). Poll upstream for new tags. Output: “milli 1.4 released 3 days ago”. Pure signal, no false positives.
2. API-surface differ (the load-bearing one). For each fork, extract the set of externally consumed symbols — for mikkaj that’s the heed-facing surface we shim; for fload the define_plugin! / host-API surface we patch at link time; for grmrs the client API our protocol crate calls. Run it against our pinned fork HEAD and against upstream HEAD. Output: an exact add/remove/signature-change list — i.e. “upstream renamed Database::open_ro and dropped RwTxn::commit_with” — not “CI is red”.
3. Contract-test differ. Each fork has (or should have) a conformance suite (ADR-014’s principle, applied upstream). Run the suite against upstream-HEAD-built artifacts in a throwaway workspace. Output: which semantic promises broke, as test names — directly re-usable as the “what broke / what still works” statement community/TOOLCHAIN.md already demands for toolchain bumps, extended to forks.

Plus a toolchain-matrix leg that automates what TOOLCHAIN.md currently mandates by hand: run the full suite on the pinned channel and on the previous supported channel, and emit the N-1 compatible / hard break classification as a build artifact rather than as prose someone has to write.

Output artifact: .xtask/fork-sync.json, plus one issue per fork per drift event, deduplicated against the last seen state (a persistent “last seen symbol set” in the repo, so the report is always incremental: “3 new upstream changes since 2026-10-02”, not a full diff of the world).

Proposed solution

Adopt C, with these deliberate scope choices:

• Detector 1 ships first and alone. It is ~50 lines, has zero false positives, and already delivers the majority of the “surprise factor” reduction — knowing a release exists is most of the information.
• Detectors 2 and 3 are the real work and are scoped per-fork by risk. Order by drift pressure: fload → mikkaj → grmrs. fload first because OQ-024 is about to rewrite its internals wholesale; a drifted base during a rewrite is the worst case.
• Detector 2’s symbol extraction reuses what the project already needs. A “which of our fork’s public symbols do we actually call” set is needed for any upgrade decision and is exactly the fload manifest metadata (OQ-021: module metadata must carry the required rustc version — the same metadata block can carry the consumed-surface fingerprint). One artifact, two consumers.
• The rustc-pin is enforced by CI, not by convention. Since OQ-024 (d) makes cross-module Rust ABI stability conditional on a single compiler version, that becomes a hard CI gate: any module artifact whose recorded rustc_version ≠ the host’s is refused at build time. Cheap, and it converts a silent assumption into an enforced invariant.
• Detector 3 is deliberately scoped to semantic claims, not coverage numbers. A test that still passes but changed meaning is the real risk (ADR-012’s storage semantics, ADR-021’s edition semantics). So the report should name tests, and mark any upstream test whose name or fixture changed as needing human review, even if green.

Why this is cheap relative to its value

The project already accepts four forked dependencies as a permanent condition of its architecture. The cost of knowing when they move is a few hundred lines of tooling. The cost of not knowing is a design premise silently falsified — and per ADR-012 the failure mode is explicitly “ongoing maintenance burden”, i.e. the thing that erodes architecture over years.