One ModuleLoader, two implementations: static-link dev/bench mode — which is also the hermetic production build

Источник: dev/fera-research/docs/oc4/idea-08-static-module-loader.md (read-only; сгенерировано scripts/sync-docs.mjs)

One ModuleLoader, two implementations: static-link dev/bench mode — which is also the hermetic production build

• kind: idea
• area: modules / tooling / trust
• evidence:
  • ~/dev/fera/project-design/context/ADR.md:16-21 — ADR-001: the app is “a thin launcher binary and a set of hot-swappable native modules (dynamic libraries)”
  • ~/dev/fera/project-design/context/ADR.md:33-35 — “Cross-boundary calls have real overhead (vtable dispatch, no inlining, marshaling)”
  • ~/dev/fera/project-design/context/OQ.md:536-540 (OQ-025) — Phase 2: “subsystem repos (render/audio/video/anim/plugins/registry), each a workspace of many small cdylibs”
  • ~/dev/fera/project-design/context/OQ.md:802-805 — “Given the very large dylib count (ADR-020 struct-per-dll), the in-memory module-storage model is itself an optimization target”
  • ~/dev/fera/project-design/context/OQ.md:874-883 — OQ-024’s nylon-ring fork intent: remove virtual tables for link-time pointer tables, “tune it close to epochs (QSBR)”, “pin the Rust compiler version so our own modules compile without #[repr(C)]” — all only relevant because the modules are dylibs
  • ~/dev/fera/project-design/context/ADR.md:442-446 — ADR-024 developer mode: “unsigned/local modules are an explicit opt-in developer mode” — i.e. still dylibs, still dlopen, still a loader
  • ~/dev/fera/fera/CLAUDE.md:13-15 — “Cross-repo deps must be git deps … local overrides via gitignored .cargo/config.toml paths” — neither exists in the tree today (verified: no [workspace.dependencies], no git deps, no .cargo/config.toml)
  • ~/dev/fera/fera/rust-toolchain.toml:2-3 — the comment justifies the pinned nightly by module ABI stability, yet #[no_mangle] extern "C" appears nowhere in the repo; nothing currently crosses a C ABI
  • ~/dev/fera/fera/clippy.toml:2 — avoid-breaking-exported-api = false — the exported-API guard is disabled in a repo whose premise is versioned hot-swappable contracts
  • ~/dev/fera/fera/core/crates/ui-contract/src/lib.rs:248-265 — the Engine trait already has a second implementation-free shape available (nothing uses dyn Engine)
  • ~/dev/fera/project-design/context/ADR.md:28-30 — “Every module boundary must be designed as a trait contract (interface crate) shared between host and module”

Суть / Thesis

ADR-001 says “design every boundary as a trait contract” — and then builds the entire runtime on a mechanism that throws trait contracts away: dynamic loading gives you vtable dispatch, no inlining, marshaling, dlopen failure modes, ABI version negotiation, and a signed-manifest pipeline to distribute bits (ADR-003, ADR-021, ADR-024).

Both halves are correct. But the consequence is that there is currently no way to develop the system without exercising the expensive half. OQ-025 plans hundreds of cdylibs. ADR-024’s “developer mode” only relaxes signing, not loading. Every inner-loop iteration — a storage trait method, a search change, a text-layout fix — means rebuilding and loading dylibs.

The fix is one trait with two implementations, and the payoff is bigger than developer ergonomics:

trait ModuleLoader {
    fn load(&mut self, entry: &ManifestEntry) -> Result<ModuleSet, LoadError>;
}
// DylibLoader  — shipping: manifest, signatures, link-time pointer tables, ERB/QSBR
// StaticLoader — everything linked at compile time; no dlopen anywhere

Same contract crates, same impl code, #[cfg]-selected. Critically: the static impl must implement the identical trait, so a missing or mis-specified method becomes a compile error rather than a runtime dispatch hole. That makes static mode a free completeness check on the contracts themselves — which is precisely the property ADR-014 claims for contract tests, achieved by the compiler.

The non-obvious payoff: dev mode and the most trustworthy production build are the same build

There is a production configuration nobody has named: a hermetic Fera — the entire official module set statically linked, with no dynamic loading at all. No dlopen, no manifest-to-artifact resolution at runtime, no code loading attack surface whatsoever. For a client whose official supported set is a known “quad” per OS (OQ-012: Engine + Native + Feature + Style-data), such a build is fully functional — it loses only hot-swap and third-party native modules.

That is a real user segment: people who want the official client, don’t want a modding runtime, and will not install a binary that dlopens signed blobs from a network endpoint (a scenario ADR-013/024 create by design). Today that build does not exist. With StaticLoader it is the same code path as the dev build — so it is maintained for free by the people who use it daily, rather than being a special build nobody ever runs.

Two modes, one implementation, and the “risky” configuration pays the maintenance cost of the one everyone uses.

Варианты / Options (with tradeoffs)

A. Everything is a dylib, forever. (status quo)

• Pros: one path; no divergence; the shipping configuration is the only configuration.
• Cons: every inner-loop iteration pays dlopen + manifest + version negotiation; contract completeness is never compiler-checked; no hermetic build; and — per clippy.toml:2 — the exported-API guard is off, so nothing catches a breaking contract change until load time.

B. Monorepo static linking (all modules statically linked, no dylib support at all).

• Pros: fastest builds, best codegen, simplest.
• Cons: destroys ADR-001’s core premise. Hot-swap is the stated reason the architecture exists (“swapping individual subsystems … prohibitively expensive” with a monolith). Storage/search/media/renderer swaps are the project’s whole reason for being micro-kernel-shaped. Not viable alone.

C. Two loaders behind one trait (proposed).

• Pros: dev iteration at full speed; compiler-checked contract completeness; hermetic build for free; benchmarks gain a measured lower bound to compare the boundary cost against.
• Cons (real):
  1. “Works static, breaks dylib” — the classic risk. Mitigation is non-negotiable: contract tests run in both modes in CI, and the dylib path is never the untested one.
  2. Perf honesty risk — static-mode benchmarks are not shipping numbers. Mitigation: the bench harness must label the mode, and ideally report both — which turns the ADR-005/FUTURE-CHECKS §3 question (“is the boundary overhead actually worth batching?”) into a directly measured delta rather than an estimate.
  3. Two code paths for anything living inside the loader — version checks, manifest parsing, load-order derivation. Mitigation: those must live above ModuleLoader, shared by both, so only the mechanical bind/relink differs.
  4. Cargo feature surface — static impls are per-crate, so N modules × 2 modes. Mitigation: the static loader handles only modules present in the workspace; anything external or user-swapped always goes dylib, which makes the fallback path real rather than theoretical.

Proposed solution

1. The trait boundary, placed deliberately

ModuleLoader sits below everything policy-shaped:

  manifest parse / signature verify / load-order derivation / version checks
                              │   (shared, above the trait, tested once)
  ┌───────────────────────────┴───────────────────────────┐
  │  DylibLoader            │  StaticLoader               │
  │  dlopen + pointer tables│  direct construction        │
  │  + ERB/QSBR epochs      │  no epochs, no relink      │
  └───────────────────────────────────────────────────────┘

Everything the trust model cares about is in the shared upper half, so the hermetic build cannot skip signature verification — it verifies the same manifest at build time and bakes the result in. That is the correct semantics (not “skip verification”), and it is a genuinely nice property: a hermetic build is a build whose manifest was verified, not a build that ignored one.

2. What each mode is for

	DylibLoader	StaticLoader
Shipping (general)	✅	✅ (hermetic build)
Inner-loop development	slow	✅ default
cargo test on module code	needs a loader harness	✅
CI contract tests	needs signing harness	✅ (both run)
Hot-swap / ERB / QSBR	✅	n/a
Contract completeness	runtime	✅ compile-time
Benchmarks	shipping numbers	lower bound / delta

3. Make the third-party-native-module story consistent with ADR-024

ADR-024 already plans “unsigned/local modules” in developer mode. With two loaders the matrix is clean and worth writing down: first-party official modules are in the binary (static); third-party native modules are always dylib (they cannot be statically linked — they arrive after the build). That is not a limitation, it is the correct trust boundary, and it means the hermetic build does not need any new policy — it simply has no third-party native modules, and says so.

4. Do this after the loader exists, not before

Honest sequencing: fload does not exist yet, trust is a 8-line placeholder (fera/core/crates/trust/src/lib.rs:8), the launcher is a println!, and the module set is one in-memory MemoryEngine. So this is not a v1 blocker. It is the proposal for when the first real dylib exists — and the reason to do it then rather than later is that ModuleLoader is much cheaper to introduce before it has a dozen call sites, and much cheaper still if avoid-breaking-exported-api is turned back on at the same time (idea-06’s neighbourhood).

5. One thing to check first

Whether nylon-ring/fload can express this at all, or whether its API bakes in dlopen so hard that a static loader means reimplementing its dispatch layer. OQ-024’s fork intent (a)–(d) is already about rewriting that library’s internals to “our approach” — so the question is likely to be answered “yes, happily” by a fork that is planned anyway. Worth one afternoon with the nylon-ring API before committing to the shape.