EBR/QSBR does not make dlclose safe — ADR-001 and ADR-020 depend on the false premise

Источник: dev/fera-research/docs/oc1/009-ebr-does-not-make-dlclose-safe.md (read-only; сгенерировано scripts/sync-docs.mjs)

EBR/QSBR does not make dlclose safe — ADR-001 and ADR-020 depend on the false premise

• kind: problem
• area: modules / trust
• evidence:
  • The false claim, stated three times:
    • OPTIM-IDEAS-5.md:22-24 — “That table is ‘protected’ by the epoch mechanism… This means if a thread is inside module X, then even if X is logically unloaded, its code isn’t freed until the epoch advances. … treat each unloaded module as ‘garbage’ in the current epoch.”
    • OPTIM-IDEAS-3-research.md:58 — “Unload / freeze: при переходе в background/idle выгружайте cold plugins (munmap или Wasm instance drop)… EBR/QSBR гарантирует safety.” (the research file asserts it, while citing Hart et al. two lines up at :18)
    • OPTIM-IDEAS-2.md:42-44 — “Plugin linking via EBR … unloaded modules sit in the reclamation queue until no thread can reach them”; OPTIM-IDEAS-2.md:77-79 / -5.md:45 — “unload code for rarely used commands until needed again (protected by our EBR scheme)”.
  • The correct statement, once: OPTIM-IDEAS-4-oq24.md:102 — “Это означает: EBR или HP не делают dlclose безопасным. Они защищают данные, а не код, который может быть выгружен.” — followed by the real policy at :99-101: “если отсутствие живого plugin-кода доказать нельзя, DLL держится до завершения процесса либо plugin запускается в отдельном процессе”, and :92-94 on libloading::Symbol<'lib, T> binding the symbol to the Library lifetime, with “Library::close может быть no-op в зависимости от платформы, а init/fini-код библиотеки выполняет произвольный код”.
  • Two Accepted ADRs rest on the false premise:
    • ADR.md:19-21 (ADR-001) — “nylon-ring (explicit version + struct-size validation on load, graceful unload with in-flight call draining, host-leased zero-copy output buffers)”.
    • ADR.md:812-813 (ADR-020 link-time dispatch addendum) — “Relink on hot-swap uses ERB (epoch-based reclamation) — new pointer table in a new epoch, old epoch drains before free”.
  • QA gate that tests the wrong thing: QA.md:129-132 — “A module can be hot-reloaded while a concurrent stream of calls is in flight, with zero dropped calls (relies on nylon-ring’s draining guarantee — write a test that actually exercises this, not just an assumption that the library provides it)”. The parenthetical is the right instinct, but the gate measures call accounting, not code liveness.
  • And QA’s “zero dropped” is itself contradicted by OQ.md:739-742 — “Host-function TTL: 1 minute, then kill … kill stragglers at 60 s”.
  • Repo reality: ~/dev/fera/fera has no dylib, no nylon dependency and no modules/ (commit ed4d66e) — so this is contained today, but the manifest/policy text is what an implementer will build against.
• severity: high

Суть / Substance

Epoch/quiescent reclamation protects memory objects from use-after-free: a reader that entered before the epoch flip is allowed to finish touching the data it holds a pointer to. It says nothing about instruction pages. Once dlclose/munmap retires the mapping, any thread that was inside the module’s code at that moment executes freed text. The reclamation queue that made the pointer safe is irrelevant to the code pointer the thread was executing.

So the chain in ADR-020:812-813 is broken in the middle:

“Relink on hot-swap uses EBR — new pointer table in a new epoch, old epoch drains before free”

“Old epoch drains” is a statement about the pointer table. If the free in question is the dylib, the epoch mechanism has not been made to say anything about it, and OPTIM-IDEAS-4:102 says so in one sentence.

This matters more than usual for Fera specifically, because ADR-024 (Accepted) makes frequent hot-swapping a product feature, not an engineering detail:

• ADR.md:427-428 — background update poll “every few hours” + explicit check-now button + a startup check.
• ADR.md:442-446 — developer mode: unsigned/local modules as an explicit opt-in toggle, so users are expected to be swapping dylibs by hand, constantly, in a running process.
• OPTIM-IDEAS-2.md:64-67 / -5.md:45 propose going further: “if chat is inactive, we could even unload code for rarely used commands until needed again” — i.e. unload/reload cycles as an energy feature. Every cycle is a dlclose on a path the corpus believes is safe.

Failure mode when it goes wrong: a call that was in flight when the mapping was retired jumps into unmapped memory. Depending on the platform and mapping layout this is usually a clean SIGSEGV, but it can equally be a silent re-entry into whatever got mapped at that address next — the non-deterministic, hard-to-diagnose class. Both ADR.md:20 and ADR.md:813 promise a guarantee the mechanism cannot deliver.

There is a second, cheaper-to-fix inconsistency stacked on top: the corpus has three answers for the reclamation scheme and two for the native ABI (see my finding 010), so the module-unload policy has never been written down as one coherent design.

Варианты решения (tradeoffs)

• (A) Accept dlclose unreliability; never dlclose in-process. Adopt OPTIM-4’s policy verbatim: swap by loading a new generation and leaking the old mapping (address space is cheap; code pages are demand- paged and get reclaimed when the process exits). Reclamation still governs data (pointer tables, arenas). Hot-swap remains fully functional — only reclamation of the mapping is given up. Cost: one mapping per generation per module; bounded by ADR-024’s “keep at most N-1” plus a cap on generations (e.g. keep last 2–3 mappings, then require a restart).
• (B) Process-per-module. Real isolation, real unload. But ADR.md:158-163 explicitly rejected separate processes for v1 (“IPC serialization overhead and deployment complexity”), carving out only media decoders. Reopening that is a bigger decision than the bug.
• (C) Function-pointer-only modules with no live code (WASM). Third-party plugins already take this path (ADR.md:52-56), where dropping an instance is safe. Doesn’t help first-party native modules.
• (D) Keep pretending. Reject.

Предлагаемое решение

(A) + fix the two ADRs.

1. New ADR addendum to ADR-020 (its link-time dispatch addendum is where the false inference lives), stating: epochs protect data, not code; a module generation’s mapping is retired only after the epoch drains and the host has proven no thread is executing in the mapping (e.g. a per-module in-flight counter reaching zero), and otherwise the mapping is retained until process exit.
2. ADR-001:20 — amend “graceful unload with in-flight call draining” to say explicitly that draining covers calls, not code pages, and that the mapping may outlive the drain.
3. Correct the three false statements in OPTIM-IDEAS-2.md:42-44, :77-79, OPTIM-IDEAS-3-research.md:58, OPTIM-IDEAS-5.md:22-24,45 (or mark those files superseded by -4, which is the file that got it right). Leaving -2/-3/-5 asserting the opposite is how this got into two ADRs in the first place.
4. Rewrite the QA gate at QA.md:129-132: it must (a) drop “zero dropped calls” — OQ.md:739-742 deliberately kills stragglers at 60 s — and (b) add the gate that actually catches this class: after a swap with a call in flight, no thread is executing in a retired mapping, plus a soak test of many swap cycles under load.
5. Binding detail worth keeping from OPTIM-4:92-94: hold module symbols as libloading::Symbol<'lib, T> so the borrow checker (not discipline) prevents a symbol outliving its Library.
6. Note the naming collision while here: the acronym is EBR, and ADR.md:812, OQ.md:639, OQ.md:741 all write “ERB” — including inside an Accepted ADR decision block.