QSBR quiescent state is per-thread, but the runtime is work-stealing — "quiescent points in the actor loops" cannot be per-thread events

Источник: dev/fera-research/docs/oc5/qsbr-work-stealing-per-thread-invariant.md (read-only; сгенерировано scripts/sync-docs.mjs)

QSBR quiescent state is per-thread, but the runtime is work-stealing — “quiescent points in the actor loops” cannot be per-thread events

• kind: risk
• area: concurrency / modules / architecture
• severity: mid
• evidence:
  • ~/dev/fera/project-design/context/OQ.md:808 — “ractor-on-tokio work-stealing remains the backbone; blocking work … goes to dedicated OS threads via spawn_blocking”
  • ~/dev/fera/project-design/context/OQ.md:869-870 (OQ-024, Reclamation) — “Inter-module pointer tables = QSBR with quiescent points placed in the actor loops (our known safe points, not arbitrary Tokio tasks)”
  • ~/dev/fera/project-design/context/OQ.md:861-862 — “spawn_blocking with semaphore/limits for short blocking ops; named OS threads with stop protocols for long-lived blocking workers (disk IO, decode)”
  • ~/dev/fera/project-design/context/OQ.md:864 — “Configurable actor-pool sizes (N=1–2 power mode, N=cores desktop)”
  • ~/dev/fera/project-design/context/OQ.md:802-805 — EBR table “must be tuned for hundreds of modules, not a handful”
  • ~/dev/fera/project-design/context/ADR.md:17-18 (ADR-001) — “hot-swappable native modules (dynamic libraries) … versioned, C-ABI-stable trait contracts”
  • ~/dev/fera/fera/core/crates/domain/src/chat.rs:8-12 — “The ractor wiring (supervision tree, mailboxes) lands with the tokio runtime pass”

Substrate

This is a defect in a premise, not in code — there is no actor runtime yet (chat.rs:8-12 defers it). It is worth writing down now because QSBR is much harder to retrofit than to choose correctly, and because the corpus has already locked the choice (“no actor-model rewrite”).

QSBR’s core invariant is per-thread. In QSBR (Brown’s quiescent-state reclamation), every thread periodically publishes that it is quiescent; an object retired in epoch E may be freed once every thread has published quiescence since that retire. The inference the algorithm relies on is:

“thread T is quiescent” ⟹ “no reference into the retired set is live anywhere reachable from T”

That inference is sound for a thread-per-actor runtime. It is false for a work-stealing executor, which is what OQ-024 specifies (OQ.md:808). Under work stealing, a future is not pinned to a thread: it can be polled on worker A, .await, then resumed on worker B. The failure mode is specific:

1. Actor task T takes a reference into module M’s pointer table (an inter-module pointer, which OQ.md:869 says is exactly what QSBR guards).
2. T .awaits. It is now suspended, not running — but its reference is live.
3. Worker A, no longer running T, is idle between polls. A conventional per-thread quiescence poll sees “worker A has no critical section in progress” and reports quiescent.
4. A retires module M’s table entry. All threads have quiesced since the retire, so the reclaimer frees it.
5. Worker B resumes T, which dereferences freed memory.

Step 3 is the whole problem: an idle worker thread is not the same as a thread with no live references, because references can be parked in suspended tasks that belong to no thread at that instant. The corpus’s own phrasing makes this worse — “our known safe points, not arbitrary Tokio tasks” (OQ.md:869-870) asserts that quiescence is a property of actor loops. An actor loop is a task, and tasks are exactly the things that migrate.

Two aggravating factors already in the plan

Long-lived blocking threads never quiesce. OQ.md:861-862 plans “named OS threads with stop protocols for long-lived blocking workers (disk IO, decode)” and spawn_blocking for sanakirja work. These threads do not run actor loops, so under the stated scheme they never publish quiescence. If they hold any inter-module reference — a pointer table read that outlives the read call, or simply a long decode holding a module handle — reclamation for that module never completes. Across “hundreds of modules” (OQ.md:802-805) this is a monotonic memory growth path, not a stall.

The power mode changes the thread count at runtime. OQ.md:864 plans actor-pool sizes “N=1–2 power mode, N=cores desktop” — a runtime transition. Every epoch-based table is sized by participant count. Resizing the epoch table while modules are loaded and references are live is a correctness-relevant operation, and nothing in the corpus addresses the transition.

Options (tradeoffs)

1. Track quiescence per-task, not per-thread. A guard that is created on task entry and marks the task non-quiescent across every poll resumption; the reclaimer waits on task-liveness, not thread liveness. This is the correct general fix. Cost: a guard type that must be re-acquired on every poll — which fights the stated goal of “near-zero reader cost on the hot path” (OQ.md:797-798).
2. A global quiescence counter incremented at every task poll boundary. Cheaper than per-task guards, and sufficient for QSBR (one increment per poll means any suspended task has had its last resume preceded by an increment). Cost: an atomic on a very hot path; still cheaper than an epoch-table scan per retirement.
3. Drop to hazard pointers for inter-module pointers. HP does not have the per-thread problem — the publishing is the task. Cost: HP costs a store-load-load-fence sequence per access on the hot path, which is exactly what QSBR was chosen to avoid (OQ.md:797-798).
4. Pin actors to threads (thread-per-actor), making per-thread QSBR sound. Cost: this is precisely the “actor-model rewrite” OQ-024 forbids (OQ.md:860-861, “no actor-model rewrite”), and it forfeits work stealing, which is the property that makes idle cost low — a named goal.
5. Keep QSBR but make the blocking threads participants explicitly (each blocking worker gets an epoch slot and quiesces at task boundaries). This fixes the never-quiesce leak but not the work-stealing defect in step 3.

Proposed solution

• Correct the stated invariant in OQ-024 before it is implemented. Replace “quiescent points placed in the actor loops” with an explicit statement that quiescence is tracked per task / per poll boundary, not per OS thread, and say why (work stealing: OQ.md:808 + OQ.md:869 are incompatible as written). This is a one-paragraph documentation fix that prevents a wrong implementation.
• Adopt option 2 (global quiescence counter bumped at poll boundaries) as the baseline, with option 1 as the upgrade if measurements justify the cost. Both keep the low hot-path cost QSBR was selected for; the status quo does not have a working algorithm at all.
• Adopt option 5 as well, for the blocking workers — otherwise the never-quiesce leak is independent of the work-stealing fix.
• Address the power-mode transition explicitly: epoch tables are participant-count-sized, and “N=1–2 power mode, N=cores desktop” (OQ.md:864) is a runtime resize with live references. Either fix the table size for the maximum and never resize, or quiesce everything across the transition.
• Add a regression test that would have caught this: an actor that takes an inter-module reference, .awaits, has its worker observed as quiescent, then the module’s entry is retired — assert the entry is not freed. This is testable without a real work-stealing runtime by simulating a task migration between two worker threads.

Calibration note: severity is mid rather than high because nothing is implemented yet, and because hazard-pointers-cannot-allow-dylib-unmap.md is the more imminent failure. But this one is harder to detect later — the step-5 use-after-free is silent and load-dependent, whereas a timed out unmap segfaults loudly. Choose the reclamation scheme before ractor wiring starts, not after.