Storage seam is frozen in the wrong shape, and the ADR-031 tier audit leaves the host-side LRU cache and outbox plaintext unassigned

Источник: dev/fera-research/docs/oc5/storage-seam-frozen-wrong-shape.md (read-only; сгенерировано scripts/sync-docs.mjs)

Storage seam is frozen in the wrong shape, and the ADR-031 tier audit leaves the host-side LRU cache and outbox plaintext unassigned

• kind: risk
• area: storage / encryption
• severity: high
• evidence:
  • ~/dev/fera/fera/core/crates/storage/src/engine.rs:27-39 — trait as implemented: -> Result<Vec<Message>, Self::Error> and put_messages(&mut self, chat: ChatId, msgs: &[Message])
  • ~/dev/fera/project-design/context/TECHSPEC.md:136-139 — trait as specified: fn get_messages(...) -> Vec<Encrypted<Message>>; / fn put_messages(&mut self, chat_id: ChatId, msgs: &[Encrypted<Message>]);
  • ~/dev/fera/project-design/context/TECHSPEC.md:154-162 — the Encrypted<T>/Plaintext<T> phantom boundary
  • ~/dev/fera/project-design/context/ADR.md:604-606 (ADR-015), :616-635 (ADR-016) — “enforces at compile time that content can’t be persisted without going through the appropriate tier’s encryption path”
  • ~/dev/fera/project-design/context/TECHSPEC.md:166-168 — “Host-side cache: LRU in front of StorageEngine … invalidated on write-through from Chat actor”
  • ~/dev/fera/project-design/context/ADR.md:130-132 (ADR-005) — “A host-side (in-process) LRU cache of recently-viewed messages sits in [front]… Only cache misses issue a [boundary call]”
  • ~/dev/fera/project-design/context/ADR.md:648-660 (ADR-031) — the per-field tier audit that QA.md makes mandatory (“New persisted fields require an audit-table entry before merge”)
  • ~/dev/fera/fera/core/crates/domain/src/chat.rs:105,109 — messages: Vec<Message> and outbox: Vec<Message>
  • ~/dev/fera/fera/core/crates/storage/src/memory.rs:20,25 — same
  • ~/dev/fera/fera/core/crates/storage/src/Cargo.toml:12-13 — sole dependency is fera-protocol; no crypto crate anywhere in the ecosystem

Substrate

There is no encryption implemented at any tier today. That is disclosed and sequenced (storage/src/lib.rs:5-8: “encryption lands in the crypto pass (TECHSPEC §10 step 6), the seam for it is kept open”). Disclosed roadmap gaps are not findings. Two things beyond that are.

1. The seam is not open — it is frozen in the wrong shape

storage/src/lib.rs says the Encrypted<T> seam “is kept open”. It is not. TECHSPEC §3 specifies the trait taking Encrypted<Message>, and the implemented trait takes bare Message:

// TECHSPEC.md:138-139 (specified)
fn get_messages(...) -> Vec<Encrypted<Message>>;
fn put_messages(&mut self, chat_id: ChatId, msgs: &[Encrypted<Message]>);

// engine.rs:32,39 (implemented)
fn get_messages(&self, ...) -> Result<Vec<Message>, Self::Error>;
fn put_messages(&mut self, chat: ChatId, msgs: &[Message]) -> Result<(), Self::Error>;

Every StorageEngine impl written now — MemoryEngine today, the sanakirja engine next — hardcodes the plaintext signature. When crypto lands, the trait changes and every impl, caller, and test changes with it. Worse, it changes in the direction that looks safe: the natural migration is to add encryption inside each impl, which leaves Message flowing freely across the trust boundary and quietly forfeits the compile-time guarantee ADR-015/ADR-016 were adopted to obtain. The phantom-type boundary’s entire value is that it is enforced by the compiler at the trait signature — a property that exists only until the first non-conforming impl is written.

The cheap fix is available now (change the trait signature while it has one impl) and becomes expensive after the sanakirja engine lands.

2. ADR-031’s field audit — the mandatory QA gate — omits two plaintext stores

ADR-031 is the reference table, and ADR.md:683-686 makes it binding: “New persisted fields require an audit-table entry before merge”. Grepping the audit (§648-660) for cache / lru returns nothing, yet:

• TECHSPEC.md:166-168 mandates a host-side LRU cache “in front of StorageEngine” holding recently-viewed messages.
• ADR-005:130-132 repeats it: “A host-side (in-process) LRU cache of recently-viewed messages sits in [front of the boundary]”.

So the design has a mandatory, in-process, plaintext message cache that the mandatory per-field tier audit never classifies. The audit enumerates storage indexes as metadata tier but not the cache that feeds them.

This matters more than a normal omission because of where the cache sits. Every tier decision in ADR-031 is a statement about data at rest. The cache is data in process memory, decrypted, and by ADR-005 it exists specifically so that boundary calls can be skipped. So the actual exposure surface — core dumps, hibernation/swap files, a process-memory scrape, a screenshot of a locked machine — is entirely outside the taxonomy that the whole encryption design is built on. ADR-031 assigns every field to “metadata tier (lightweight cipher)” or “content tier (XChaCha20-Poly1305)”; neither phrase means anything for a heap buffer that was decrypted to avoid a syscall.

Note ADR-031’s own framing: it is a field audit, and it correctly notes the main search index stays “plaintext-at-rest-with-disk-encryption by default (speed)”. The cache is the same trade — speed, plaintext — but it never got written down, so it cannot be reviewed or defended.

3. The outbox is also unclassified, and holds plaintext indefinitely

ADR-031:650-651 puts “outbox status + retry info” in the metadata tier. The status and retry counters are not the message. An outbox entry is a full Message (domain/chat.rs:109 — outbox: Vec<Message>), and per ADR-010 it is persisted. Under a literal reading of the audit, an unsent message body would be classified as metadata-tier (cheap cipher) because the row is an outbox row. Message text in the content tier is supposed to get Argon2id-derived XChaCha20-Poly1305.

Worse, parked entries are never evicted: domain/src/outbox.rs:62-66 returns None (“parked”) once attempts >= max_attempts, but the row is retained indefinitely with status = Failed. There is no TTL and no eviction path in storage/src/memory.rs. So plaintext bodies accumulate without bound in a store the audit believes it has classified.

4. Searchable encryption is accepted with a known dictionary attack

ADR-031:665-674 adopts deterministic H = HMAC(key_search, term) and records the limitation honestly: “frequency analysis (‘the most frequent tag is the’) — dictionary attacks on the tag space. Mitigations (salted pairs) exist but add complexity; treat as a hardening direction, not a launch blocker.”

Unsalted deterministic HMAC over a term is not merely frequency-analysable — it is fully dictionary-attackable offline by anyone holding the index: enumerate a wordlist, HMAC each term under key_search, look up the tags. There is no secret to protect the attack, only rate. Real deployable hardening is cheap and standard (per-chat salt stored unencrypted, or HMAC(key, salt || term), or a length-preserving substitution with a secret prefix). Declaring the insecure variant shippable-by-default for secret chats — the feature whose entire value proposition is confidentiality — is a risk acceptance that deserves an explicit decision rather than a footnote deferral.

Options (tradeoffs)

1. Freeze the trait on Encrypted<T> now. One impl, no users, one commit. Preserves ADR-015/016’s compile-time guarantee at the moment it is still free. Cost: blocks the sanakirja engine until a placeholder Encrypted constructor exists (can be a #[cfg(test)] passthrough plus a sealed marker).
2. Land storage plaintext, then retrofit. Cost: every impl and caller churns; high risk the retrofit takes the “encrypt inside the impl” shortcut and silently drops the type-level guarantee. This is the path the code is currently on.
3. Classify the LRU cache explicitly — new ADR-031 rows for “host-side cache entries (in-memory)” with an explicit decision: accept plaintext-in-memory, or mlock/no-swap/no-core guidance.
4. Salted searchable encryption now — HMAC(key_search, chat_salt || term) with chat_salt random per chat, stored unencrypted (it is an index, not a secret). Removes both frequency analysis and cross-chat term correlation. ~10 lines.

Proposed solution

• Do option 1 immediately (before the sanakirja engine). Change StorageEngine::{get_messages,put_messages} to Encrypted<Message> per TECHSPEC §3, with a single test-only constructor. If that is genuinely blocked, record an explicit ADR amendment saying the compile-time guarantee is deferred — do not leave the doc and the signature silently disagreeing.
• Add ADR-031 rows for the host-side LRU cache and for outbox entries, and state for the cache that the tier taxonomy is a at-rest taxonomy which does not cover process memory — then either accept it explicitly or add mlockall/swap-off guidance. Right now the one store that is guaranteed to hold plaintext is the one with no classification.
• Make outbox bodies content-tier explicitly, and add a retention policy for parked entries (TTL + eviction) so plaintext bodies do not accumulate forever.
• Take option 4 for searchable encryption — the unsalted form is dictionary-attackable, which is stronger than “frequency analysis” suggests, and the mitigation is genuinely small.
• Note the dependency-ordering consequence: MemoryEngine is the only StorageEngine impl today, so the trait-change cost is one file. That is the cheapest it will ever be.