ADR-031 contradicts itself on the main search index: content tier vs plaintext-at-rest

Источник: dev/fera-research/docs/oc1/004-adr031-self-contradiction-search-index.md (read-only; сгенерировано scripts/sync-docs.mjs)

ADR-031 contradicts itself on the main search index: content tier vs plaintext-at-rest

• kind: problem
• area: storage / trust
• evidence — both halves inside /home/nikki/dev/fera/project-design/context/ADR.md:
  • ADR.md:653-655 — “Content tier (XChaCha20-Poly1305): message text + entities, media blobs + captions, profile contact info / bio, search-index terms.” Rationale given: “the milli fork indexes plaintext, so terms inherit the content classification”.
  • ADR.md:675-677 — “The main index stays plaintext-at-rest with disk-encryption by default (speed); the same searchable-encryption machinery may be offered as a ‘paranoid mode’ option for the main instance later.”
  • Same table, ADR.md:650-652: chat-list entries (titles) and the predictor event log are metadata tier; predictor events are metadata, so chat-open behaviour is plaintext-adjacent too.
  • TECHSPEC never mentions either mode — TECHSPEC.md:154-164 states the two tiers and stops, inheriting the contradiction.
  • Nothing anywhere defines “disk-encryption”: no ADR, no OQ, no requirement that sanakirja stores be encrypted at rest (full-disk encryption is an OS feature, not something the app can rely on).
• severity: high

Суть / Substance

ADR-031 is the field audit for the whole encryption tiering, and QA gates on it (ADR.md:683-686: “New persisted fields require an audit-table entry before merge”). It cannot be the reference if it lists the same field in two tiers 20 lines apart.

The two readings have opposite security properties:

• Reading A (“search-index terms = content tier”): every indexed word of every message is encrypted under the passcode-derived key. Search therefore requires unlocking, and — critically — full-text search is impossible while locked. That is a real product regression: Telegram clients let you search your history without unlocking, and Fera’s own benchmark/QA stories assume the search path works.
• Reading B (“main index plaintext-at-rest”): search works locked, but an attacker with read access to the sanakirja store (stolen laptop, backup, a malicious plugin if any storage access is ever granted, or the file being copied into a support bundle) recovers every message term even though the surrounding message bodies are XChaCha20-encrypted and the UI is presumably showing a lock screen. Encryption at rest is then decorative for the highest-value field in the store: the plaintext is sitting in the index next to the ciphertext.

Reading B is what the prose says last and what is operationally practical; Reading A is what the audit table says. Nobody chose. And “with disk-encryption” is a phantom mitigation: the app cannot assert that the user’s disk is encrypted, and if they use LUKS/BitLocker the index is already protected — in which case Reading B’s own justification (“speed”) buys nothing against a threat model that assumes FDE anyway.

A third, unstated option exists and is probably the right answer: deterministic HMAC tags for the main index too (exactly the mechanism ADR-031 already specifies for secret chats at ADR.md:666-670), giving locked search and no plaintext terms, at the cost of the accepted frequency-analysis weakness — which ADR-031:670-674 already classifies as “a hardening direction, not a launch blocker”. The corpus built the machine and then didn’t use it for the main index.

Варианты решения (tradeoffs)

• (A) Deterministic HMAC tags for the main index as well (extend the secret-chat scheme). Locked search preserved, no plaintext terms. Costs: frequency analysis on the whole corpus (not just secret chats), no prefix/fuzzy/regex-over-index paths without extra structures, multi-word AND-queries tag fine but phrase queries degrade, and the index can no longer be rebuilt from plaintext offline.
• (B) Keep the index plaintext, amend the table (make “search-index terms = plaintext-at-rest” the recorded decision, documented as an explicit accepted trade-off with the threat model stated: “protects against someone reading message bodies; does not protect against someone reading the index”). Cheap, honest, but the threat model is weak.
• (C) Per-account/per-chat subtree encryption with an unlocked-key ring, i.e. real tier separation only for chats marked private. Big product surface nobody specced.
• (D) Drop full-text search while locked; require unlock. Simplest security story, worst UX, and contradicts OQ.md’s two-phase search design which assumes the index is queryable.

Предлагаемое решение

(B) as the recorded decision, with (A) as the documented v1.5 upgrade path — and amend the audit table in the same pass.

1. Fix ADR.md:653-655: move search-index terms out of “content tier” into a third, explicitly named tier — “index tier: plaintext at rest, optional HMAC-tag mode” — so the table stops lying and the choice becomes visible to QA’s per-field gate.
2. State the threat model in one sentence, because that is what the choice actually turns on: this protects message bodies against a stolen store; it does not protect search terms against a stolen store.
3. Delete “with disk-encryption” or replace with “when the platform provides full-disk encryption” — the app must not claim disk encryption it does not perform.
4. Record the frequency-analysis consequence as applying to the main index too once (A) lands, so the “hardening direction” note at ADR.md:670-674 stops being scoped to secret chats only.
5. Block on: does anyone actually want locked search? That is a product question the corpus has never asked, and it is the whole decision. File it as an OQ before the audit table is re-frozen.