perf: overhaul session loading, caching, and runtime isolation (#2360)

Improve OpenChamber responsiveness under large session workloads while fixing
cache, synchronization, and persistence correctness across runtimes, projects,
directories, and worktrees.

- prioritize selected and visible sessions during bootstrap and defer
  non-critical enrichment work
- reduce redundant message loading, event processing, store publication, and
  hidden sidebar work
- prevent stale session and message requests from overwriting newer
  authoritative state
- preserve existing data when authoritative fetches fail instead of treating
  failures as successful empty responses
- scope session materialization, messages, drafts, queues, todos, pins,
  permissions, folders, tabs, Git state, and pull request data by runtime and
  directory identity
- harden runtime switching, reconnect, cleanup, mutation reconciliation, and
  persisted-state ordering
- preserve live subagent Task linkage when metadata arrives after an older
  message request or while streaming parts are suspended
- coalesce overlapping tail refreshes without losing newer refresh demand
- improve cold-session loading by moving deferrable work out of the critical
  bootstrap path
- isolate URL authentication, mobile credentials, native secrets, and other
  runtime-owned state across endpoint changes
- bound long-lived caches and remove avoidable allocations from event and
  rendering hot paths
- limit virtualization to archive collections where it improves rendering
  without disrupting active sidebar layout
- stabilize session folders, pin ordering, expanded state, and persisted
  sidebar behavior
- open skill files through the same secure editor and outside-workspace grant
  flow used by file navigation, including worktree sessions
- expand regression coverage for stale completions, runtime collisions,
  reconnect behavior, persistence races, authoritative empty results, and
  subagent refresh ordering
- document the updated synchronization, cache ownership, performance, and
  runtime-isolation invariants

.agents/skills/openchamber-change-discipline/SKILL.md

@@ -82,7 +82,7 @@ Use `package.json` scripts as the command source of truth.
 | Executable source | Focused tests plus package-scoped type-check and lint |
 | Cross-workspace/shared contract | Workspace-wide type-check and lint plus affected builds/tests |
 | Added/deleted/renamed source file, export/type/entrypoint/import shape | `bun run dead-code` in addition to relevant checks |
-| Persisted or external contract | Compatibility and round-trip tests; conversion/malformed-old-data tests when old data needs migration; failed-write/migration rollback tests |
+| Persisted or external contract | Compatibility and round-trip tests plus the applicable failure/ordering cases: missing-versus-empty, malformed data, stale reads versus newer mutations, out-of-order writes, lifecycle handling for debounced writes, conversion, and failed-write/migration rollback |
 | Dependency or lockfile | Workspace-wide checks and affected builds |
 | Generated asset | Regeneration check plus consumer build/test |
 | Docs-only or isolated config | Narrow syntax/schema/link validation; do not run unrelated full suites |

.agents/skills/performance-engineering/SKILL.md

@@ -37,6 +37,8 @@ Do not optimize against a toy fixture when the report provides production scale.
 
 Do not infer a bottleneck from code appearance when a trace or counter can identify it.
 
+Profiling identifies where time is spent; it does not prove behavioral equivalence. Separately verify the applicable state, identity, layout, and lifecycle transitions for every structural optimization.
+
 ### 2. Write The Cost Equation
 
 Name every multiplying dimension:
@@ -109,6 +111,9 @@ Prefer indexes keyed by stable IDs. Keep high-frequency runtime state out of met
 - Preserve references for unaffected entities and buckets.
 - Keep streaming state out of broadly consumed stores.
 - Never rely on `React.memo`, `useMemo`, or Zustand equality to prevent selector execution upstream.
+- Treat every custom memo/equality comparator as a correctness boundary. Inventory every render-relevant value that comparator gates and observe its canonical identity or an explicit semantic version covering the same semantics.
+- Do not compare a proxy, aggregate, fallback, or differently resolved identity when the gated render path uses another source. Stable entity IDs do not imply stable rendered content; changes to comparator-gated semantics under the same ID must invalidate affected consumers, while semantically equivalent replacements may remain stable.
+- Prefer leaf subscriptions for isolated high-frequency state over threading broad state through custom comparators. Keep comparator work bounded so render fanout is not merely replaced by recursive comparison fanout.
 - Do not sort structural lists from token/delta-frequency fields.
 - Coalesce repeated same-entity events and skip no-op reducer updates.
 - Ensure hidden or disabled surfaces perform no ongoing work.
@@ -117,6 +122,20 @@ Prefer indexes keyed by stable IDs. Keep high-frequency runtime state out of met
 - Avoid textarea auto-size shrink/expand cycles when content only grows.
 - Freeze structural ordering during high-frequency updates and reorder at an explicit lifecycle edge.
 
+## Virtualization Contracts
+
+Virtualization changes layout, mounting, measurement, focus, and scroll semantics. It is not behaviorally equivalent merely because steady-state visible rows look the same.
+
+Before virtualizing a collection, define:
+
+- the actual scrolling element and whether it directly contains the virtualizer or is an ancestor;
+- how total virtual height and the final item remain reachable from that scroller;
+- estimated versus measured sizes, including expanded, nested, and dynamically resized items;
+- initialization, remount, and activation-threshold behavior;
+- interactions that depend on mounted DOM, including incremental reveal, focus, selection, drag-and-drop, menus, and accessibility traversal.
+
+When activation is threshold-based, test threshold minus one, threshold, and threshold plus one. Also test applicable collapsed/expanded, hidden/visible, filtered/unfiltered, and short/long transitions. If the current DOM or scroll topology cannot expose the virtual tail reliably, correct that topology or retain normal rendering rather than virtualizing solely by item count.
+
 ## Caching Rules
 
 Add a cache only when all are explicit:
@@ -141,6 +160,9 @@ Require both correctness and performance guards:
 - repeated-event test for streaming/polling paths;
 - no-op and unrelated-entity update tests;
 - reference-stability test for unaffected buckets;
+- when custom comparators change, tests proving both directions: unrelated or semantically equivalent updates preserve the boundary, while changes to comparator-gated identity, membership, content, and source semantics invalidate it;
+- when memoized tree/list consumers change, same-ID replacements and rebuilt-container fixtures covering both semantic change and semantic equivalence;
+- when virtualization changes, tests using the real scrolling ancestor that prove final-item/control reachability and stable scroll, focus, and interactions; include activation-boundary cases when such a boundary exists;
 - failure, partial-data, empty-success, and stale-async-completion tests;
 - memory/cache growth check for long-running paths;
 - production build or equivalent runtime profile for UI interactions.
@@ -181,4 +203,6 @@ If the interaction remains above budget, do not call the mitigation the complete
 - [ ] Partial failure cannot trigger destructive cleanup.
 - [ ] Representative benchmark meets the stated budget.
 - [ ] Operation-count or repeated-event regression test prevents recurrence.
+- [ ] Structural optimizations have transition-focused correctness coverage independent of performance measurements.
+- [ ] When mount topology or activation boundaries change, instrumentation distinguishes those transitions from steady state.
 - [ ] Correctness, type, lint, and relevant runtime validations pass.

.agents/skills/sync-state-invariants/SKILL.md

@@ -35,6 +35,14 @@ Never swallow an SDK/API error into `[]`, `{}`, or another valid empty success.
 
 Track completeness at the smallest entity/scope. One failed project or directory blocks destructive work for itself, not for unrelated complete scopes.
 
+Inferring destructive cleanup from disappearance between snapshots requires an established authoritative baseline. This is separate from applying a complete snapshot whose contract explicitly authorizes first-load replacement.
+
+- Never infer a disappearance event from the first snapshot, startup-empty state, filtered/visible subsets, or partially loaded scopes.
+- Compare two complete authoritative snapshots from the same runtime and logical scope before treating disappearance as removal.
+- Key disappearance by stable entity identity. Owner, directory, grouping, category, or presentation moves are not deletion unless the authoritative contract says so.
+- Reset the baseline when runtime identity or authoritative scope changes.
+- Prefer explicit deletion events; snapshot-difference cleanup is a fallback that requires completeness guarantees.
+
 ## Live And Historical State
 
 - Use historical state to restore context, not to infer ongoing execution.
@@ -61,6 +69,10 @@ For streaming-frequency work, also load `performance-engineering`.
 - Treat startup 502/503 as transient with bounded retry/recovery.
 - A retry loop requires a real failure signal; swallowed errors disable retries.
 - Preserve previous authoritative state during transient bootstrap/reconnect failures.
+- Distinguish stale-scope rejection from same-scope mutation reconciliation. A generation token rejects obsolete owners but does not protect mutations made while a still-valid request is in flight.
+- Capture a mutation revision when an authoritative load starts. At commit time, read current state and preserve or overlay entity mutations newer than that revision.
+- Record removals as mutations even when the entity is already absent, so an in-flight response cannot resurrect it.
+- Return committed reconciled state, not the raw fetched snapshot, when callers depend on the result.
 
 ## Optimistic Updates
 
@@ -85,6 +97,18 @@ For streaming-frequency work, also load `performance-engineering`.
 - Key runtime-scoped caches by runtime identity when IDs or paths can collide.
 - Clean optimistic and local cache state after partial failures.
 
+## Persisted Snapshot Ordering
+
+When state exists in memory and one or more persistent stores, define an explicit authority and ordering protocol:
+
+- Distinguish a missing snapshot from authoritative empty data, malformed data, and read failure.
+- Preserve mutation order independently per owner by serializing writes or attaching monotonic revisions and rejecting stale writes. Do not rely on uncontrolled wall-clock timestamps.
+- Capture runtime/owner identity with every debounced or asynchronous operation and verify it again before commit.
+- Pending writes must complete against their captured owner, drain before an owner switch, or be canceled only under an explicit durability/data-loss contract. Apply the strongest available guarantee at page hide/freeze and shutdown boundaries.
+- During hydration, capture the local mutation revision and do not replace state after newer local mutations.
+- Validate persisted payload shape before granting authority. Malformed data is failure, not empty success.
+- Define retention explicitly; never silently evict older owner namespaces unless bounded retention and resulting data loss are intentional contracts.
+
 ## Verification
 
 Cover the relevant lifecycle, not only static state:
@@ -97,6 +121,10 @@ Cover the relevant lifecycle, not only static state:
 - create, stream, abort, permission, archive/delete, and revisit when session behavior changes;
 - partial multi-directory/project failure;
 - runtime or worktree switch with dynamic directory resolution.
+- snapshot-difference cleanup establishing its first authoritative baseline without deletion, then cleaning a later authoritative disappearance exactly once;
+- identity-preserving moves/category changes and runtime/scope changes resetting cleanup baselines;
+- create, update, move, archive, and delete mutations surviving responses started before those mutations;
+- missing versus empty persistence, malformed payloads, out-of-order writes, hydration races, and lifecycle durability behavior.
 
 ## Red Flags
 
@@ -107,3 +135,6 @@ Cover the relevant lifecycle, not only static state:
 - Queue reads current model/agent at send time.
 - New session lookup assumes SSE already indexed it.
 - Optimistic data has no shadow entry or rollback.
+- Snapshot-difference cleanup treats its first startup snapshot as a disappearance event.
+- Missing or malformed persistence becomes authoritative empty state.
+- Debounced writes are canceled on owner/lifecycle change without completing against the captured owner or an explicit durability/data-loss contract.