fix(ui): correct markdown cache identity, streaming churn, and redundant tiers
Review follow-up on the #2769 highlight caches. Fingerprint strength. The block/highlight caches are now global and content-addressed, so a hash collision no longer mis-colors a block — it returns a *different* block's rendered HTML and shows the user source they never wrote. Length + one 32-bit FNV-1a is not enough key space for that failure mode at session scale. `contentFingerprint` now combines two independent 32-bit multiplicative hashes with a final avalanche (~64 bits); two multiplies per character are free next to Shiki tokenization. Streaming churn. Content addressing made every streaming step of the trailing `live` block insert a new cache entry, so one long message evicted the settled `full` blocks the fix exists to keep warm. `full` and `live` blocks now use separate caches; the live cache is small (32 entries / 2MB) because it only has to absorb repeat renders of the same step. Redundant worker-side caches. `markdown-worker.ts` is the only sender to the Shiki worker, and its client cache is larger than the worker-side ones, so the worker caches could not serve a hit the client had not already served — they only duplicated up to 48MB of payloads in a second heap. Removed; the reason memoization belongs on the client is now documented there, along with why only `highlightTokens` carries a theme in its key. Dead `cacheKey` plumbing. `renderMarkdownBlocks` kept a `cacheKey` parameter it only `void`-ed. Removed it and the now-unused `useMorphdomMarkdown` prop; the remaining call-site local is renamed `fadeKey` for what it actually keys. Tests: image-mode cache identity, streaming-does-not-evict-settled-blocks, live-cache reuse, and a 20k same-length-source fingerprint collision check. Each new guard was verified to fail without its fix.
This commit is contained in:
@@ -18,6 +18,15 @@ import type { MarkdownTokenRun, MarkdownWorkerRequest, MarkdownWorkerResponse }
|
||||
// re-highlight load in openchamber/openchamber#2769. In-flight requests with
|
||||
// the same key coalesce so remount storms share one round-trip. Cache keys are
|
||||
// fingerprints (not full source) so large files are not duplicated in the Map.
|
||||
//
|
||||
// This module is the only sender to the worker, so memoizing here is sufficient
|
||||
// and the worker itself stays stateless apart from the Shiki instance. A second
|
||||
// cache inside the worker would only duplicate these payloads in another heap.
|
||||
//
|
||||
// `highlight` / `highlightLines` results are theme-independent: the worker
|
||||
// tokenizes with the CSS-variable `MARKDOWN_SHIKI_THEME`, so a theme switch
|
||||
// repaints via CSS and must not invalidate these entries. Only
|
||||
// `highlightTokens` resolves concrete colors, so only its key carries a theme.
|
||||
|
||||
type PendingResolver = (response: MarkdownWorkerResponse | null) => void;
|
||||
|
||||
|
||||
Reference in New Issue
Block a user