perf: fix streaming lag, memory leaks, stuck spinners, and proxy timeout (#483)

* perf: fix streaming lag, memory leaks, and proxy timeout

- PERF-001: Batch all streaming parts via requestAnimationFrame instead of
  per-token Zustand set() calls (~100/sec → 1 per frame)
- PERF-002: Fix direct state.sessionMemoryState mutation inside set() callback
- PERF-003: Debounce messageStore→sessionStore subscription via rAF + 500ms
  title computation delay
- PERF-004: Stabilize SSE callbacks with refs to prevent reconnection storms;
  add 5-min stuck session idle timeout
- PERF-005: Bound messageCache (500 max, LRU eviction), cap registry Maps,
  cleanup on session eviction
- PERF-006: Replace toast duration: Infinity with 30s + id-based dedup
- Fix proxy timeout: POST /session/:id/message 45s → 4min (matches CLI)
- Add vitest + jsdom test infrastructure (61 tests across 7 files)

Addresses: #476 (stuck spinner), #358 (34GB memory leak), #190 (browser lag)

* perf: virtualize tool output rendering (read, edit, write)

- PERF-007: Replace per-line <SyntaxHighlighter> with VirtualizedCodeBlock:
  - ONE Prism.highlight() call for entire file instead of N per-line calls
  - @tanstack/react-virtual renders only visible rows (~30 vs 2000+)
  - Applied to: ToolPart (read, DiffPreview, WriteInputPreview) and
    ToolOutputDialog (unified diff, read content)
- PERF-008: Memoize parseReadToolOutput/parseDiffToUnified via useMemo
  to prevent re-parsing on every re-render

A 2000-line file read now mounts ~30 DOM nodes instead of 2000
SyntaxHighlighter instances, eliminating the main-thread blocking
that caused UI freezes during file operations.

* fix: resolve lint errors (unused vars in tests and VirtualizedCodeBlock)

* chore: trim PR scope to core perf fixes

* fix: restore tool-card highlight stability

---------

Co-authored-by: Bohdan Triapitsyn <artmore@protonmail.com>
This commit is contained in:
Nguyễn Ngô Thượng
2026-02-23 12:05:53 +02:00
committed by GitHub
co-authored by Bohdan Triapitsyn
parent 989593ed72
commit 3cd6d051cb
11 changed files with 756 additions and 407 deletions
+69 -40
View File
@@ -851,6 +851,10 @@ export const useSessionStore = create<SessionStore>()(
),
);
// rAF debounce IDs for useMessageStore -> useSessionStore sync
let messageStoreSyncRafId: number | null = null;
let userSummaryTitlesRafId: ReturnType<typeof setTimeout> | null = null;
useSessionManagementStore.subscribe((state, prevState) => {
if (
@@ -885,7 +889,8 @@ useSessionManagementStore.subscribe((state, prevState) => {
});
useMessageStore.subscribe((state, prevState) => {
// Early-return equality check stays outside the rAF so we skip scheduling
// entirely when nothing relevant changed.
if (
state.messages === prevState.messages &&
state.sessionMemoryState === prevState.sessionMemoryState &&
@@ -900,48 +905,72 @@ useMessageStore.subscribe((state, prevState) => {
return;
}
const userSummaryTitles = new Map<string, { title: string; createdAt: number | null }>();
state.messages.forEach((messageList, sessionId) => {
if (!Array.isArray(messageList) || messageList.length === 0) {
return;
// Debounce the expensive sessionStore update to at most once per animation
// frame. Multiple messageStore updates within the same frame (e.g. several
// SSE tokens arriving before the next paint) collapse into a single setState.
if (messageStoreSyncRafId !== null) {
cancelAnimationFrame(messageStoreSyncRafId);
}
messageStoreSyncRafId = requestAnimationFrame(() => {
messageStoreSyncRafId = null;
// Read the LATEST state at flush time, not the stale state captured by
// the subscription closure.
const latest = useMessageStore.getState();
useSessionStore.setState({
messages: latest.messages,
sessionMemoryState: latest.sessionMemoryState,
messageStreamStates: latest.messageStreamStates,
sessionCompactionUntil: latest.sessionCompactionUntil,
sessionAbortFlags: latest.sessionAbortFlags,
streamingMessageIds: latest.streamingMessageIds,
abortControllers: latest.abortControllers,
lastUsedProvider: latest.lastUsedProvider,
isSyncing: latest.isSyncing,
});
// Sidebar titles don't need real-time updates; debounce separately at
// 500 ms so the expensive per-message iteration doesn't happen every
// frame during streaming.
if (userSummaryTitlesRafId !== null) {
clearTimeout(userSummaryTitlesRafId);
}
for (let index = messageList.length - 1; index >= 0; index -= 1) {
const entry = messageList[index];
if (!entry || !entry.info) {
continue;
}
const info = entry.info as Message & {
summary?: { title?: string | null } | null;
time?: { created?: number | null };
};
if (info.role === "user") {
const title = info.summary?.title;
if (typeof title === "string") {
const trimmed = title.trim();
if (trimmed.length > 0) {
const createdAt =
info.time && typeof info.time.created === "number"
? info.time.created
: null;
userSummaryTitles.set(sessionId, { title: trimmed, createdAt });
break;
userSummaryTitlesRafId = setTimeout(() => {
userSummaryTitlesRafId = null;
const titleState = useMessageStore.getState();
const userSummaryTitles = new Map<string, { title: string; createdAt: number | null }>();
titleState.messages.forEach((messageList, sessionId) => {
if (!Array.isArray(messageList) || messageList.length === 0) {
return;
}
for (let index = messageList.length - 1; index >= 0; index -= 1) {
const entry = messageList[index];
if (!entry || !entry.info) {
continue;
}
const info = entry.info as Message & {
summary?: { title?: string | null } | null;
time?: { created?: number | null };
};
if (info.role === "user") {
const title = info.summary?.title;
if (typeof title === "string") {
const trimmed = title.trim();
if (trimmed.length > 0) {
const createdAt =
info.time && typeof info.time.created === "number"
? info.time.created
: null;
userSummaryTitles.set(sessionId, { title: trimmed, createdAt });
break;
}
}
}
}
}
}
});
useSessionStore.setState({
messages: state.messages,
sessionMemoryState: state.sessionMemoryState,
messageStreamStates: state.messageStreamStates,
sessionCompactionUntil: state.sessionCompactionUntil,
sessionAbortFlags: state.sessionAbortFlags,
streamingMessageIds: state.streamingMessageIds,
abortControllers: state.abortControllers,
lastUsedProvider: state.lastUsedProvider,
isSyncing: state.isSyncing,
userSummaryTitles,
});
useSessionStore.setState({ userSummaryTitles });
}, 500);
});
});