Shyamalan KannanandBohdan Triapitsyn
a9ee499ae4
* fix: add concurrency controls for multiple sessions using the same provider Adds OS-inspired scheduling primitives (from HiveMind/AIMD research) to prevent concurrent sessions from the same provider from experiencing slowdowns, random stops, and cascading failures. Server-side: - Health check skips OpenCode restart when sessions are actively busy — a busy server under concurrent load can fail the health check timeout without being dead. Staleness guard forces restart if unhealthy+busy persists >2 minutes. - Upstream SSE stall timeout scaled from 20s to 60s to avoid unnecessary reconnections when multiple sessions are waiting for LLM responses. Client-side (HiveMind primitives, arXiv:2604.17111): - Transparent retry with exponential backoff (1s→2s→4s, max 32s) for 429/502/503/504 errors — the #1 most effective primitive from the paper. - Circuit breaker: opens after 3 consecutive retryable errors, cooldown doubles each trip (30s→60s→120s, capped 128s), matching TCP AIMD. - Per-provider session tracking with TTL eviction (1h idle sweep). - Fetch-level retry gated on AbortError/TypeError only (not DNS failures). Refs github-code-review skill findings (all 8 issues resolved). * fix: use definite assignment assertion for response variable Fixes TS2454: Variable 'response' is used before being assigned in strict mode. The for-loop body always assigns it on every path that reaches the post-loop code, but TS can't prove that. * fix: add cleanupSession to error paths and remove unreachable code P1 fixes (Greptile review): - cleanupSession called on fetch error throw path - cleanupSession called on non-retryable HTTP error throw path - Removed unreachable post-loop code (loop always terminates via return or throw) Adds explicit post-loop throw to satisfy TypeScript strict return check. * fix: address Greptile review feedback on concurrent session controls Removes client-side session tracking that leaked on normal completion paths. The session tracking was redundant — the server-side health check already reads from sessionRuntime.getSessionActivitySnapshot() for busy-session detection. Changes: - Remove activeSessions Set and all session-tracking functions from provider-tracker - Remove trackSessionStarted/cleanupSession calls from client.ts - Remove unreachable (response as Response) block after retry loop - Make upstreamStallTimeoutMs conditional: 60s when >1 sessions, 20s otherwise Refs #1069 * fix: enforce dynamic concurrency safeguards --------- Co-authored-by: Bohdan Triapitsyn <artmore@protonmail.com>
26 lines
615 B
JavaScript
26 lines
615 B
JavaScript
export {
|
|
MESSAGE_STREAM_GLOBAL_WS_PATH,
|
|
MESSAGE_STREAM_DIRECTORY_WS_PATH,
|
|
MESSAGE_STREAM_WS_HEARTBEAT_INTERVAL_MS,
|
|
parseSseEventEnvelope,
|
|
sendMessageStreamWsFrame,
|
|
sendMessageStreamWsEvent,
|
|
} from './protocol.js';
|
|
|
|
export {
|
|
createGlobalUiEventBroadcaster,
|
|
createMessageStreamWsRuntime,
|
|
} from './runtime.js';
|
|
|
|
export {
|
|
MESSAGE_STREAM_GLOBAL_REPLAY_LIMIT,
|
|
createGlobalMessageStreamHub,
|
|
} from './global-hub.js';
|
|
|
|
export {
|
|
DEFAULT_UPSTREAM_RECONNECT_DELAY_MS,
|
|
DEFAULT_UPSTREAM_STALL_TIMEOUT_MS,
|
|
UPSTREAM_STALL_TIMEOUT_CONCURRENT_MS,
|
|
createUpstreamSseReader,
|
|
} from './upstream-reader.js';
|