Bohdan Triapitsyn
28f0736d69
Adds a server-side "small model" capability: direct, cheap LLM calls that reuse the user's existing OpenCode provider logins — the mechanism OpenCode uses internally for titles and summaries but does not expose through the SDK or plugins. Zero new dependencies; plain fetch with per-provider wire formats, credentials never leave the server. Core (packages/web/server/lib/small-model): - Resolution mirrors OpenCode's session scoping: explicit settings override → small_model from the OpenCode config → family scan within the session's provider → the session's own model. The global provider scan only serves callers without a session context, and background callers forbid it entirely (restrictToPreferredProvider), so conversation content never reaches a provider the user didn't pick — explicit choices excepted. - Per-provider auth replicating OpenCode's plugin loaders: GitHub Copilot (device token as bearer, no exchange), ChatGPT plan via the codex Responses API (single-flight OAuth refresh written back to auth.json), Anthropic messages, Google generateContent, generic OpenAI-compatible. - OpenCode's free models (opencode/big-pickle, *-free) are never called directly; unauthenticated providers are skipped by design. - Prompt clamping to the model's catalog context limit; thinking disabled where a wire switch exists (Z.AI/GLM, MiniMax-M3, Gemini Flash); robust content parsing with a clear error when a thinking model spends its whol budget on reasoning. - Settings → Sessions gains a Small Model group: use-default checkbox plus an override picker limited to authenticated providers, persisted with web/desktop/VS Code sanitization parity. Consumers: - Session assist: a server-side watcher on the global SSE hub generates a short recap and one suggested follow-up after a session idles quietly fo a minute, stored on session metadata (openchamber.assist). Freshness is keyed to the last assistant message id, so new activity invalidates the payload everywhere with no extra writes. The chat shows the recap under the last message after five quiet minutes and the suggestion as a dismissible chip above the composer (tap fills the input, never sends). Gated by a new Chat setting (default on) that is a hard generation switch. Language is anchored to the conversation itself, with a script-mismatch guard against model/backend language hallucination. - TTS: a third input mode, summarized — long replies are condensed to spoken prose before playback on any TTS engine. - Git: commit-message and PR generation moved off the active chat session onto the small model fed with real diffs and the commit list (bodies included), with a session-transport fallback for free-model-only setups. - Notes: Add to notes distills long selections into 1-3 dense sentences preserving exact identifiers, with verbatim fallback on failure. Fixes along the way: - The global event watcher now starts unconditionally; it was gated behind the desktop-notify env, leaving the server-side event hub dead in packaged apps. - OpenCode re-emits message.updated for old user messages after idle; the watcher no longer mistakes those for new activity. - Session metadata merges from a fresh read right before the PATCH, so writes made during the generation window (suggestion dismissals, review links) are preserved; the assist runtime stops during graceful shutdown.
65 lines
3.3 KiB
Markdown
65 lines
3.3 KiB
Markdown
# Session Assist
|
|
|
|
Server-side watcher that generates a short recap of the agent's last reply
|
|
and one suggested user follow-up with the small model
|
|
(`lib/small-model`), storing both on the session's metadata under
|
|
`metadata.openchamber.assist`.
|
|
|
|
## Flow
|
|
|
|
1. `createSessionAssistRuntime` is a consumer of the server's global SSE
|
|
fan-out (`index.js` → `onPayload`), riding the same upstream connection as
|
|
notifications. Purely event-driven — dormant sessions never generate
|
|
anything, there is no backfill and no session scanning.
|
|
2. `session.status: idle` arms a 60-second per-session timer; any `busy`/
|
|
`retry` status or a user `message.updated` clears it (the "1 minute of
|
|
quiet" rule).
|
|
3. On fire: fetch the session (skip sub-agent sessions with `parentID`),
|
|
take the LAST exchange only — the final assistant reply plus the user
|
|
message it answered (assistant `parentID` → user id) — and call
|
|
`generateSmallModelText` with the
|
|
session's own provider/model taken from the last assistant message — so
|
|
the utility call spends the same subscription as the conversation.
|
|
`restrictToPreferredProvider` forbids the resolver's global fallback:
|
|
conversation content never goes to a provider the user didn't pick for
|
|
the session, unless the small model was chosen explicitly (settings
|
|
override or opencode config). A resolver 404 is silently skipped.
|
|
4. The `{recap, suggestion}` JSON is clamped and PATCHed onto the session
|
|
metadata together with `forMessageID` (the last assistant message id) and
|
|
`generatedAt`. Before writing, the session tail is re-checked (a stale
|
|
result is dropped) and the metadata is merged from a fresh session read so
|
|
concurrent metadata writes made during generation are preserved.
|
|
|
|
## Settings gate
|
|
|
|
`sessionAssistEnabled` in OpenChamber settings (Settings → Chat, default on)
|
|
is a hard generation switch checked at fire time: when off, no small-model
|
|
calls run and nothing is written. Existing payloads keep rendering and can
|
|
still be dismissed — the switch is about generation, not visibility.
|
|
|
|
## Freshness contract (no clearing writes)
|
|
|
|
Clients do not need the payload to be deleted: they render it only while
|
|
`assist.forMessageID` still equals the session's last assistant message id
|
|
(and the session is idle). Any new message invalidates the payload
|
|
everywhere instantly and offline; the next idle cycle overwrites it.
|
|
|
|
## UI consumers (packages/ui)
|
|
|
|
- `lib/sessionAssistMetadata.ts` — payload parsing.
|
|
- `hooks/useSessionAssist.ts` — freshness gating + the 5-minute quiet window
|
|
for the recap (single timeout to the boundary, no polling).
|
|
- `components/chat/SessionRecapSpacer.tsx` — renders the recap inside the
|
|
fixed-height reserved gap under the last message (height never changes).
|
|
- `components/chat/SessionSuggestionChip.tsx` — one tappable suggestion chip
|
|
near the composer (desktop chips row + above the mobile pill); hidden as
|
|
soon as the composer has any content. Tap fills the input, never sends.
|
|
|
|
## Limitations
|
|
|
|
- The watcher lives in the web server, so VS Code (extension-only, no web
|
|
server) does not generate assists; it still renders payloads produced by a
|
|
web/desktop instance of the same OpenCode server via `session.updated`.
|
|
- Metadata payloads ride every `session.updated` event — keep the clamps
|
|
(`RECAP_CHAR_LIMIT`, `SUGGESTION_CHAR_LIMIT`) small.
|