c8a08588d9cf1881a53de3fa5644b5eda22b84bb
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Bohdan Triapitsyn
|
28f0736d69 |
feat: small-model utility calls on existing OpenCode providers (#2049)
Adds a server-side "small model" capability: direct, cheap LLM calls that reuse the user's existing OpenCode provider logins — the mechanism OpenCode uses internally for titles and summaries but does not expose through the SDK or plugins. Zero new dependencies; plain fetch with per-provider wire formats, credentials never leave the server. Core (packages/web/server/lib/small-model): - Resolution mirrors OpenCode's session scoping: explicit settings override → small_model from the OpenCode config → family scan within the session's provider → the session's own model. The global provider scan only serves callers without a session context, and background callers forbid it entirely (restrictToPreferredProvider), so conversation content never reaches a provider the user didn't pick — explicit choices excepted. - Per-provider auth replicating OpenCode's plugin loaders: GitHub Copilot (device token as bearer, no exchange), ChatGPT plan via the codex Responses API (single-flight OAuth refresh written back to auth.json), Anthropic messages, Google generateContent, generic OpenAI-compatible. - OpenCode's free models (opencode/big-pickle, *-free) are never called directly; unauthenticated providers are skipped by design. - Prompt clamping to the model's catalog context limit; thinking disabled where a wire switch exists (Z.AI/GLM, MiniMax-M3, Gemini Flash); robust content parsing with a clear error when a thinking model spends its whol budget on reasoning. - Settings → Sessions gains a Small Model group: use-default checkbox plus an override picker limited to authenticated providers, persisted with web/desktop/VS Code sanitization parity. Consumers: - Session assist: a server-side watcher on the global SSE hub generates a short recap and one suggested follow-up after a session idles quietly fo a minute, stored on session metadata (openchamber.assist). Freshness is keyed to the last assistant message id, so new activity invalidates the payload everywhere with no extra writes. The chat shows the recap under the last message after five quiet minutes and the suggestion as a dismissible chip above the composer (tap fills the input, never sends). Gated by a new Chat setting (default on) that is a hard generation switch. Language is anchored to the conversation itself, with a script-mismatch guard against model/backend language hallucination. - TTS: a third input mode, summarized — long replies are condensed to spoken prose before playback on any TTS engine. - Git: commit-message and PR generation moved off the active chat session onto the small model fed with real diffs and the commit list (bodies included), with a session-transport fallback for free-model-only setups. - Notes: Add to notes distills long selections into 1-3 dense sentences preserving exact identifiers, with verbatim fallback on failure. Fixes along the way: - The global event watcher now starts unconditionally; it was gated behind the desktop-notify env, leaving the server-side event hub dead in packaged apps. - OpenCode re-emits message.updated for old user messages after idle; the watcher no longer mistakes those for new activity. - Session metadata merges from a fresh read right before the PATCH, so writes made during the generation window (suggestion dismissals, review links) are preserved; the assist runtime stops during graceful shutdown. |
||
|
Bohdan Triapitsyn
|
de1b85ac56 |
feat(voice): first-class voice input and local TTS across web, desktop, and mobile (#2018)
Complete rebuild of voice input on a server-authoritative streaming architecture, replacing the legacy Web Speech / whole-blob / WASM engines and the dead voice-agent layer (~4k lines removed). Speech-to-text (dictation): - Client streams 16 kHz mono PCM16 chunks over /api/dictation/ws with seq/ack ordering; buffered audio is retained and replayed on reconnect - Server transcribes and streams live partial transcripts back; segments auto-commit every ~15s with silence suppression and adaptive finalization timeouts - Local provider (default, zero config): sherpa-onnx models in a forked worker process — auto-download with progress, staged extraction with verification, corrupt-model auto-recovery, idle shutdown after 5 min - Model catalog with settings picker (accuracy/speed ratings, sizes, download/delete): Parakeet TDT v2 (English) and v3 (25 European languages, auto-detected), Whisper base and tiny (multilingual, light) - OpenAI-compatible provider for any Whisper endpoint - Composer overlay with live transcript, volume meter, timer, and cancel / insert / insert-and-send actions; failed transcriptions keep their audio for retry or accepting the partial text as-is - Configurable keyboard shortcut (default mod+alt+v) toggles dictation; Enter confirms and Escape cancels while recording - Overlay is pixel-aligned with the composer (measured footer height, matching paddings/typography/gaps) — no layout shift when toggling Text-to-speech: - Local Kokoro provider (English, 11 voices) synthesized in the same worker via /api/dictation/tts/speak, managed by the shared model pipeline; sentence-pipelined playback keeps time-to-first-audio at ~1 sentence regardless of message length, and stop cancels in-flight synthesis - Sanitizer keeps inline-code content (strips backticks only), reads interword slashes aloud, and removes only absolute file paths Settings: - Voice page unified: a single read-aloud toggle owns all playback options (the confusing "Enable Voice Mode" is gone); a new "Enable voice input" toggle (default on, persisted to settings.json) hides the composer mic entirely when disabled Mobile and transport: - iOS/Android microphone permissions added (dictation was previously impossible on mobile) - Fixed Android WebSocket upgrades: the Capacitor WebView origin (https://localhost) was missing from the packaged-client allowlist, 403-ing every WS connection — root cause of the old mobile SSE lock, which is now removed for all transports Security and conventions: - All HTTP routes sit behind the global /api auth gate; the WS upgrade explicitly validates the UI session and origin, with oc_url_token narrowly allowlisted and covered by tests; the dictation socket mints a fresh URL token before connecting - Routes register before the generic OpenCode proxy; the client goes through runtimeFetch/getRuntimeUrlResolver, and runtime switches reset the dictation socket - VS Code deliberately reports dictation as unavailable (no server process in that runtime) CI: workflow Node bumped 20 -> 22 to match the repo engines and fix better-sqlite3 installs broken by node-gyp@latest on Node 20. New dependency: sherpa-onnx-node (prebuilt N-API; macOS/Linux x64+arm64, Windows x64 — Windows-on-ARM falls back to the OpenAI-compatible provider) |
||
|
yangyaofeiandBohdan Triapitsyn
|
ba95e13b36 |
feat(tts): add TTS buttons to PlanView and FilesView markdown preview with configurable input mode (#1443)
Add TTS controls to markdown preview surfaces so users can listen to plans and markdown files directly from the preview toolbar. Changes: - Add read-aloud / stop buttons to PlanView markdown preview. - Add read-aloud / stop buttons to FilesView markdown preview for markdown files. - Respect the existing showMessageTTSButtons preference in both preview views. - Add a persisted ttsInputMode setting with sanitized/raw modes. - Keep sanitized mode as the default for backward compatibility. - Allow raw markdown only for server TTS providers that can handle markdown. - Always use sanitized text for browser and macOS say fallback paths. - Add Voice Settings controls for TTS input mode. - Add i18n keys for the new preview buttons and setting labels. - Merge latest main and preserve newer FilesView toolbar/editor changes. Validation: - bun test packages/ui/src/stores/useConfigStore.test.ts packages/ui/src/components/chat/message/parts/ToolPart.test.ts packages/web/server/lib/tts/routes.test.js - bun run type-check - bun run lint - git diff --check --------- Co-authored-by: Bohdan Triapitsyn <artmore@protonmail.com> |
||
|
Bohdan Triapitsyn
|
174fa4e96d |
chore: retire zen-backed summarization
Disable the active Zen summarization flow because the unauthenticated/free Zen provider is no longer available and now returns usage-limit errors for this feature.
Keep /api/text/summarize as an API-compatible stub that returns local sanitized or distilled fallback text with summarized=false, rather than attempting external model calls.
Remove notification and voice playback summary behavior from runtime paths. Notification {last_message} now always uses normalized truncated text, and TTS playback ignores historical summarize request fields.
Hide the notification summary settings and voice summarize-before-playback controls while preserving legacy persisted settings for compatibility. Also disable Zen model startup validation and make Zen model list routes return empty results.
Update module documentation and tests to describe the retired provider behavior and the remaining compatibility stubs.
|
||
|
Bohdan Triapitsyn
|
c7f1501ad2 |
fix: harden custom TTS URL handling and availability checks
Validate and normalize custom TTS server URLs before proxying requests. Gate server TTS availability by provider mode to avoid OpenAI/custom misrouting. Keep browser and message TTS hooks aligned with the new provider-specific checks. |
||
|
Alexander BusseandAlexander Busse
|
055b6f6af0 |
feat(tts): add OpenAI-compatible custom server TTS provider with configurable model, pitch, and volume (#859)
Co-authored-by: Alexander Busse <alex@ableph.net> |
||
|
Bohdan Triapitsyn
|
8587db89b1 |
perf: reduce UI render fanout and scroll jitter
- Cut broad render fanout across the app by replacing shared-store whole-object subscriptions with leaf selectors, memoizing hot chrome boundaries, and isolating disabled global providers from live session/message state. This keeps header controls, composer toolbars, side panels, and other non-hot UI surfaces from repainting on every assistant update or keystroke. - Rework sidebar session ordering so recent, project groups, and worktree groups derive from one ordering source while avoiding streaming-time thrash. The sidebar now uses a stabilized session snapshot, preserves structural identity for unchanged rows, reads live row status/details per session, and applies a one-shot sort bump on idle->busy instead of continuously resorting during activity. - Fix chat/input scroll instability by separating viewport-resize handling from message-growth handling, disabling conflicting native scroll anchoring, and stopping textarea autosize from collapsing on every growth keystroke. This removes the multiline typing jiggle during streaming and reduces unnecessary composer rerenders. - Also gate voice context wiring behind voice-mode enablement and codify the learned render/scroll/order anti-patterns in AGENTS.md so future changes avoid the same classes of regressions. |
||
|
Bohdan Triapitsyn
|
1231fd773e |
feat: improve VS Code dev flow and stabilize sidebar/chat behavior (#754)
* fix: improve session sidebar tooltip and truncation behavior - Keep new-draft tooltip anchored to its trigger button - Fix minimal-mode worktree/group header text truncation - Tune minimal-mode right padding to reduce early label clipping * fix: render reasoning through markdown pipeline - Use Streamdown rendering for reasoning in live chat mode - Remove italic styling from reasoning text - Render expanded reasoning content with MarkdownRenderer * chore: remove legacy electron dependencies - Removed unused Electron packages from root and UI manifests - Deleted obsolete Electron context menu type declaration - Regenerated lockfile after dependency cleanup * fix: handle non-repository folders in git status API - Prevent 500 errors when status is requested outside a valid Git repo - Improve repository detection using `git rev-parse --git-dir` - Reduce noisy server logs for expected non-repo status checks * fix unloaded session chat layout flicker * fix: reduce noisy TTS status polling Cache and dedupe TTS status requests, and only check provider availability when the related voice features are enabled so disabled voice setups stay quiet. * perf: throttle background PR git status refreshes * fix: improve VS Code Explorer file drop mentions in chat - Add Explorer context action to insert selected files as @mentions. - Handle Explorer drag-and-drop to prefill @file mentions instead of attachments. - Prevent duplicate plain-path text when dropping multiple files. * fix: deduplicate recent sessions in VS Code sidebar - Hide sessions from main list when already shown in recent - Apply dedup only in VS Code runtime - Keep session search behavior unchanged * feat: add true HMR dev flow for VS Code extension - Load VS Code webview from Vite dev server with React refresh preamble - Add `vscode:dev` runner that starts watchers and opens Extension Development Host - Update VS Code dev docs and scripts to use the new HMR startup flow * feat: polish VS Code session sidebar and attachment UX - Add resizable sessions sidebar in VS Code layout - Tighten session list spacing and hover behavior in VS Code - Remove bulk file/image attach success toasts while keeping error toasts |
||
|
gsxdsmandBohdan Triapitsyn
|
1ed5316ac7 |
feat(voice): add voice input/output support with multiple providers (#281)
* feat(voice): add voice input/output support with multiple providers - Add BrowserVoiceButton component for Web Speech API voice input - Add VoiceProvider context for managing voice state across the app - Add TTS (Text-to-Speech) support with browser, macOS Say, and OpenAI providers - Add message TTS buttons to read assistant messages aloud - Add VoiceSettings page in OpenChamber settings - Add server endpoints for TTS and summarization services - Include slider component for voice rate/pitch/volume controls - Add hidden session support for background voice operations - Add Caddyfile for HTTPS support (required for microphone access) * fix: Build errors fixed and removed outdated ElevenLabs test code. * refactor(voice): use zen API with gpt-5-nano for TTS summarization Replace the hidden session + OpenCode SDK approach with direct calls to the opencode.ai zen API (same pattern used for commit message and PR description generation). - Rewrite summarization-service.js to call zen/v1/responses with gpt-5-nano - Remove hidden session logic (hiddenSession.ts, sessionStore filtering) - Remove summarizeModel setting and model selector from VoiceSettings - Simplify client-side summarize.ts to no longer pass model params - Clean up callers in useMessageTTS and useBrowserVoice * fix(voice): remove false 'voice not supported' warning in settings Mobile Safari does support voice but the isSupported check was incorrectly flagging it. Remove the warning banner entirely. * feat(voice): add configurable summary length limit for TTS output Add a slider (50-2000 chars) in voice settings to control max summary length. The limit is passed through the summarize endpoint and speak endpoint to the zen API prompt, with token budget scaled accordingly. * fix(voice): add diagnostic logging and sanitize TTS fallback Add console logging throughout the summarization flow (client + server) to trace why text may not be summarized. Fix silent error swallowing in /api/tts/speak. Always apply sanitizeForTTS even when summarization is disabled so raw markdown/code is never spoken verbatim. * fix(voice): fix token budget starving model of output tokens max_output_tokens includes both reasoning and output tokens. With effort:'low', reasoning alone consumes ~128 tokens, so a budget of 100 left zero tokens for the actual summary text. Use a fixed 1000 token budget (matching commit message generation) and control output length via the prompt's character limit instruction instead. * chore(voice): remove diagnostic logging from summarization flow * fix(voice): don't request mic permission on mobile page load Remove the useEffect that pre-requested microphone permission when the BrowserVoiceButton component mounted on mobile. This caused an unwanted permission prompt immediately on page load before the user tapped the mic icon. Permission is now only requested on explicit user interaction. * fix(voice): remove unused BrowserVoiceButton binding * fix(voice): desktop mic flow + non-continuous draft mode * fix(voice): stabilize continuous loop and polish controls * feat(settings): mark voice section experimental --------- Co-authored-by: Bohdan Triapitsyn <artmore@protonmail.com> |