feat: small-model utility calls on existing OpenCode providers (#2049)
Adds a server-side "small model" capability: direct, cheap LLM calls that reuse the user's existing OpenCode provider logins — the mechanism OpenCode uses internally for titles and summaries but does not expose through the SDK or plugins. Zero new dependencies; plain fetch with per-provider wire formats, credentials never leave the server. Core (packages/web/server/lib/small-model): - Resolution mirrors OpenCode's session scoping: explicit settings override → small_model from the OpenCode config → family scan within the session's provider → the session's own model. The global provider scan only serves callers without a session context, and background callers forbid it entirely (restrictToPreferredProvider), so conversation content never reaches a provider the user didn't pick — explicit choices excepted. - Per-provider auth replicating OpenCode's plugin loaders: GitHub Copilot (device token as bearer, no exchange), ChatGPT plan via the codex Responses API (single-flight OAuth refresh written back to auth.json), Anthropic messages, Google generateContent, generic OpenAI-compatible. - OpenCode's free models (opencode/big-pickle, *-free) are never called directly; unauthenticated providers are skipped by design. - Prompt clamping to the model's catalog context limit; thinking disabled where a wire switch exists (Z.AI/GLM, MiniMax-M3, Gemini Flash); robust content parsing with a clear error when a thinking model spends its whol budget on reasoning. - Settings → Sessions gains a Small Model group: use-default checkbox plus an override picker limited to authenticated providers, persisted with web/desktop/VS Code sanitization parity. Consumers: - Session assist: a server-side watcher on the global SSE hub generates a short recap and one suggested follow-up after a session idles quietly fo a minute, stored on session metadata (openchamber.assist). Freshness is keyed to the last assistant message id, so new activity invalidates the payload everywhere with no extra writes. The chat shows the recap under the last message after five quiet minutes and the suggestion as a dismissible chip above the composer (tap fills the input, never sends). Gated by a new Chat setting (default on) that is a hard generation switch. Language is anchored to the conversation itself, with a script-mismatch guard against model/backend language hallucination. - TTS: a third input mode, summarized — long replies are condensed to spoken prose before playback on any TTS engine. - Git: commit-message and PR generation moved off the active chat session onto the small model fed with real diffs and the commit list (bodies included), with a session-transport fallback for free-model-only setups. - Notes: Add to notes distills long selections into 1-3 dense sentences preserving exact identifiers, with verbatim fallback on failure. Fixes along the way: - The global event watcher now starts unconditionally; it was gated behind the desktop-notify env, leaving the server-side event hub dead in packaged apps. - OpenCode re-emits message.updated for old user messages after idle; the watcher no longer mistakes those for new activity. - Session metadata merges from a fresh read right before the PATCH, so writes made during the generation window (suggestion dismissals, review links) are preserved; the assist runtime stops during graceful shutdown.
This commit is contained in:
Bohdan Triapitsyn
committed by
GitHub
GitHub
parent
e5b03493da
commit
28f0736d69
@@ -39,6 +39,9 @@ export const DefaultsSettings: React.FC = () => {
|
||||
const [defaultModel, setDefaultModel] = React.useState<string | undefined>();
|
||||
const [defaultVariant, setDefaultVariant] = React.useState<string | undefined>();
|
||||
const [defaultAgent, setDefaultAgent] = React.useState<string | undefined>();
|
||||
const [smallModelUseDefault, setSmallModelUseDefault] = React.useState(true);
|
||||
const [smallModelOverride, setSmallModelOverride] = React.useState<string | undefined>();
|
||||
const [smallModelProviders, setSmallModelProviders] = React.useState<string[] | undefined>();
|
||||
const [isLoading, setIsLoading] = React.useState(true);
|
||||
|
||||
const parsedModel = React.useMemo(() => getDisplayModel(defaultModel), [defaultModel]);
|
||||
@@ -50,6 +53,8 @@ export const DefaultsSettings: React.FC = () => {
|
||||
defaultModel?: string;
|
||||
defaultVariant?: string;
|
||||
defaultAgent?: string;
|
||||
smallModelUseDefault?: boolean;
|
||||
smallModelOverride?: string;
|
||||
} | null = null;
|
||||
|
||||
if (!data) {
|
||||
@@ -59,13 +64,16 @@ export const DefaultsSettings: React.FC = () => {
|
||||
const result = await runtimeSettings.load();
|
||||
const settings = result?.settings;
|
||||
if (settings) {
|
||||
const raw = settings as Record<string, unknown>;
|
||||
data = {
|
||||
defaultModel: typeof settings.defaultModel === 'string' ? settings.defaultModel : undefined,
|
||||
defaultVariant:
|
||||
typeof (settings as Record<string, unknown>).defaultVariant === 'string'
|
||||
? ((settings as Record<string, unknown>).defaultVariant as string)
|
||||
typeof raw.defaultVariant === 'string'
|
||||
? (raw.defaultVariant as string)
|
||||
: undefined,
|
||||
defaultAgent: typeof settings.defaultAgent === 'string' ? settings.defaultAgent : undefined,
|
||||
smallModelUseDefault: typeof raw.smallModelUseDefault === 'boolean' ? raw.smallModelUseDefault : undefined,
|
||||
smallModelOverride: typeof raw.smallModelOverride === 'string' ? raw.smallModelOverride : undefined,
|
||||
};
|
||||
}
|
||||
} catch {
|
||||
@@ -101,6 +109,10 @@ export const DefaultsSettings: React.FC = () => {
|
||||
if (model !== undefined) setDefaultModel(model);
|
||||
if (variant !== undefined) setDefaultVariant(variant);
|
||||
if (agent !== undefined) setDefaultAgent(agent);
|
||||
if (typeof data.smallModelUseDefault === 'boolean') setSmallModelUseDefault(data.smallModelUseDefault);
|
||||
if (typeof data.smallModelOverride === 'string' && data.smallModelOverride.trim()) {
|
||||
setSmallModelOverride(data.smallModelOverride.trim());
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn('Failed to load defaults settings:', error);
|
||||
@@ -189,6 +201,53 @@ export const DefaultsSettings: React.FC = () => {
|
||||
[setAgent, setSettingsDefaultAgent]
|
||||
);
|
||||
|
||||
const handleSmallModelUseDefaultChange = React.useCallback(
|
||||
async (useDefault: boolean) => {
|
||||
setSmallModelUseDefault(useDefault);
|
||||
try {
|
||||
await updateDesktopSettings({ smallModelUseDefault: useDefault });
|
||||
} catch (error) {
|
||||
console.warn('Failed to save small model preference:', error);
|
||||
}
|
||||
},
|
||||
[]
|
||||
);
|
||||
|
||||
const handleSmallModelOverrideChange = React.useCallback(
|
||||
async (providerId: string, modelId: string) => {
|
||||
const newValue = providerId && modelId ? `${providerId}/${modelId}` : undefined;
|
||||
setSmallModelOverride(newValue);
|
||||
try {
|
||||
await updateDesktopSettings({ smallModelOverride: newValue ?? '' });
|
||||
} catch (error) {
|
||||
console.warn('Failed to save small model override:', error);
|
||||
}
|
||||
},
|
||||
[]
|
||||
);
|
||||
|
||||
const parsedSmallModel = React.useMemo(() => getDisplayModel(smallModelOverride), [smallModelOverride]);
|
||||
|
||||
React.useEffect(() => {
|
||||
if (smallModelUseDefault || smallModelProviders !== undefined) return;
|
||||
let cancelled = false;
|
||||
(async () => {
|
||||
try {
|
||||
const response = await runtimeFetch('/api/small-model', { method: 'GET', headers: { Accept: 'application/json' } });
|
||||
if (!response.ok) return;
|
||||
const payload = await response.json().catch(() => null) as { authenticatedProviders?: unknown } | null;
|
||||
if (!cancelled && Array.isArray(payload?.authenticatedProviders)) {
|
||||
setSmallModelProviders(payload.authenticatedProviders.filter((id): id is string => typeof id === 'string'));
|
||||
}
|
||||
} catch {
|
||||
// leave undefined — picker falls back to showing all providers
|
||||
}
|
||||
})();
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, [smallModelUseDefault, smallModelProviders]);
|
||||
|
||||
const availableVariants = React.useMemo(() => {
|
||||
if (!parsedModel.providerId || !parsedModel.modelId) return [];
|
||||
const provider = providers.find((p) => p.id === parsedModel.providerId);
|
||||
@@ -305,6 +364,56 @@ export const DefaultsSettings: React.FC = () => {
|
||||
</div>
|
||||
|
||||
</section>
|
||||
|
||||
<div className="mt-6 mb-0.5 px-1">
|
||||
<div className="flex items-center gap-2">
|
||||
<h3 className="typography-ui-header font-medium text-foreground">{t('settings.openchamber.defaults.smallModel.title')}</h3>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section className="px-2 pb-2 pt-0 space-y-0">
|
||||
<div className="mt-0 mb-1 typography-meta text-muted-foreground">
|
||||
{t('settings.openchamber.defaults.smallModel.description')}
|
||||
</div>
|
||||
|
||||
<div
|
||||
data-settings-item="sessions.small-model"
|
||||
className="group flex cursor-pointer items-center gap-2 py-1"
|
||||
role="button"
|
||||
tabIndex={0}
|
||||
aria-pressed={smallModelUseDefault}
|
||||
onClick={() => void handleSmallModelUseDefaultChange(!smallModelUseDefault)}
|
||||
onKeyDown={(event) => {
|
||||
if (event.key === ' ' || event.key === 'Enter') {
|
||||
event.preventDefault();
|
||||
void handleSmallModelUseDefaultChange(!smallModelUseDefault);
|
||||
}
|
||||
}}
|
||||
>
|
||||
<Checkbox
|
||||
checked={smallModelUseDefault}
|
||||
onChange={(checked) => void handleSmallModelUseDefaultChange(checked)}
|
||||
ariaLabel={t('settings.openchamber.defaults.smallModel.useDefaultAria')}
|
||||
/>
|
||||
<span className="typography-ui-label text-foreground">{t('settings.openchamber.defaults.smallModel.useDefault')}</span>
|
||||
</div>
|
||||
|
||||
{!smallModelUseDefault ? (
|
||||
<div className="flex flex-col gap-2 py-1 sm:flex-row sm:items-center sm:gap-8">
|
||||
<div className="flex min-w-0 flex-col sm:w-56 shrink-0">
|
||||
<span className="typography-ui-label text-foreground">{t('settings.openchamber.defaults.smallModel.overrideModel')}</span>
|
||||
</div>
|
||||
<div className="flex min-w-0 flex-1 items-center gap-2 sm:w-fit sm:flex-initial">
|
||||
<ModelSelector
|
||||
providerId={parsedSmallModel.providerId}
|
||||
modelId={parsedSmallModel.modelId}
|
||||
onChange={handleSmallModelOverrideChange}
|
||||
allowedProviderIds={smallModelProviders}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
) : null}
|
||||
</section>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
@@ -144,7 +144,7 @@ const VisualSectionContent: React.FC = () => {
|
||||
|
||||
// Chat section: User message rendering, Diff layout, Mobile status bar, Show reasoning traces, Follow-up behavior, Persist draft
|
||||
const ChatSectionContent: React.FC = () => {
|
||||
return <OpenChamberVisualSettings visibleSettings={['chatRenderMode', 'messageTransport', 'activityRenderMode', 'userMessageRendering', 'mermaidRendering', 'reasoning', 'showToolFileIcons', 'showTurnChangedFiles', 'expandedTools', 'collapsibleUserMessages', 'stickyUserHeader', 'wideChatLayout', 'splitAssistantMessageActions', 'diffLayout', 'dotfiles', 'fileViewerPreview', 'followUpBehavior', 'persistDraft', 'inputSpellcheck']} />;
|
||||
return <OpenChamberVisualSettings visibleSettings={['sessionAssist', 'chatRenderMode', 'messageTransport', 'activityRenderMode', 'userMessageRendering', 'mermaidRendering', 'reasoning', 'showToolFileIcons', 'showTurnChangedFiles', 'expandedTools', 'collapsibleUserMessages', 'stickyUserHeader', 'wideChatLayout', 'splitAssistantMessageActions', 'diffLayout', 'dotfiles', 'fileViewerPreview', 'followUpBehavior', 'persistDraft', 'inputSpellcheck']} />;
|
||||
};
|
||||
|
||||
// Sessions section: Default model & agent, Session retention
|
||||
|
||||
@@ -245,7 +245,7 @@ const normalizeUserMessageRenderingMode = (mode: unknown): 'markdown' | 'plain'
|
||||
return mode === 'markdown' ? 'markdown' : 'plain';
|
||||
};
|
||||
|
||||
type VisibleSetting = 'theme' | 'pwaInstallName' | 'pwaOrientation' | 'mobileKeyboardMode' | 'timeFormat' | 'weekStart' | 'fontSize' | 'terminalFontSize' | 'spacing' | 'inputBarOffset' | 'mermaidRendering' | 'userMessageRendering' | 'chatRenderMode' | 'messageTransport' | 'activityRenderMode' | 'collapsibleUserMessages' | 'stickyUserHeader' | 'wideChatLayout' | 'splitAssistantMessageActions' | 'diffLayout' | 'mobileStatusBar' | 'dotfiles' | 'fileViewerPreview' | 'reasoning' | 'showToolFileIcons' | 'showTurnChangedFiles' | 'expandedTools' | 'followUpBehavior' | 'terminalQuickKeys' | 'fileEditorKeymap' | 'persistDraft' | 'inputSpellcheck' | 'reportUsage' | 'expandedEditorToolbar';
|
||||
type VisibleSetting = 'sessionAssist' | 'theme' | 'pwaInstallName' | 'pwaOrientation' | 'mobileKeyboardMode' | 'timeFormat' | 'weekStart' | 'fontSize' | 'terminalFontSize' | 'spacing' | 'inputBarOffset' | 'mermaidRendering' | 'userMessageRendering' | 'chatRenderMode' | 'messageTransport' | 'activityRenderMode' | 'collapsibleUserMessages' | 'stickyUserHeader' | 'wideChatLayout' | 'splitAssistantMessageActions' | 'diffLayout' | 'mobileStatusBar' | 'dotfiles' | 'fileViewerPreview' | 'reasoning' | 'showToolFileIcons' | 'showTurnChangedFiles' | 'expandedTools' | 'followUpBehavior' | 'terminalQuickKeys' | 'fileEditorKeymap' | 'persistDraft' | 'inputSpellcheck' | 'reportUsage' | 'expandedEditorToolbar';
|
||||
|
||||
interface OpenChamberVisualSettingsProps {
|
||||
/** Which settings to show. If undefined, shows all. */
|
||||
@@ -259,6 +259,8 @@ export const OpenChamberVisualSettings: React.FC<OpenChamberVisualSettingsProps>
|
||||
const { browserTab } = usePwaDetection();
|
||||
const directoryShowHidden = useDirectoryShowHidden();
|
||||
const showReasoningTraces = useUIStore(state => state.showReasoningTraces);
|
||||
const sessionAssistEnabled = useUIStore(state => state.sessionAssistEnabled);
|
||||
const setSessionAssistEnabled = useUIStore(state => state.setSessionAssistEnabled);
|
||||
const setShowReasoningTraces = useUIStore(state => state.setShowReasoningTraces);
|
||||
const collapsibleThinkingBlocks = useUIStore(state => state.collapsibleThinkingBlocks);
|
||||
const setCollapsibleThinkingBlocks = useUIStore(state => state.setCollapsibleThinkingBlocks);
|
||||
@@ -1775,8 +1777,31 @@ export const OpenChamberVisualSettings: React.FC<OpenChamberVisualSettingsProps>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{(shouldShow('collapsibleUserMessages') || shouldShow('stickyUserHeader') || shouldShow('wideChatLayout') || shouldShow('splitAssistantMessageActions') || shouldShow('dotfiles') || shouldShow('fileViewerPreview') || shouldShow('persistDraft') || shouldShow('showToolFileIcons') || shouldShow('showTurnChangedFiles') || (!isMobile && shouldShow('inputSpellcheck')) || shouldShow('reasoning')) && (
|
||||
{(shouldShow('sessionAssist') || shouldShow('collapsibleUserMessages') || shouldShow('stickyUserHeader') || shouldShow('wideChatLayout') || shouldShow('splitAssistantMessageActions') || shouldShow('dotfiles') || shouldShow('fileViewerPreview') || shouldShow('persistDraft') || shouldShow('showToolFileIcons') || shouldShow('showTurnChangedFiles') || (!isMobile && shouldShow('inputSpellcheck')) || shouldShow('reasoning')) && (
|
||||
<section className="p-2 space-y-0.5">
|
||||
{shouldShow('sessionAssist') && (
|
||||
<div
|
||||
data-settings-item="chat.session-assist"
|
||||
className="group flex cursor-pointer items-center gap-2 py-0.5"
|
||||
role="button"
|
||||
tabIndex={0}
|
||||
aria-pressed={sessionAssistEnabled}
|
||||
onClick={() => setSessionAssistEnabled(!sessionAssistEnabled)}
|
||||
onKeyDown={(event) => {
|
||||
if (event.key === ' ' || event.key === 'Enter') {
|
||||
event.preventDefault();
|
||||
setSessionAssistEnabled(!sessionAssistEnabled);
|
||||
}
|
||||
}}
|
||||
>
|
||||
<Checkbox
|
||||
checked={sessionAssistEnabled}
|
||||
onChange={setSessionAssistEnabled}
|
||||
ariaLabel={t('settings.openchamber.visual.field.sessionAssistAria')}
|
||||
/>
|
||||
<span className="typography-ui-label text-foreground">{t('settings.openchamber.visual.field.sessionAssist')}</span>
|
||||
</div>
|
||||
)}
|
||||
{shouldShow('reasoning') && (
|
||||
<div
|
||||
data-settings-item="chat.reasoning-traces"
|
||||
|
||||
@@ -1133,6 +1133,15 @@ export const VoiceSettings: React.FC = () => {
|
||||
>
|
||||
{t('settings.voice.page.field.ttsInputModeRaw')}
|
||||
</Button>
|
||||
<Button
|
||||
variant="chip"
|
||||
size="xs"
|
||||
aria-pressed={ttsInputMode === 'summarized'}
|
||||
onClick={() => setTtsInputMode('summarized')}
|
||||
className="!font-normal"
|
||||
>
|
||||
{t('settings.voice.page.field.ttsInputModeSummarized')}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
Reference in New Issue
Block a user