BuzzbeeSCD/openchamber
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: small-model utility calls on existing OpenCode providers (#2049)

Browse Source 
Adds a server-side "small model" capability: direct, cheap LLM calls that
reuse the user's existing OpenCode provider logins — the mechanism OpenCode
uses internally for titles and summaries but does not expose through the
SDK or plugins. Zero new dependencies; plain fetch with per-provider wire
formats, credentials never leave the server.

Core (packages/web/server/lib/small-model):
- Resolution mirrors OpenCode's session scoping: explicit settings override
  → small_model from the OpenCode config → family scan within the session's
  provider → the session's own model. The global provider scan only serves
  callers without a session context, and background callers forbid it
  entirely (restrictToPreferredProvider), so conversation content never
  reaches a provider the user didn't pick — explicit choices excepted.
- Per-provider auth replicating OpenCode's plugin loaders: GitHub Copilot
  (device token as bearer, no exchange), ChatGPT plan via the codex
  Responses API (single-flight OAuth refresh written back to auth.json),
  Anthropic messages, Google generateContent, generic OpenAI-compatible.
- OpenCode's free models (opencode/big-pickle, *-free) are never called
  directly; unauthenticated providers are skipped by design.
- Prompt clamping to the model's catalog context limit; thinking disabled
  where a wire switch exists (Z.AI/GLM, MiniMax-M3, Gemini Flash); robust
  content parsing with a clear error when a thinking model spends its whol
  budget on reasoning.
- Settings → Sessions gains a Small Model group: use-default checkbox plus
  an override picker limited to authenticated providers, persisted with
  web/desktop/VS Code sanitization parity.

Consumers:
- Session assist: a server-side watcher on the global SSE hub generates a
  short recap and one suggested follow-up after a session idles quietly fo
  a minute, stored on session metadata (openchamber.assist). Freshness is
  keyed to the last assistant message id, so new activity invalidates the
  payload everywhere with no extra writes. The chat shows the recap under
  the last message after five quiet minutes and the suggestion as a
  dismissible chip above the composer (tap fills the input, never sends).
  Gated by a new Chat setting (default on) that is a hard generation
  switch. Language is anchored to the conversation itself, with a
  script-mismatch guard against model/backend language hallucination.
- TTS: a third input mode, summarized — long replies are condensed to
  spoken prose before playback on any TTS engine.
- Git: commit-message and PR generation moved off the active chat session
  onto the small model fed with real diffs and the commit list (bodies
  included), with a session-transport fallback for free-model-only setups.
- Notes: Add to notes distills long selections into 1-3 dense sentences
  preserving exact identifiers, with verbatim fallback on failure.

Fixes along the way:
- The global event watcher now starts unconditionally; it was gated behind
  the desktop-notify env, leaving the server-side event hub dead in
  packaged apps.
- OpenCode re-emits message.updated for old user messages after idle; the
  watcher no longer mistakes those for new activity.
- Session metadata merges from a fresh read right before the PATCH, so
  writes made during the generation window (suggestion dismissals, review
  links) are preserved; the assist runtime stops during graceful shutdown.
This commit is contained in:
Bohdan Triapitsyn
2026-07-05 23:19:10 +03:00
committed by GitHub
parent e5b03493da
commit 28f0736d69
61 changed files with 2455 additions and 103 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/ui/src/lib/gitApi.ts
+158 -41
View File
@@ -2,6 +2,7 @@
import * as gitHttp from './gitApiHttp';
import { opencodeClient } from './opencode/client';
import { renderMagicPrompt } from './magicPrompts';
import { runtimeFetch } from './runtime-fetch';
import { materializeOpenDraftSession, useSessionUIStore } from '@/sync/session-ui-store';
import { useSelectionStore } from '@/sync/selection-store';
import { useConfigStore } from '@/stores/useConfigStore';
@@ -210,6 +211,71 @@ export async function deleteRemoteBranch(directory: string, payload: import('./a
return gitHttp.deleteRemoteBranch(directory, payload);
}
const COMMIT_DIFF_FILE_LIMIT = 30;
const COMMIT_DIFF_TOTAL_CHAR_LIMIT = 120_000;
const collectSelectedFileDiffs = async (directory: string, files: string[]): Promise<string> => {
const limited = files.slice(0, COMMIT_DIFF_FILE_LIMIT);
const chunks = await Promise.all(limited.map(async (path) => {
try {
const [staged, unstaged] = await Promise.all([
gitHttp.getGitDiff(directory, { path, staged: true }).catch(() => null),
gitHttp.getGitDiff(directory, { path, staged: false }).catch(() => null),
]);
const text = [staged?.diff, unstaged?.diff]
.filter((diff): diff is string => typeof diff === 'string' && diff.trim().length > 0)
.join('\n');
return text ? text : `--- ${path} (no textual diff available)`;
} catch {
return `--- ${path} (diff unavailable)`;
}
}));
let total = '';
for (const chunk of chunks) {
if (total.length + chunk.length > COMMIT_DIFF_TOTAL_CHAR_LIMIT) {
total += '\n[remaining diffs truncated]';
break;
}
total += (total ? '\n\n' : '') + chunk;
}
if (files.length > limited.length) {
total += `\n[${files.length - limited.length} more selected files omitted]`;
}
return total;
};
const parseCommitStructured = (structured: Record<string, unknown> | null): { subject: string; highlights: string[] } => {
const subject = typeof structured?.subject === 'string' ? structured.subject.trim() : '';
const highlights = Array.isArray(structured?.highlights)
? structured.highlights.filter((item) => typeof item === 'string').map((item) => item.trim()).filter(Boolean).slice(0, 3)
: [];
if (!subject) {
throw new Error('Structured output missing subject');
}
return { subject, highlights };
};
// Legacy transport: run the structured generation inside the active chat
// session. Kept as the fallback for setups with no direct provider login
// (vanilla installs on OpenCode's free models), where the small-model
// endpoint has nothing to call but the session itself still works.
async function generateCommitMessageViaSession(
directory: string,
visiblePrompt: string,
hiddenPrompt: string,
): Promise<{ message: import('./api/types').GeneratedCommitMessage }> {
const generationSession = await resolveGenerationSessionContext();
const structured = await runStructuredGenerationInActiveSession({
directory,
visiblePrompt,
hiddenPrompt,
generationSession,
kind: 'commit',
});
return { message: parseCommitStructured(structured) };
}
export async function generateCommitMessage(
directory: string,
files: string[],
@@ -217,17 +283,12 @@ export async function generateCommitMessage(
): Promise<{ message: import('./api/types').GeneratedCommitMessage }> {
const startedAt = Date.now();
void options;
const generationSession = await resolveGenerationSessionContext();
console.info('[git-generation][browser] request', {
transport: 'session',
transport: 'small-model',
kind: 'commit',
directory,
selectedFiles: files.length,
sessionId: generationSession.sessionId,
providerId: generationSession.providerID,
modelId: generationSession.modelID,
agent: generationSession.agent,
});
const visiblePrompt = await renderMagicPrompt('git.commit.generate.visible');
@@ -236,26 +297,44 @@ export async function generateCommitMessage(
});
try {
const structured = await runStructuredGenerationInActiveSession({
directory,
visiblePrompt,
hiddenPrompt,
generationSession,
kind: 'commit',
const diffs = await collectSelectedFileDiffs(directory, files);
const { currentProviderId, currentModelId } = useConfigStore.getState();
const response = await runtimeFetch('/api/small-model/generate', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
system: visiblePrompt,
prompt: `${hiddenPrompt}\n\nDiffs of the selected files:\n${diffs}`,
directory,
...(currentProviderId ? { preferredProviderID: currentProviderId } : {}),
...(currentModelId ? { preferredModelID: currentModelId } : {}),
}),
});
const subject = typeof structured.subject === 'string' ? structured.subject.trim() : '';
const highlights = Array.isArray(structured.highlights)
? structured.highlights.filter((item) => typeof item === 'string').map((item) => item.trim()).filter(Boolean).slice(0, 3)
: [];
if (!subject) {
throw new Error('Structured output missing subject');
if (response.status === 404) {
// No authenticated provider has a small model — fall back to the
// session transport so free-model-only setups keep a working button.
console.info('[git-generation][browser] small model unavailable, falling back to session transport');
const result = await generateCommitMessageViaSession(directory, visiblePrompt, hiddenPrompt);
console.info('[git-generation][browser] success', {
transport: 'session-fallback',
kind: 'commit',
elapsedMs: Date.now() - startedAt,
subjectLength: result.message.subject.length,
highlightsCount: result.message.highlights.length,
});
return result;
}
const result = { message: { subject, highlights } };
const payload = await response.json().catch(() => null) as { text?: unknown; error?: unknown } | null;
if (!response.ok || typeof payload?.text !== 'string') {
const message = typeof payload?.error === 'string' ? payload.error : `HTTP ${response.status}`;
throw new Error(message);
}
const result = { message: parseCommitStructured(extractJsonObject(payload.text)) };
console.info('[git-generation][browser] success', {
transport: 'session',
transport: 'small-model',
kind: 'commit',
elapsedMs: Date.now() - startedAt,
subjectLength: result.message.subject.length,
@@ -264,7 +343,7 @@ export async function generateCommitMessage(
return result;
} catch (error) {
console.error('[git-generation][browser] failed', {
transport: 'session',
transport: 'small-model',
kind: 'commit',
elapsedMs: Date.now() - startedAt,
message: error instanceof Error ? error.message : String(error),
@@ -279,18 +358,19 @@ export async function generatePullRequestDescription(
payload: { base: string; head: string; context?: string; zenModel?: string; providerId?: string; modelId?: string }
): Promise<import('./api/types').GeneratedPullRequestDescription> {
const startedAt = Date.now();
const generationSession = await resolveGenerationSessionContext();
const commitLog = await getGitLog(directory, {
from: payload.base,
to: payload.head,
maxCount: 50,
});
const COMMIT_BODY_CHAR_LIMIT = 2_000;
const commits = (Array.isArray(commitLog?.all) ? commitLog.all : [])
.filter((entry) => typeof entry?.hash === 'string' && entry.hash.length > 0)
.map((entry) => ({
hash: entry.hash,
subject: typeof entry.message === 'string' ? entry.message.trim() : '',
body: typeof entry.body === 'string' ? entry.body.trim().slice(0, COMMIT_BODY_CHAR_LIMIT) : '',
}));
if (commits.length === 0) {
@@ -317,13 +397,9 @@ export async function generatePullRequestDescription(
const changedFiles = Array.from(filesSet).sort().slice(0, 300);
console.info('[git-generation][browser] request', {
transport: 'session',
transport: 'small-model',
kind: 'pr',
directory,
sessionId: generationSession.sessionId,
providerId: generationSession.providerID,
modelId: generationSession.modelID,
agent: generationSession.agent,
base: payload.base,
head: payload.head,
commits: commits.length,
@@ -334,26 +410,67 @@ export async function generatePullRequestDescription(
const hiddenPrompt = await renderMagicPrompt('git.pr.generate.instructions', {
base_branch: payload.base,
head_branch: payload.head,
commits: commits.map((commit) => `- ${commit.hash.slice(0, 7)} ${commit.subject || '(no subject)'}`).join('\n'),
commits: commits.map((commit) => {
const line = `- ${commit.hash.slice(0, 7)} ${commit.subject || '(no subject)'}`;
if (!commit.body) return line;
const indentedBody = commit.body.split('\n').map((bodyLine) => ` ${bodyLine}`).join('\n');
return `${line}\n${indentedBody}`;
}).join('\n'),
changed_files: changedFiles.length > 0 ? changedFiles.map((file) => `- ${file}`).join('\n') : '- none detected',
additional_context_block: payload.context?.trim() ? `\nAdditional context:\n${payload.context.trim()}` : '',
});
const parsePrStructured = (structured: Record<string, unknown> | null) => ({
title: typeof structured?.title === 'string' ? structured.title.trim() : '',
body: typeof structured?.body === 'string' ? structured.body.trim() : '',
});
try {
const structured = await runStructuredGenerationInActiveSession({
directory,
visiblePrompt,
hiddenPrompt,
generationSession,
kind: 'pr',
const { currentProviderId, currentModelId } = useConfigStore.getState();
const response = await runtimeFetch('/api/small-model/generate', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
system: visiblePrompt,
prompt: hiddenPrompt,
directory,
...(currentProviderId ? { preferredProviderID: currentProviderId } : {}),
...(currentModelId ? { preferredModelID: currentModelId } : {}),
}),
});
const result = {
title: typeof structured.title === 'string' ? structured.title.trim() : '',
body: typeof structured.body === 'string' ? structured.body.trim() : '',
};
if (response.status === 404) {
// No authenticated provider has a small model — fall back to the
// session transport so free-model-only setups keep working.
console.info('[git-generation][browser] small model unavailable, falling back to session transport');
const generationSession = await resolveGenerationSessionContext();
const structured = await runStructuredGenerationInActiveSession({
directory,
visiblePrompt,
hiddenPrompt,
generationSession,
kind: 'pr',
});
const result = parsePrStructured(structured);
console.info('[git-generation][browser] success', {
transport: 'session-fallback',
kind: 'pr',
elapsedMs: Date.now() - startedAt,
titleLength: result.title.length,
bodyLength: result.body.length,
});
return result;
}
const payload = await response.json().catch(() => null) as { text?: unknown; error?: unknown } | null;
if (!response.ok || typeof payload?.text !== 'string') {
const message = typeof payload?.error === 'string' ? payload.error : `HTTP ${response.status}`;
throw new Error(message);
}
const result = parsePrStructured(extractJsonObject(payload.text));
console.info('[git-generation][browser] success', {
transport: 'session',
transport: 'small-model',
kind: 'pr',
elapsedMs: Date.now() - startedAt,
titleLength: result.title.length,
@@ -362,7 +479,7 @@ export async function generatePullRequestDescription(
return result;
} catch (error) {
console.error('[git-generation][browser] failed', {
transport: 'session',
transport: 'small-model',
kind: 'pr',
elapsedMs: Date.now() - startedAt,
message: error instanceof Error ? error.message : String(error),