feat: small-model utility calls on existing OpenCode providers (#2049)
Adds a server-side "small model" capability: direct, cheap LLM calls that reuse the user's existing OpenCode provider logins — the mechanism OpenCode uses internally for titles and summaries but does not expose through the SDK or plugins. Zero new dependencies; plain fetch with per-provider wire formats, credentials never leave the server. Core (packages/web/server/lib/small-model): - Resolution mirrors OpenCode's session scoping: explicit settings override → small_model from the OpenCode config → family scan within the session's provider → the session's own model. The global provider scan only serves callers without a session context, and background callers forbid it entirely (restrictToPreferredProvider), so conversation content never reaches a provider the user didn't pick — explicit choices excepted. - Per-provider auth replicating OpenCode's plugin loaders: GitHub Copilot (device token as bearer, no exchange), ChatGPT plan via the codex Responses API (single-flight OAuth refresh written back to auth.json), Anthropic messages, Google generateContent, generic OpenAI-compatible. - OpenCode's free models (opencode/big-pickle, *-free) are never called directly; unauthenticated providers are skipped by design. - Prompt clamping to the model's catalog context limit; thinking disabled where a wire switch exists (Z.AI/GLM, MiniMax-M3, Gemini Flash); robust content parsing with a clear error when a thinking model spends its whol budget on reasoning. - Settings → Sessions gains a Small Model group: use-default checkbox plus an override picker limited to authenticated providers, persisted with web/desktop/VS Code sanitization parity. Consumers: - Session assist: a server-side watcher on the global SSE hub generates a short recap and one suggested follow-up after a session idles quietly fo a minute, stored on session metadata (openchamber.assist). Freshness is keyed to the last assistant message id, so new activity invalidates the payload everywhere with no extra writes. The chat shows the recap under the last message after five quiet minutes and the suggestion as a dismissible chip above the composer (tap fills the input, never sends). Gated by a new Chat setting (default on) that is a hard generation switch. Language is anchored to the conversation itself, with a script-mismatch guard against model/backend language hallucination. - TTS: a third input mode, summarized — long replies are condensed to spoken prose before playback on any TTS engine. - Git: commit-message and PR generation moved off the active chat session onto the small model fed with real diffs and the commit list (bodies included), with a session-transport fallback for free-model-only setups. - Notes: Add to notes distills long selections into 1-3 dense sentences preserving exact identifiers, with verbatim fallback on failure. Fixes along the way: - The global event watcher now starts unconditionally; it was gated behind the desktop-notify env, leaving the server-side event hub dead in packaged apps. - OpenCode re-emits message.updated for old user messages after idle; the watcher no longer mistakes those for new activity. - Session metadata merges from a fresh read right before the PATCH, so writes made during the generation window (suggestion dismissals, review links) are preserved; the assist runtime stops during graceful shutdown.
This commit is contained in:
Bohdan Triapitsyn
committed by
GitHub
GitHub
parent
e5b03493da
commit
28f0736d69
@@ -0,0 +1,78 @@
|
||||
# Small Model
|
||||
|
||||
Server-side direct LLM calls that reuse the user's existing OpenCode provider
|
||||
logins (`~/.local/share/opencode/auth.json`). OpenCode uses a "small model"
|
||||
internally (titles, summaries) but does not expose it through the SDK or
|
||||
plugins — this module replicates that mechanism as an OpenChamber runtime API.
|
||||
|
||||
## Security boundary
|
||||
|
||||
Credentials never leave the server process. The client sends only a prompt;
|
||||
auth resolution, OAuth refresh, and provider dispatch all happen server-side.
|
||||
Routes live under `/api/*` and are gated by the ui-auth middleware like every
|
||||
other runtime API.
|
||||
|
||||
## Files
|
||||
|
||||
- `index.js` — orchestration: `generateSmallModelText()` / `describeSmallModel()`.
|
||||
- `resolve.js` — model selection, mirroring OpenCode's `getSmallModel` chain:
|
||||
0. OpenChamber's own settings override (Settings → Sessions → Small Model):
|
||||
when `smallModelUseDefault` is `false`, `smallModelOverride`
|
||||
(`provider/model`) outranks everything below. Sanitized in
|
||||
`settings-helpers.js` (server), `persistence.ts` (client), and
|
||||
`bridge-settings-runtime.ts` (VS Code).
|
||||
1. `small_model` from the merged OpenCode config layers (`provider/model`).
|
||||
2. Family-priority scan (`gemini-flash` → `gpt-nano` → `claude-haiku`)
|
||||
**within the session's provider first** (`preferredProviderID`, like
|
||||
OpenCode resolves within the current provider), then over the other
|
||||
providers with a usable auth entry, newest `release_date` first.
|
||||
3. GitHub Copilot hidden utility models (`gpt-*-nano/mini`) — these never
|
||||
appear in the catalog, so they participate as the `gpt-nano` family entry
|
||||
and as a final utility fallback.
|
||||
4. Last resort: the session's own model (`preferredModelID`) when no small
|
||||
model resolves anywhere — costlier, but always valid.
|
||||
- Input clamp: the prompt is truncated to the resolved model's catalog
|
||||
`limit.context` (minus an output reserve, ~4 chars/token estimate;
|
||||
conservative default when the model is not in the catalog). Truncation is
|
||||
reported as `inputTruncated: true` in the response.
|
||||
- `call.js` — wire formats and per-provider auth, replicating OpenCode's
|
||||
plugin auth loaders:
|
||||
- **GitHub Copilot**: OpenAI-compatible `/chat/completions` on
|
||||
`https://api.githubcopilot.com` (or `copilot-api.<enterprise>`) with the
|
||||
stored device-OAuth token as the bearer — no token exchange, no expiry.
|
||||
- **OpenAI OAuth (ChatGPT plan)**: streaming Responses API on
|
||||
`https://chatgpt.com/backend-api/codex/responses` with
|
||||
`ChatGPT-Account-Id`; expired tokens are refreshed against
|
||||
`auth.openai.com` (single-flight) and written back to `auth.json`.
|
||||
- **Anthropic** (`type: api`): `/v1/messages` with `x-api-key`.
|
||||
- **Google** (`type: api`): `generateContent` with `x-goog-api-key`.
|
||||
- Everything else: OpenAI-compatible `/chat/completions` against the
|
||||
provider's models.dev base URL with `Authorization: Bearer <key>`.
|
||||
- `catalog.js` — models.dev catalog via the shared in-process cache
|
||||
(`../opencode/models-metadata.js`, also serving
|
||||
`/api/openchamber/models-metadata`).
|
||||
- `routes.js` — `GET /api/small-model` (resolution preview) and
|
||||
`POST /api/small-model/generate` (`{ prompt, system?, maxOutputTokens?,
|
||||
model?, directory? }` → `{ text, providerID, modelID, source }`).
|
||||
|
||||
## Registration
|
||||
|
||||
Mounted lazily from `feature-routes-runtime.js` (same pattern as quota): the
|
||||
module is imported on first request, not at server startup.
|
||||
|
||||
## Known limitations
|
||||
|
||||
- OpenCode's free models (`opencode/big-pickle`, `*-free`) work without a
|
||||
token only through OpenCode's own server — direct calls are rejected, and
|
||||
piggybacking on their subsidized infra is out of bounds by design. Every
|
||||
resolution step therefore requires a usable auth entry for the provider:
|
||||
a session on an unauthenticated `opencode` provider falls through to the
|
||||
global scan (or a clean 404 on a vanilla setup with no logins).
|
||||
|
||||
- Anthropic OAuth (Claude Pro/Max) entries are not supported — OpenCode itself
|
||||
keeps those outside `auth.json` in this generation; only `type: api` keys
|
||||
work for Anthropic.
|
||||
- Amazon Bedrock, GitLab, Azure and other credential-chain providers are out
|
||||
of scope; they need more than a key/token (regions, resource names).
|
||||
- Responses from the codex backend are collected from the SSE stream; the
|
||||
endpoint itself is non-streaming by design (small utility calls).
|
||||
@@ -0,0 +1,380 @@
|
||||
import { readAuthFile, writeAuthFile } from '../opencode/auth.js';
|
||||
import { getCatalogProvider } from './catalog.js';
|
||||
import { getAuthEntryForProvider } from './resolve.js';
|
||||
|
||||
// Direct, non-streaming text generation against the provider APIs, replicating
|
||||
// how OpenCode authenticates each of them (see the plugin auth loaders in the
|
||||
// opencode repo). auth.json credentials never leave this process.
|
||||
|
||||
const REQUEST_TIMEOUT_MS = 60_000;
|
||||
// Generous default: thinking models that can't be switched off (DeepSeek,
|
||||
// Qwen, …) spend part of this budget on reasoning before the actual answer.
|
||||
const DEFAULT_MAX_OUTPUT_TOKENS = 4_000;
|
||||
|
||||
const USER_AGENT = 'opencode/1.0 openchamber';
|
||||
|
||||
const CODEX_TOKEN_URL = 'https://auth.openai.com/oauth/token';
|
||||
const CODEX_CLIENT_ID = 'app_EMoamEEZ73f0CkXaXp7hrann';
|
||||
const CODEX_RESPONSES_URL = 'https://chatgpt.com/backend-api/codex/responses';
|
||||
|
||||
const httpError = async (response, provider) => {
|
||||
const body = await response.text().catch(() => '');
|
||||
const snippet = body ? `: ${body.slice(0, 300)}` : '';
|
||||
return new Error(`${provider} request failed with ${response.status}${snippet}`);
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// OpenAI OAuth (ChatGPT plan / codex) token refresh — single-flight, with the
|
||||
// refreshed token written back to auth.json exactly like OpenCode does.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
let openaiRefreshPromise = null;
|
||||
|
||||
const decodeJwtClaims = (token) => {
|
||||
try {
|
||||
const payload = token.split('.')[1];
|
||||
return JSON.parse(Buffer.from(payload, 'base64url').toString('utf8'));
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
const extractChatgptAccountId = (accessToken) => {
|
||||
const claims = decodeJwtClaims(accessToken);
|
||||
const auth = claims?.['https://api.openai.com/auth'];
|
||||
const value = auth?.chatgpt_account_id;
|
||||
return typeof value === 'string' && value ? value : null;
|
||||
};
|
||||
|
||||
const refreshOpenaiOauth = async (entry) => {
|
||||
if (!openaiRefreshPromise) {
|
||||
openaiRefreshPromise = (async () => {
|
||||
const response = await fetch(CODEX_TOKEN_URL, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
grant_type: 'refresh_token',
|
||||
refresh_token: entry.refresh,
|
||||
client_id: CODEX_CLIENT_ID,
|
||||
}),
|
||||
signal: AbortSignal.timeout(30_000),
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw await httpError(response, 'OpenAI token refresh');
|
||||
}
|
||||
const payload = await response.json();
|
||||
const access = typeof payload?.access_token === 'string' ? payload.access_token : '';
|
||||
if (!access) {
|
||||
throw new Error('OpenAI token refresh returned no access token');
|
||||
}
|
||||
const refreshed = {
|
||||
...entry,
|
||||
type: 'oauth',
|
||||
access,
|
||||
refresh: typeof payload?.refresh_token === 'string' && payload.refresh_token
|
||||
? payload.refresh_token
|
||||
: entry.refresh,
|
||||
expires: Date.now() + (Number(payload?.expires_in) > 0 ? Number(payload.expires_in) : 3600) * 1000,
|
||||
};
|
||||
const auth = readAuthFile();
|
||||
auth.openai = refreshed;
|
||||
writeAuthFile(auth);
|
||||
return refreshed;
|
||||
})().finally(() => {
|
||||
openaiRefreshPromise = null;
|
||||
});
|
||||
}
|
||||
return openaiRefreshPromise;
|
||||
};
|
||||
|
||||
const ensureFreshOpenaiOauth = async (entry) => {
|
||||
if (entry.access && Number(entry.expires) > Date.now()) {
|
||||
return entry;
|
||||
}
|
||||
if (!entry.refresh) {
|
||||
throw new Error('OpenAI OAuth entry has no refresh token');
|
||||
}
|
||||
return refreshOpenaiOauth(entry);
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Wire formats
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const callOpenaiCompatible = async ({ baseURL, headers, modelID, prompt, system, maxOutputTokens, providerLabel, extraBody }) => {
|
||||
const trimmedBase = baseURL.replace(/\/+$/, '');
|
||||
const response = await fetch(`${trimmedBase}/chat/completions`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
Accept: 'application/json',
|
||||
...headers,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: modelID,
|
||||
messages: [
|
||||
...(system ? [{ role: 'system', content: system }] : []),
|
||||
{ role: 'user', content: prompt },
|
||||
],
|
||||
max_tokens: maxOutputTokens,
|
||||
stream: false,
|
||||
...(extraBody || {}),
|
||||
}),
|
||||
signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw await httpError(response, providerLabel);
|
||||
}
|
||||
const payload = await response.json();
|
||||
const message = payload?.choices?.[0]?.message;
|
||||
|
||||
// Providers disagree on the content shape: plain string, an array of
|
||||
// typed parts, or (thinking models) an empty content with the budget spent
|
||||
// on reasoning_content.
|
||||
let text = '';
|
||||
if (typeof message?.content === 'string') {
|
||||
text = message.content;
|
||||
} else if (Array.isArray(message?.content)) {
|
||||
text = message.content
|
||||
.map((part) => (typeof part?.text === 'string' ? part.text : ''))
|
||||
.join('');
|
||||
}
|
||||
if (!text.trim() && typeof message?.reasoning_content === 'string' && message.reasoning_content.trim()) {
|
||||
const finishReason = payload?.choices?.[0]?.finish_reason;
|
||||
throw new Error(
|
||||
`${providerLabel} spent the output budget on reasoning and returned no answer`
|
||||
+ (finishReason ? ` (finish_reason: ${finishReason})` : ''),
|
||||
);
|
||||
}
|
||||
if (!text.trim()) {
|
||||
throw new Error(`${providerLabel} returned no message content`);
|
||||
}
|
||||
return text;
|
||||
};
|
||||
|
||||
const callAnthropic = async ({ apiKey, modelID, prompt, system, maxOutputTokens }) => {
|
||||
const response = await fetch('https://api.anthropic.com/v1/messages', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
Accept: 'application/json',
|
||||
'x-api-key': apiKey,
|
||||
'anthropic-version': '2023-06-01',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: modelID,
|
||||
max_tokens: maxOutputTokens,
|
||||
...(system ? { system } : {}),
|
||||
messages: [{ role: 'user', content: prompt }],
|
||||
}),
|
||||
signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw await httpError(response, 'Anthropic');
|
||||
}
|
||||
const payload = await response.json();
|
||||
const text = (payload?.content || [])
|
||||
.filter((part) => part?.type === 'text' && typeof part.text === 'string')
|
||||
.map((part) => part.text)
|
||||
.join('');
|
||||
if (!text) {
|
||||
throw new Error('Anthropic returned no text content');
|
||||
}
|
||||
return text;
|
||||
};
|
||||
|
||||
const callGoogle = async ({ apiKey, modelID, prompt, system, maxOutputTokens }) => {
|
||||
const url = `https://generativelanguage.googleapis.com/v1beta/models/${encodeURIComponent(modelID)}:generateContent`;
|
||||
const response = await fetch(url, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
Accept: 'application/json',
|
||||
'x-goog-api-key': apiKey,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
contents: [{ role: 'user', parts: [{ text: prompt }] }],
|
||||
...(system ? { systemInstruction: { parts: [{ text: system }] } } : {}),
|
||||
// thinkingBudget 0 switches Gemini Flash thinking off; Flash is the only
|
||||
// family the small-model resolver picks for Google.
|
||||
generationConfig: { maxOutputTokens, thinkingConfig: { thinkingBudget: 0 } },
|
||||
}),
|
||||
signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw await httpError(response, 'Google');
|
||||
}
|
||||
const payload = await response.json();
|
||||
const text = (payload?.candidates?.[0]?.content?.parts || [])
|
||||
.map((part) => (typeof part?.text === 'string' ? part.text : ''))
|
||||
.join('');
|
||||
if (!text) {
|
||||
throw new Error('Google returned no text content');
|
||||
}
|
||||
return text;
|
||||
};
|
||||
|
||||
// ChatGPT-plan traffic goes to the codex backend, which only speaks the
|
||||
// streaming Responses API — collect the output_text deltas from the SSE body.
|
||||
const callCodexResponses = async ({ accessToken, accountId, modelID, prompt, system }) => {
|
||||
const response = await fetch(CODEX_RESPONSES_URL, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
Accept: 'text/event-stream',
|
||||
Authorization: `Bearer ${accessToken}`,
|
||||
...(accountId ? { 'ChatGPT-Account-Id': accountId } : {}),
|
||||
originator: 'opencode',
|
||||
'User-Agent': USER_AGENT,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: modelID,
|
||||
...(system ? { instructions: system } : {}),
|
||||
input: [
|
||||
{
|
||||
type: 'message',
|
||||
role: 'user',
|
||||
content: [{ type: 'input_text', text: prompt }],
|
||||
},
|
||||
],
|
||||
// The codex backend rejects max_output_tokens (OpenCode forces it to
|
||||
// undefined for this provider too).
|
||||
stream: true,
|
||||
store: false,
|
||||
}),
|
||||
signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw await httpError(response, 'OpenAI (ChatGPT plan)');
|
||||
}
|
||||
|
||||
const raw = await response.text();
|
||||
let text = '';
|
||||
let completedText = '';
|
||||
for (const line of raw.split('\n')) {
|
||||
if (!line.startsWith('data:')) continue;
|
||||
const data = line.slice(5).trim();
|
||||
if (!data || data === '[DONE]') continue;
|
||||
let event;
|
||||
try {
|
||||
event = JSON.parse(data);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (event?.type === 'response.output_text.delta' && typeof event.delta === 'string') {
|
||||
text += event.delta;
|
||||
}
|
||||
if (event?.type === 'response.output_text.done' && typeof event.text === 'string') {
|
||||
completedText = event.text;
|
||||
}
|
||||
if (event?.type === 'response.failed' || event?.type === 'error') {
|
||||
const message = event?.response?.error?.message || event?.message || 'response failed';
|
||||
throw new Error(`OpenAI (ChatGPT plan) stream error: ${message}`);
|
||||
}
|
||||
}
|
||||
const result = completedText || text;
|
||||
if (!result) {
|
||||
throw new Error('OpenAI (ChatGPT plan) returned no text output');
|
||||
}
|
||||
return result;
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Dispatch
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export async function callSmallModel({ auth, catalog, providerID, modelID, prompt, system, maxOutputTokens }) {
|
||||
const tokens = Number(maxOutputTokens) > 0 ? Number(maxOutputTokens) : DEFAULT_MAX_OUTPUT_TOKENS;
|
||||
const entry = getAuthEntryForProvider(auth, providerID);
|
||||
if (!entry) {
|
||||
throw new Error(`No OpenCode login found for provider "${providerID}"`);
|
||||
}
|
||||
|
||||
if (providerID === 'github-copilot') {
|
||||
// OpenCode uses the stored device-OAuth token directly as the bearer —
|
||||
// access === refresh, no exchange, no expiry.
|
||||
const token = entry.refresh || entry.access || entry.key;
|
||||
if (!token) {
|
||||
throw new Error('GitHub Copilot login has no token');
|
||||
}
|
||||
const baseURL = entry.enterpriseUrl
|
||||
? `https://copilot-api.${String(entry.enterpriseUrl).replace(/^https?:\/\//, '').replace(/\/+$/, '')}`
|
||||
: 'https://api.githubcopilot.com';
|
||||
return callOpenaiCompatible({
|
||||
baseURL,
|
||||
headers: {
|
||||
Authorization: `Bearer ${token}`,
|
||||
'User-Agent': USER_AGENT,
|
||||
'Openai-Intent': 'conversation-edits',
|
||||
'x-initiator': 'agent',
|
||||
'X-GitHub-Api-Version': '2026-06-01',
|
||||
},
|
||||
modelID,
|
||||
prompt,
|
||||
system,
|
||||
maxOutputTokens: tokens,
|
||||
providerLabel: 'GitHub Copilot',
|
||||
});
|
||||
}
|
||||
|
||||
if (providerID === 'openai' && entry.type === 'oauth') {
|
||||
const fresh = await ensureFreshOpenaiOauth(entry);
|
||||
return callCodexResponses({
|
||||
accessToken: fresh.access,
|
||||
accountId: fresh.accountId || extractChatgptAccountId(fresh.access),
|
||||
modelID,
|
||||
prompt,
|
||||
system,
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = entry.type === 'api' ? entry.key
|
||||
: entry.type === 'wellknown' ? entry.token
|
||||
: entry.access;
|
||||
if (!apiKey) {
|
||||
throw new Error(`OpenCode login for "${providerID}" has no usable credential`);
|
||||
}
|
||||
|
||||
if (providerID === 'anthropic') {
|
||||
return callAnthropic({ apiKey, modelID, prompt, system, maxOutputTokens: tokens });
|
||||
}
|
||||
if (providerID === 'google') {
|
||||
return callGoogle({ apiKey, modelID, prompt, system, maxOutputTokens: tokens });
|
||||
}
|
||||
|
||||
// Everything else: OpenAI-compatible chat completions against the catalog's
|
||||
// base URL for that provider (openai itself included).
|
||||
const provider = getCatalogProvider(catalog, providerID);
|
||||
const baseURL = providerID === 'openai'
|
||||
? 'https://api.openai.com/v1'
|
||||
: typeof provider?.api === 'string' && provider.api
|
||||
? provider.api
|
||||
: null;
|
||||
if (!baseURL) {
|
||||
throw new Error(`Provider "${providerID}" has no known API base URL`);
|
||||
}
|
||||
|
||||
// Thinking models burn the output budget on reasoning and leave content
|
||||
// empty — disable thinking where a wire-format switch exists (mirrors
|
||||
// OpenCode's smallOptions/variants special cases). There is NO universal
|
||||
// parameter: unknown body fields 400 on some providers, so this stays an
|
||||
// explicit allowlist. Models without a switch (DeepSeek, Qwen, Kimi, …)
|
||||
// just get the generous output budget.
|
||||
const lowerModel = modelID.toLowerCase();
|
||||
const supportsThinkingToggle = providerID.includes('zai')
|
||||
|| providerID.includes('zhipu')
|
||||
|| lowerModel.includes('glm')
|
||||
|| lowerModel.includes('minimax-m3');
|
||||
const extraBody = supportsThinkingToggle ? { thinking: { type: 'disabled' } } : undefined;
|
||||
|
||||
return callOpenaiCompatible({
|
||||
baseURL,
|
||||
headers: { Authorization: `Bearer ${apiKey}` },
|
||||
modelID,
|
||||
prompt,
|
||||
system,
|
||||
maxOutputTokens: tokens,
|
||||
providerLabel: provider?.name || providerID,
|
||||
extraBody,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
import { getModelsMetadata } from '../opencode/models-metadata.js';
|
||||
|
||||
// The models.dev catalog is shared with the /api/openchamber/models-metadata
|
||||
// route through one in-process cache — no extra fetches, no cache files.
|
||||
export async function getModelCatalog() {
|
||||
const { metadata } = await getModelsMetadata();
|
||||
return metadata;
|
||||
}
|
||||
|
||||
export function getCatalogProvider(catalog, providerID) {
|
||||
const entry = catalog?.[providerID];
|
||||
return entry && typeof entry === 'object' ? entry : null;
|
||||
}
|
||||
@@ -0,0 +1,167 @@
|
||||
import fs from 'fs';
|
||||
import os from 'os';
|
||||
import path from 'path';
|
||||
import { readAuthFile } from '../opencode/auth.js';
|
||||
import { readConfigLayers } from '../opencode/shared.js';
|
||||
import { getModelCatalog } from './catalog.js';
|
||||
import { resolveSmallModel, parseModelRef, isUsableAuthEntry, getAuthEntryForProvider } from './resolve.js';
|
||||
import { callSmallModel } from './call.js';
|
||||
|
||||
const OPENCHAMBER_SETTINGS_FILE = path.join(
|
||||
process.env.OPENCHAMBER_DATA_DIR
|
||||
? path.resolve(process.env.OPENCHAMBER_DATA_DIR)
|
||||
: path.join(os.homedir(), '.config', 'openchamber'),
|
||||
'settings.json',
|
||||
);
|
||||
|
||||
// OpenChamber's own settings: when the user unchecks "use default small model"
|
||||
// their explicit override outranks every other resolution step.
|
||||
const readSmallModelSettingsOverride = () => {
|
||||
try {
|
||||
const raw = fs.readFileSync(OPENCHAMBER_SETTINGS_FILE, 'utf8');
|
||||
const settings = JSON.parse(raw);
|
||||
if (!settings || typeof settings !== 'object') return null;
|
||||
if (settings.smallModelUseDefault !== false) return null;
|
||||
const override = typeof settings.smallModelOverride === 'string' ? settings.smallModelOverride.trim() : '';
|
||||
return override || null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
// Rough safety clamp so a huge input never blows the model's context window.
|
||||
// Token estimate is ~4 chars/token; when the catalog has no limit for the
|
||||
// model (Copilot/codex utility models are not listed) a conservative default
|
||||
// applies.
|
||||
const DEFAULT_CONTEXT_TOKENS = 64_000;
|
||||
const OUTPUT_RESERVE_TOKENS = 4_000;
|
||||
|
||||
const clampPromptToModelLimit = ({ prompt, catalog, providerID, modelID }) => {
|
||||
const limit = catalog?.[providerID]?.models?.[modelID]?.limit;
|
||||
const contextTokens = Number(limit?.context) > 0 ? Number(limit.context) : DEFAULT_CONTEXT_TOKENS;
|
||||
const inputBudgetTokens = Math.max(1_000, contextTokens - OUTPUT_RESERVE_TOKENS);
|
||||
const maxChars = inputBudgetTokens * 4;
|
||||
if (prompt.length <= maxChars) {
|
||||
return { prompt, truncated: false };
|
||||
}
|
||||
return { prompt: `${prompt.slice(0, maxChars)}…`, truncated: true };
|
||||
};
|
||||
|
||||
const readConfiguredSmallModel = (workingDirectory) => {
|
||||
try {
|
||||
const { mergedConfig } = readConfigLayers(workingDirectory);
|
||||
const value = mergedConfig?.small_model;
|
||||
return typeof value === 'string' ? value : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Generates text with the user's small model, resolved and authenticated
|
||||
* entirely server-side from the OpenCode config and auth store.
|
||||
*/
|
||||
export async function generateSmallModelText({ prompt, system, maxOutputTokens, model, directory, preferredProviderID, preferredModelID, restrictToPreferredProvider = false }) {
|
||||
if (typeof prompt !== 'string' || !prompt.trim()) {
|
||||
throw Object.assign(new Error('prompt is required'), { statusCode: 400 });
|
||||
}
|
||||
|
||||
const auth = readAuthFile();
|
||||
const catalog = await getModelCatalog().catch(() => ({}));
|
||||
|
||||
const explicit = parseModelRef(model);
|
||||
const resolved = explicit
|
||||
? { ...explicit, source: 'request' }
|
||||
: resolveSmallModel({
|
||||
auth,
|
||||
catalog,
|
||||
settingsSmallModel: readSmallModelSettingsOverride(),
|
||||
configSmallModel: readConfiguredSmallModel(directory),
|
||||
preferredProviderID,
|
||||
preferredModelID,
|
||||
});
|
||||
|
||||
if (!resolved) {
|
||||
throw Object.assign(
|
||||
new Error('No small model available — no authenticated provider has a suitable model'),
|
||||
{ statusCode: 404 },
|
||||
);
|
||||
}
|
||||
|
||||
// Callers with a session context can forbid silently switching providers:
|
||||
// an explicit user choice (settings override, opencode config, request
|
||||
// model) is always allowed, anything else must stay on the session's
|
||||
// provider.
|
||||
if (restrictToPreferredProvider
|
||||
&& !['settings', 'config', 'request'].includes(resolved.source)
|
||||
&& resolved.providerID !== preferredProviderID) {
|
||||
throw Object.assign(
|
||||
new Error('No small model available within the session provider'),
|
||||
{ statusCode: 404 },
|
||||
);
|
||||
}
|
||||
|
||||
const clamped = clampPromptToModelLimit({
|
||||
prompt: prompt.trim(),
|
||||
catalog,
|
||||
providerID: resolved.providerID,
|
||||
modelID: resolved.modelID,
|
||||
});
|
||||
|
||||
const text = await callSmallModel({
|
||||
auth,
|
||||
catalog,
|
||||
providerID: resolved.providerID,
|
||||
modelID: resolved.modelID,
|
||||
prompt: clamped.prompt,
|
||||
system: typeof system === 'string' && system.trim() ? system.trim() : undefined,
|
||||
maxOutputTokens,
|
||||
});
|
||||
|
||||
return {
|
||||
text: text.trim(),
|
||||
providerID: resolved.providerID,
|
||||
modelID: resolved.modelID,
|
||||
source: resolved.source,
|
||||
...(clamped.truncated ? { inputTruncated: true } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Provider ids with a usable OpenCode login — the set the small model can
|
||||
* actually call. Used by the settings override picker to hide providers that
|
||||
* would only ever fail (e.g. opencode free models without a token).
|
||||
*/
|
||||
export function listAuthenticatedProviders() {
|
||||
try {
|
||||
const auth = readAuthFile();
|
||||
const ids = new Set(
|
||||
Object.keys(auth || {}).filter((providerID) => isUsableAuthEntry(auth[providerID])),
|
||||
);
|
||||
// The catalog id is github-copilot while legacy auth entries may sit
|
||||
// under the copilot alias.
|
||||
if (isUsableAuthEntry(getAuthEntryForProvider(auth, 'github-copilot'))) {
|
||||
ids.add('github-copilot');
|
||||
}
|
||||
return Array.from(ids);
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Reports which model would be used, without calling it.
|
||||
*/
|
||||
export async function describeSmallModel({ directory, preferredProviderID, preferredModelID } = {}) {
|
||||
const auth = readAuthFile();
|
||||
const catalog = await getModelCatalog().catch(() => ({}));
|
||||
const resolved = resolveSmallModel({
|
||||
auth,
|
||||
catalog,
|
||||
settingsSmallModel: readSmallModelSettingsOverride(),
|
||||
configSmallModel: readConfiguredSmallModel(directory),
|
||||
preferredProviderID,
|
||||
preferredModelID,
|
||||
});
|
||||
return resolved;
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
import { getCatalogProvider } from './catalog.js';
|
||||
|
||||
// Mirrors OpenCode's getSmallModel fallback chain:
|
||||
// 1. `small_model` from the merged config layers ("provider/model").
|
||||
// 2. GitHub Copilot's hidden utility models when Copilot is logged in.
|
||||
// 3. Family-priority scan of the authenticated providers' catalog models.
|
||||
const FAMILY_PRIORITY = ['gemini-flash', 'gpt-nano', 'claude-haiku'];
|
||||
const COPILOT_UTILITY_MODELS = ['gpt-5.4-nano', 'gpt-4.1', 'gpt-4o', 'gpt-4o-mini'];
|
||||
// The ChatGPT-plan codex backend only accepts a small allowlist of models
|
||||
// (nano/API-key models are rejected with 400) — this is its cheapest one.
|
||||
const OPENAI_OAUTH_SMALL_MODEL = 'gpt-5.4-mini';
|
||||
|
||||
const AUTH_PROVIDER_ALIASES = {
|
||||
'github-copilot': ['github-copilot', 'copilot'],
|
||||
};
|
||||
|
||||
export function getAuthEntryForProvider(auth, providerID) {
|
||||
const aliases = AUTH_PROVIDER_ALIASES[providerID] || [providerID];
|
||||
for (const alias of aliases) {
|
||||
const entry = auth?.[alias];
|
||||
if (entry && typeof entry === 'object') {
|
||||
return entry;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export function isUsableAuthEntry(entry) {
|
||||
if (!entry || typeof entry !== 'object') return false;
|
||||
if (entry.type === 'api') return typeof entry.key === 'string' && entry.key.length > 0;
|
||||
if (entry.type === 'oauth') {
|
||||
return (typeof entry.access === 'string' && entry.access.length > 0)
|
||||
|| (typeof entry.refresh === 'string' && entry.refresh.length > 0);
|
||||
}
|
||||
if (entry.type === 'wellknown') return typeof entry.token === 'string' && entry.token.length > 0;
|
||||
return false;
|
||||
}
|
||||
|
||||
export function parseModelRef(value) {
|
||||
if (typeof value !== 'string') return null;
|
||||
const trimmed = value.trim();
|
||||
const slash = trimmed.indexOf('/');
|
||||
if (slash <= 0 || slash === trimmed.length - 1) return null;
|
||||
return {
|
||||
providerID: trimmed.slice(0, slash),
|
||||
modelID: trimmed.slice(slash + 1),
|
||||
};
|
||||
}
|
||||
|
||||
const pickByFamily = (models, family) => {
|
||||
const matches = Object.values(models)
|
||||
.filter((model) => model && typeof model === 'object' && model.family === family);
|
||||
if (matches.length === 0) return null;
|
||||
matches.sort((a, b) => String(b.release_date || '').localeCompare(String(a.release_date || '')));
|
||||
return matches[0];
|
||||
};
|
||||
|
||||
// Small-model candidates within ONE provider, by family priority. Copilot and
|
||||
// ChatGPT-plan OpenAI have fixed small models that never appear in the
|
||||
// catalog; everyone else is scanned through the catalog families.
|
||||
const pickWithinProvider = (providerID, auth, catalog, family) => {
|
||||
if (providerID === 'openai' && auth.openai?.type === 'oauth') {
|
||||
return family === 'gpt-nano'
|
||||
? { providerID, modelID: OPENAI_OAUTH_SMALL_MODEL, source: 'codex-small' }
|
||||
: null;
|
||||
}
|
||||
if (providerID === 'github-copilot') {
|
||||
return family === 'gpt-nano'
|
||||
? { providerID, modelID: COPILOT_UTILITY_MODELS[0], source: 'copilot-utility' }
|
||||
: null;
|
||||
}
|
||||
const provider = getCatalogProvider(catalog, providerID);
|
||||
if (!provider || !provider.models || typeof provider.models !== 'object') return null;
|
||||
const model = pickByFamily(provider.models, family);
|
||||
return model?.id ? { providerID, modelID: model.id, source: 'family-scan' } : null;
|
||||
};
|
||||
|
||||
export function resolveSmallModel({ auth, catalog, settingsSmallModel, configSmallModel, preferredProviderID, preferredModelID }) {
|
||||
// OpenChamber's own setting (Settings → Sessions → Small Model override)
|
||||
// outranks everything, including the OpenCode config.
|
||||
const fromSettings = parseModelRef(settingsSmallModel);
|
||||
if (fromSettings) {
|
||||
return { ...fromSettings, source: 'settings' };
|
||||
}
|
||||
|
||||
const explicit = parseModelRef(configSmallModel);
|
||||
if (explicit) {
|
||||
return { ...explicit, source: 'config' };
|
||||
}
|
||||
|
||||
// Like OpenCode: when the caller has a session context, the utility call
|
||||
// stays on the session's provider. Scan its families for a small model,
|
||||
// otherwise run on the session's own model — never silently switch to a
|
||||
// different provider's subscription.
|
||||
const preferred = typeof preferredProviderID === 'string' && preferredProviderID
|
||||
? preferredProviderID
|
||||
: null;
|
||||
if (preferred && isUsableAuthEntry(getAuthEntryForProvider(auth, preferred))) {
|
||||
for (const family of FAMILY_PRIORITY) {
|
||||
const match = pickWithinProvider(preferred, auth, catalog, family);
|
||||
if (match) return match;
|
||||
}
|
||||
if (typeof preferredModelID === 'string' && preferredModelID) {
|
||||
return { providerID: preferred, modelID: preferredModelID, source: 'session-model' };
|
||||
}
|
||||
}
|
||||
|
||||
// No session context (or its provider has no usable login): scan all
|
||||
// authenticated providers by family priority.
|
||||
const authedProviders = Object.keys(auth || {}).filter((providerID) =>
|
||||
providerID !== preferred && isUsableAuthEntry(auth[providerID]));
|
||||
|
||||
for (const family of FAMILY_PRIORITY) {
|
||||
for (const providerID of authedProviders) {
|
||||
const match = pickWithinProvider(providerID, auth, catalog, family);
|
||||
if (match) return match;
|
||||
}
|
||||
}
|
||||
|
||||
// Copilot's utility fallback for legacy auth aliases the loop above missed.
|
||||
const copilotEntry = getAuthEntryForProvider(auth, 'github-copilot');
|
||||
if (isUsableAuthEntry(copilotEntry)) {
|
||||
return {
|
||||
providerID: 'github-copilot',
|
||||
modelID: COPILOT_UTILITY_MODELS[0],
|
||||
source: 'copilot-utility',
|
||||
};
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
@@ -0,0 +1,197 @@
|
||||
import { describe, it, expect } from 'bun:test';
|
||||
import { resolveSmallModel, parseModelRef, isUsableAuthEntry } from './resolve.js';
|
||||
|
||||
const catalog = {
|
||||
google: {
|
||||
id: 'google',
|
||||
models: {
|
||||
'gemini-2.5-flash': { id: 'gemini-2.5-flash', family: 'gemini-flash', release_date: '2025-06-01' },
|
||||
'gemini-2.0-flash': { id: 'gemini-2.0-flash', family: 'gemini-flash', release_date: '2024-12-01' },
|
||||
'gemini-2.5-pro': { id: 'gemini-2.5-pro', family: 'gemini-pro', release_date: '2025-06-01' },
|
||||
},
|
||||
},
|
||||
anthropic: {
|
||||
id: 'anthropic',
|
||||
models: {
|
||||
'claude-haiku-4-5': { id: 'claude-haiku-4-5', family: 'claude-haiku', release_date: '2025-10-01' },
|
||||
'claude-sonnet-4-5': { id: 'claude-sonnet-4-5', family: 'claude-sonnet', release_date: '2025-09-01' },
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
describe('parseModelRef', () => {
|
||||
it('splits provider/model on the first slash', () => {
|
||||
expect(parseModelRef('anthropic/claude-haiku-4-5')).toEqual({
|
||||
providerID: 'anthropic',
|
||||
modelID: 'claude-haiku-4-5',
|
||||
});
|
||||
});
|
||||
|
||||
it('keeps slashes inside the model id', () => {
|
||||
expect(parseModelRef('openrouter/google/gemini-2.5-flash')).toEqual({
|
||||
providerID: 'openrouter',
|
||||
modelID: 'google/gemini-2.5-flash',
|
||||
});
|
||||
});
|
||||
|
||||
it('rejects values without a provider or model part', () => {
|
||||
expect(parseModelRef('anthropic/')).toBeNull();
|
||||
expect(parseModelRef('/model')).toBeNull();
|
||||
expect(parseModelRef('plain')).toBeNull();
|
||||
expect(parseModelRef(undefined)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('isUsableAuthEntry', () => {
|
||||
it('accepts api keys, oauth tokens, and wellknown tokens', () => {
|
||||
expect(isUsableAuthEntry({ type: 'api', key: 'sk-x' })).toBe(true);
|
||||
expect(isUsableAuthEntry({ type: 'oauth', access: 'a', refresh: 'r', expires: 0 })).toBe(true);
|
||||
expect(isUsableAuthEntry({ type: 'wellknown', key: 'k', token: 't' })).toBe(true);
|
||||
});
|
||||
|
||||
it('rejects empty or malformed entries', () => {
|
||||
expect(isUsableAuthEntry({ type: 'api', key: '' })).toBe(false);
|
||||
expect(isUsableAuthEntry({ type: 'oauth' })).toBe(false);
|
||||
expect(isUsableAuthEntry(null)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveSmallModel', () => {
|
||||
it('gives the OpenChamber settings override top priority', () => {
|
||||
const result = resolveSmallModel({
|
||||
auth: { anthropic: { type: 'api', key: 'sk-x' } },
|
||||
catalog,
|
||||
settingsSmallModel: 'anthropic/claude-haiku-4-5',
|
||||
configSmallModel: 'openai/gpt-4o-mini',
|
||||
preferredProviderID: 'anthropic',
|
||||
});
|
||||
expect(result).toEqual({ providerID: 'anthropic', modelID: 'claude-haiku-4-5', source: 'settings' });
|
||||
});
|
||||
|
||||
it('prefers the configured small_model', () => {
|
||||
const result = resolveSmallModel({
|
||||
auth: { anthropic: { type: 'api', key: 'sk-x' } },
|
||||
catalog,
|
||||
configSmallModel: 'openai/gpt-4o-mini',
|
||||
});
|
||||
expect(result).toEqual({ providerID: 'openai', modelID: 'gpt-4o-mini', source: 'config' });
|
||||
});
|
||||
|
||||
it('scans authenticated providers by family priority, newest first', () => {
|
||||
const result = resolveSmallModel({
|
||||
auth: {
|
||||
google: { type: 'api', key: 'g-key' },
|
||||
anthropic: { type: 'api', key: 'sk-x' },
|
||||
},
|
||||
catalog,
|
||||
configSmallModel: null,
|
||||
});
|
||||
expect(result).toEqual({ providerID: 'google', modelID: 'gemini-2.5-flash', source: 'family-scan' });
|
||||
});
|
||||
|
||||
it('skips providers without a usable credential', () => {
|
||||
const result = resolveSmallModel({
|
||||
auth: {
|
||||
google: { type: 'api', key: '' },
|
||||
anthropic: { type: 'api', key: 'sk-x' },
|
||||
},
|
||||
catalog,
|
||||
configSmallModel: null,
|
||||
});
|
||||
expect(result).toEqual({ providerID: 'anthropic', modelID: 'claude-haiku-4-5', source: 'family-scan' });
|
||||
});
|
||||
|
||||
it('falls back to Copilot utility models when only Copilot is logged in', () => {
|
||||
const result = resolveSmallModel({
|
||||
auth: { 'github-copilot': { type: 'oauth', access: 't', refresh: 't', expires: 0 } },
|
||||
catalog,
|
||||
configSmallModel: null,
|
||||
});
|
||||
expect(result?.providerID).toBe('github-copilot');
|
||||
expect(result?.source).toBe('copilot-utility');
|
||||
});
|
||||
|
||||
it('returns null when nothing is authenticated', () => {
|
||||
expect(resolveSmallModel({ auth: {}, catalog, configSmallModel: null })).toBeNull();
|
||||
});
|
||||
|
||||
it('prefers the session provider over other authenticated providers', () => {
|
||||
const result = resolveSmallModel({
|
||||
auth: {
|
||||
google: { type: 'api', key: 'g-key' },
|
||||
anthropic: { type: 'api', key: 'sk-x' },
|
||||
},
|
||||
catalog,
|
||||
configSmallModel: null,
|
||||
preferredProviderID: 'anthropic',
|
||||
});
|
||||
expect(result).toEqual({ providerID: 'anthropic', modelID: 'claude-haiku-4-5', source: 'family-scan' });
|
||||
});
|
||||
|
||||
it('ignores a preferred provider without a usable login', () => {
|
||||
const result = resolveSmallModel({
|
||||
auth: { google: { type: 'api', key: 'g-key' } },
|
||||
catalog,
|
||||
configSmallModel: null,
|
||||
preferredProviderID: 'anthropic',
|
||||
});
|
||||
expect(result).toEqual({ providerID: 'google', modelID: 'gemini-2.5-flash', source: 'family-scan' });
|
||||
});
|
||||
|
||||
it('never uses a session provider without a login (opencode free models)', () => {
|
||||
// Vanilla setups default the picker to opencode/big-pickle with no
|
||||
// opencode token — those free models only work through OpenCode itself
|
||||
// and must never be called directly, so the session context is ignored.
|
||||
const result = resolveSmallModel({
|
||||
auth: { openai: { type: 'oauth', access: 'a', refresh: 'r', expires: Date.now() + 60_000 } },
|
||||
catalog,
|
||||
configSmallModel: null,
|
||||
preferredProviderID: 'opencode',
|
||||
preferredModelID: 'big-pickle',
|
||||
});
|
||||
expect(result).toEqual({ providerID: 'openai', modelID: 'gpt-5.4-mini', source: 'codex-small' });
|
||||
});
|
||||
|
||||
it('resolves nothing on a vanilla setup with no logins at all', () => {
|
||||
const result = resolveSmallModel({
|
||||
auth: {},
|
||||
catalog,
|
||||
configSmallModel: null,
|
||||
preferredProviderID: 'opencode',
|
||||
preferredModelID: 'big-pickle',
|
||||
});
|
||||
expect(result).toBeNull();
|
||||
});
|
||||
|
||||
it('falls back to the session model instead of scanning other providers', () => {
|
||||
const result = resolveSmallModel({
|
||||
auth: {
|
||||
'opencode-go': { type: 'api', key: 'oc-key' },
|
||||
openai: { type: 'oauth', access: 'a', refresh: 'r', expires: Date.now() + 60_000 },
|
||||
},
|
||||
catalog: {
|
||||
'opencode-go': {
|
||||
id: 'opencode-go',
|
||||
models: {
|
||||
'deepseek-v4-flash': { id: 'deepseek-v4-flash', family: 'deepseek-flash', release_date: '2026-01-01' },
|
||||
},
|
||||
},
|
||||
},
|
||||
configSmallModel: null,
|
||||
preferredProviderID: 'opencode-go',
|
||||
preferredModelID: 'deepseek-v4-flash',
|
||||
});
|
||||
expect(result).toEqual({ providerID: 'opencode-go', modelID: 'deepseek-v4-flash', source: 'session-model' });
|
||||
});
|
||||
|
||||
it('falls back to the session model itself when nothing resolves', () => {
|
||||
const result = resolveSmallModel({
|
||||
auth: { mistral: { type: 'api', key: 'm-key' } },
|
||||
catalog,
|
||||
configSmallModel: null,
|
||||
preferredProviderID: 'mistral',
|
||||
preferredModelID: 'mistral-large-latest',
|
||||
});
|
||||
expect(result).toEqual({ providerID: 'mistral', modelID: 'mistral-large-latest', source: 'session-model' });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,44 @@
|
||||
export function registerSmallModelRoutes(app, { getSmallModelService }) {
|
||||
app.get('/api/small-model', async (req, res) => {
|
||||
try {
|
||||
const { describeSmallModel, listAuthenticatedProviders } = await getSmallModelService();
|
||||
const resolved = await describeSmallModel({
|
||||
directory: typeof req.query.directory === 'string' ? req.query.directory : undefined,
|
||||
preferredProviderID: typeof req.query.providerID === 'string' ? req.query.providerID : undefined,
|
||||
preferredModelID: typeof req.query.modelID === 'string' ? req.query.modelID : undefined,
|
||||
});
|
||||
res.json({
|
||||
available: Boolean(resolved),
|
||||
model: resolved,
|
||||
authenticatedProviders: listAuthenticatedProviders(),
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Failed to resolve small model:', error);
|
||||
res.status(500).json({ error: error.message || 'Failed to resolve small model' });
|
||||
}
|
||||
});
|
||||
|
||||
app.post('/api/small-model/generate', async (req, res) => {
|
||||
try {
|
||||
const { generateSmallModelText } = await getSmallModelService();
|
||||
const { prompt, system, maxOutputTokens, model, directory, preferredProviderID, preferredModelID, restrictToPreferredProvider } = req.body || {};
|
||||
const result = await generateSmallModelText({
|
||||
prompt,
|
||||
system,
|
||||
maxOutputTokens,
|
||||
model,
|
||||
directory,
|
||||
preferredProviderID,
|
||||
preferredModelID,
|
||||
restrictToPreferredProvider: restrictToPreferredProvider === true,
|
||||
});
|
||||
res.json(result);
|
||||
} catch (error) {
|
||||
const statusCode = Number(error?.statusCode) || 500;
|
||||
if (statusCode >= 500) {
|
||||
console.error('Small model generation failed:', error);
|
||||
}
|
||||
res.status(statusCode).json({ error: error.message || 'Small model generation failed' });
|
||||
}
|
||||
});
|
||||
}
|
||||
Reference in New Issue
Block a user