BuzzbeeSCD/openchamber
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
728bf54825b1bf24141932b7faaddfa579288821
openchamber/packages/web/server/lib/small-model/DOCUMENTATION.md
T
Bohdan Triapitsyn 34d0ff7383 feat(walkthrough): guided AI walkthrough for diffs, branches, and PRs (#2572) 
A diff is ordered by file path, which is almost never the order in which a
change makes sense. This adds a Walkthrough surface that reorders it: the model
groups related hunks into stops, explains what each group changes about
behavior, and orders the stops so each builds on the last. It explains and
orders; judging code stays with the existing Review action.

Reviews uncommitted work (all, staged, unstaged), a branch against its base, or
a pull request. Generation is always user-initiated — nothing runs on a timer,
on a file change, or as a side effect of opening a panel.

Invariants worth preserving:

- Hunk identity is derived on the server and only there. Ids are content
  hashes, so an anchor that no longer resolves is proof the code it described
  changed, and staleness needs no heuristics. The client matches ids to ids and
  never recomputes them; two implementations would have to agree forever.
- The digest is never truncated. A diff that does not fit the model's context
  is refused with an actionable reason, because a walkthrough written against
  half a diff reads as confident and is wrong.
- Nothing disappears. Lockfiles and other generated output are excluded from
  the model's input by name — never by size — and everything no stop covers is
  listed at the end, so "have I seen all of it" stays answerable.
- Cost is explicit. Results are content-addressed, so returning the working
  tree to an earlier state costs nothing; generation outlives its request, so a
  refresh detaches the client rather than discarding paid-for work, and only an
  explicit cancel stops it.

Supporting changes to shared modules:

- git: expose the existing getRangeDiff as GET /api/git
  listUntrackedPaths and getUntrackedDiffs. The latter resolve the repository
  once for a batch instead of per file, taking a panel
  ~340ms on an 80-file working tree.
- small-model: structured output across four wire forma
  and abort signal, and an onOverflow policy so an oversized prompt fails
  loudly instead of being silently clipped. A provider
  remembered so the prompt-side fallback goes first next time.
- models.dev metadata: surface structured_output as tri
  false blocks a model, a missing field does not, because the catalog omits it
  for roughly half of all models.

Desktop and tablet only: VS Code serves Git through its
these routes, and the mobile shell does not consume the surface registry.

Docs: packages/docs walkthrough page in English and all eight locales.
2026-08-02 16:22:55 +03:00

7.6 KiB
Raw Blame History

Small Model

Server-side direct LLM calls that reuse the user's existing OpenCode provider logins (~/.local/share/opencode/auth.json). OpenCode uses a "small model" internally (titles, summaries) but does not expose it through the SDK or plugins — this module replicates that mechanism as an OpenChamber runtime API.

Security boundary

Credentials never leave the server process. The client sends only a prompt; auth resolution, OAuth refresh, and provider dispatch all happen server-side. Routes live under /api/* and are gated by the ui-auth middleware like every other runtime API.

Files

  • index.js — orchestration: generateSmallModelText() / describeSmallModel().
  • resolve.js — model selection, mirroring OpenCode's getSmallModel chain: 0. OpenChamber's own settings override (Settings → Sessions → Small Model): when smallModelUseDefault is false, smallModelOverride (provider/model) outranks everything below. Sanitized in settings-helpers.js (server), persistence.ts (client), and bridge-settings-runtime.ts (VS Code).
    1. small_model from the merged OpenCode config layers (provider/model).
    2. Family-priority scan (gemini-flash → gpt-nano → claude-haiku) within the session's provider first (preferredProviderID, like OpenCode resolves within the current provider), then over the other providers with a usable auth entry, newest release_date first.
    3. GitHub Copilot hidden utility models (gpt-*-nano/mini) — these never appear in the catalog, so they participate as the gpt-nano family entry and as a final utility fallback.
    4. Last resort: the session's own model (preferredModelID) when no small model resolves anywhere — costlier, but always valid.
  • Input clamp: the prompt is measured against the resolved model's catalog limit.context (minus an output reserve, ~4 chars/token estimate; conservative default when the model is not in the catalog). onOverflow decides what an oversized prompt means:
    • truncate (default) clips the tail and reports inputTruncated: true. Correct for callers that degrade gracefully (summaries, commit messages).
    • error throws a 413 with code: 'context-too-small' plus requiredChars/availableChars. Correct for callers whose output would be quietly wrong on a clipped input, so they can ask the user for a roomier model instead of returning confident nonsense.
  • Structured output: pass responseSchema (a JSON Schema) to get schema-shaped JSON back as text. Wire support differs per format — response_format: {type: 'json_schema'} for OpenAI-compatible chat, text.format for the Responses API, a forced single tool call for the Anthropic messages API, and generationConfig.responseSchema for Google (whose OpenAPI-flavored dialect drops unknown JSON Schema keywords). The ChatGPT-plan codex backend has no equivalent and rejects a schema request with code: 'structured-output-unsupported' rather than silently returning prose.
  • Output budget: maxOutputTokens is capped at the catalog's limit.output for the model, and the same number is reserved from the input allowance. The two must not drift — a caller that asks for a large answer while the reserve stays at the default overruns the context, and the failure looks like a truncation bug rather than a budgeting one. describeSmallModel takes outputReserveTokens so readiness checks agree with what generation will do.
  • Reasoning models can spend the entire output budget thinking and return nothing. That case (empty content with finish_reason: 'length', or content empty while reasoning_content is populated) throws with code: 'output-exhausted' so callers can offer a different model instead of showing a transport error.
  • timeoutMs overrides the 60s default per call; signal lets a caller abort a request that is no longer wanted. Both apply to every wire format.
  • describeSmallModel() additionally reports inputCharBudget, contextTokens, contextKnown, and structuredOutput. The last is tri-state: true/false from the catalog, null when the catalog omits the field — which it does for roughly half of all models, aggregators and proxies especially. Callers must treat null as "try it", not "unsupported".
  • call.js — wire formats and per-provider auth, replicating OpenCode's plugin auth loaders:
    • GitHub Copilot: fetches the requested model's authenticated /models metadata from https://api.githubcopilot.com (or copilot-api.<enterprise>) and honors its advertised endpoint, preferring Anthropic-compatible /v1/messages, then OpenAI /responses, then /chat/completions. Models without supported_endpoints retain the legacy Chat Completions default; metadata, missing-model, and unsupported-endpoint failures are surfaced instead of guessing. The stored device-OAuth token is used as the bearer with no token exchange or expiry.
    • OpenAI OAuth (ChatGPT plan): streaming Responses API on https://chatgpt.com/backend-api/codex/responses with ChatGPT-Account-Id; expired tokens are refreshed against auth.openai.com (single-flight) and written back to auth.json.
    • Anthropic (type: api): /v1/messages with x-api-key.
    • Google (type: api): generateContent with x-goog-api-key; Gemini 3 uses thinkingLevel while older Flash models use thinkingBudget: 0.
    • Everything else: OpenAI-compatible /chat/completions against the provider's base URL, resolved from (1) provider.<id>.options.baseURL in the OpenCode config, (2) the hardcoded https://api.openai.com/v1 endpoint, or (3) the provider's api field from the models.dev catalog. Configured API keys honor OpenCode's {env:NAME} and {file:path} substitutions; file contents and resolved credentials remain server-side.
    • [small-model:diagnostic] logs record provider/model, input character counts, output budget, thinking toggle, HTTP/finish status, and content/reasoning lengths without logging prompts, response text, or credentials. Goal audit parsing similarly emits [session-goal:diagnostic] structural verdict metadata.
  • catalog.js — models.dev catalog via the shared in-process cache (../opencode/models-metadata.js, also serving /api/openchamber/models-metadata).
  • routes.js — GET /api/small-model (resolution preview) and POST /api/small-model/generate ({ prompt, system?, maxOutputTokens?, model?, directory? } → { text, providerID, modelID, source }).

Registration

Mounted lazily from feature-routes-runtime.js (same pattern as quota): the module is imported on first request, not at server startup.

Known limitations

  • OpenCode's free models (opencode/big-pickle, *-free) work without a token only through OpenCode's own server — direct calls are rejected, and piggybacking on their subsidized infra is out of bounds by design. Every resolution step therefore requires a usable auth entry for the provider: a session on an unauthenticated opencode provider falls through to the global scan (or a clean 404 on a vanilla setup with no logins).

  • Anthropic OAuth (Claude Pro/Max) entries are not supported — OpenCode itself keeps those outside auth.json in this generation; only type: api keys work for Anthropic.

  • Amazon Bedrock, GitLab, Azure and other credential-chain providers are out of scope; they need more than a key/token (regions, resource names).

  • Responses from the codex backend are collected from the SSE stream; the endpoint itself is non-streaming by design (small utility calls).

Reference in New Issue View Git Blame Copy Permalink