Bohdan Triapitsyn
de1b85ac56
Complete rebuild of voice input on a server-authoritative streaming architecture, replacing the legacy Web Speech / whole-blob / WASM engines and the dead voice-agent layer (~4k lines removed). Speech-to-text (dictation): - Client streams 16 kHz mono PCM16 chunks over /api/dictation/ws with seq/ack ordering; buffered audio is retained and replayed on reconnect - Server transcribes and streams live partial transcripts back; segments auto-commit every ~15s with silence suppression and adaptive finalization timeouts - Local provider (default, zero config): sherpa-onnx models in a forked worker process — auto-download with progress, staged extraction with verification, corrupt-model auto-recovery, idle shutdown after 5 min - Model catalog with settings picker (accuracy/speed ratings, sizes, download/delete): Parakeet TDT v2 (English) and v3 (25 European languages, auto-detected), Whisper base and tiny (multilingual, light) - OpenAI-compatible provider for any Whisper endpoint - Composer overlay with live transcript, volume meter, timer, and cancel / insert / insert-and-send actions; failed transcriptions keep their audio for retry or accepting the partial text as-is - Configurable keyboard shortcut (default mod+alt+v) toggles dictation; Enter confirms and Escape cancels while recording - Overlay is pixel-aligned with the composer (measured footer height, matching paddings/typography/gaps) — no layout shift when toggling Text-to-speech: - Local Kokoro provider (English, 11 voices) synthesized in the same worker via /api/dictation/tts/speak, managed by the shared model pipeline; sentence-pipelined playback keeps time-to-first-audio at ~1 sentence regardless of message length, and stop cancels in-flight synthesis - Sanitizer keeps inline-code content (strips backticks only), reads interword slashes aloud, and removes only absolute file paths Settings: - Voice page unified: a single read-aloud toggle owns all playback options (the confusing "Enable Voice Mode" is gone); a new "Enable voice input" toggle (default on, persisted to settings.json) hides the composer mic entirely when disabled Mobile and transport: - iOS/Android microphone permissions added (dictation was previously impossible on mobile) - Fixed Android WebSocket upgrades: the Capacitor WebView origin (https://localhost) was missing from the packaged-client allowlist, 403-ing every WS connection — root cause of the old mobile SSE lock, which is now removed for all transports Security and conventions: - All HTTP routes sit behind the global /api auth gate; the WS upgrade explicitly validates the UI session and origin, with oc_url_token narrowly allowlisted and covered by tests; the dictation socket mints a fresh URL token before connecting - Routes register before the generic OpenCode proxy; the client goes through runtimeFetch/getRuntimeUrlResolver, and runtime switches reset the dictation socket - VS Code deliberately reports dictation as unavailable (no server process in that runtime) CI: workflow Node bumped 20 -> 22 to match the repo engines and fix better-sqlite3 installs broken by node-gyp@latest on Node 20. New dependency: sherpa-onnx-node (prebuilt N-API; macOS/Linux x64+arm64, Windows x64 — Windows-on-ARM falls back to the OpenAI-compatible provider)
61 lines
1.7 KiB
YAML
61 lines
1.7 KiB
YAML
name: Publish VS Code Extension
|
|
|
|
on:
|
|
push:
|
|
tags:
|
|
- 'v*'
|
|
workflow_dispatch:
|
|
|
|
permissions:
|
|
contents: write
|
|
|
|
jobs:
|
|
publish:
|
|
runs-on: ubuntu-latest
|
|
env:
|
|
VSCE_PAT: ${{ secrets.VSCE_PAT }}
|
|
OVSX_PAT: ${{ secrets.OVSX_PAT }}
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
|
|
|
- name: Setup bun
|
|
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
|
|
with:
|
|
node-version: '22'
|
|
|
|
- name: Install dependencies
|
|
run: bun install --frozen-lockfile
|
|
|
|
- name: Build VS Code extension
|
|
run: bun run --cwd packages/vscode build
|
|
|
|
- name: Package extension
|
|
run: cd packages/vscode && bunx vsce package --no-dependencies
|
|
|
|
- name: Publish to VS Code Marketplace
|
|
if: ${{ env.VSCE_PAT != '' }}
|
|
run: cd packages/vscode && bunx vsce publish -p "$VSCE_PAT" --no-dependencies
|
|
|
|
- name: Publish to Open VSX
|
|
if: ${{ env.OVSX_PAT != '' }}
|
|
run: bunx ovsx publish packages/vscode/*.vsix -p "$OVSX_PAT"
|
|
|
|
- name: Upload VSIX artifact
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
|
with:
|
|
name: openchamber-vscode-vsix
|
|
path: packages/vscode/*.vsix
|
|
|
|
- name: Attach VSIX to GitHub Release
|
|
if: startsWith(github.ref, 'refs/tags/')
|
|
uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2.6.2
|
|
with:
|
|
files: packages/vscode/*.vsix
|
|
generate_release_notes: false
|
|
env:
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|