BuzzbeeSCD/openchamber
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
9a7d7a4379fd78c8005f5d975e57247c60599433
openchamber/packages/mobile
T
History
Bohdan Triapitsyn de1b85ac56 feat(voice): first-class voice input and local TTS across web, desktop, and mobile (#2018) 
Complete rebuild of voice input on a server-authoritative streaming
architecture, replacing the legacy Web Speech / whole-blob / WASM engines
and the dead voice-agent layer (~4k lines removed).

Speech-to-text (dictation):
- Client streams 16 kHz mono PCM16 chunks over /api/dictation/ws with
  seq/ack ordering; buffered audio is retained and replayed on reconnect
- Server transcribes and streams live partial transcripts back;
  segments auto-commit every ~15s with silence suppression and adaptive
  finalization timeouts
- Local provider (default, zero config): sherpa-onnx models in a forked
  worker process — auto-download with progress, staged extraction with
  verification, corrupt-model auto-recovery, idle shutdown after 5 min
- Model catalog with settings picker (accuracy/speed ratings, sizes,
  download/delete): Parakeet TDT v2 (English) and v3 (25 European
  languages, auto-detected), Whisper base and tiny (multilingual, light)
- OpenAI-compatible provider for any Whisper endpoint
- Composer overlay with live transcript, volume meter, timer, and
  cancel / insert / insert-and-send actions; failed transcriptions keep
  their audio for retry or accepting the partial text as-is
- Configurable keyboard shortcut (default mod+alt+v) toggles dictation;
  Enter confirms and Escape cancels while recording
- Overlay is pixel-aligned with the composer (measured footer height,
  matching paddings/typography/gaps) — no layout shift when toggling

Text-to-speech:
- Local Kokoro provider (English, 11 voices) synthesized in the same
  worker via /api/dictation/tts/speak, managed by the shared model
  pipeline; sentence-pipelined playback keeps time-to-first-audio at
  ~1 sentence regardless of message length, and stop cancels in-flight
  synthesis
- Sanitizer keeps inline-code content (strips backticks only), reads
  interword slashes aloud, and removes only absolute file paths

Settings:
- Voice page unified: a single read-aloud toggle owns all playback
  options (the confusing "Enable Voice Mode" is gone); a new "Enable
  voice input" toggle (default on, persisted to settings.json) hides
  the composer mic entirely when disabled

Mobile and transport:
- iOS/Android microphone permissions added (dictation was previously
  impossible on mobile)
- Fixed Android WebSocket upgrades: the Capacitor WebView origin
  (https://localhost) was missing from the packaged-client allowlist,
  403-ing every WS connection — root cause of the old mobile SSE lock,
  which is now removed for all transports

Security and conventions:
- All HTTP routes sit behind the global /api auth gate; the WS upgrade
  explicitly validates the UI session and origin, with oc_url_token
  narrowly allowlisted and covered by tests; the dictation socket mints
  a fresh URL token before connecting
- Routes register before the generic OpenCode proxy; the client goes
  through runtimeFetch/getRuntimeUrlResolver, and runtime switches
  reset the dictation socket
- VS Code deliberately reports dictation as unavailable (no server
  process in that runtime)

CI: workflow Node bumped 20 -> 22 to match the repo engines and fix
better-sqlite3 installs broken by node-gyp@latest on Node 20.

New dependency: sherpa-onnx-node (prebuilt N-API; macOS/Linux x64+arm64,
Windows x64 — Windows-on-ARM falls back to the OpenAI-compatible provider)
2026-07-04 02:48:07 +03:00
..
android
feat(voice): first-class voice input and local TTS across web, desktop, and mobile (#2018)
2026-07-04 02:48:07 +03:00
assets
feat: native iOS & Android mobile apps (Capacitor) (#1954)
2026-07-01 09:55:41 +03:00
ios
feat(voice): first-class voice input and local TTS across web, desktop, and mobile (#2018)
2026-07-04 02:48:07 +03:00
scripts
feat: native iOS & Android mobile apps (Capacitor) (#1954)
2026-07-01 09:55:41 +03:00
capacitor.config.ts
feat: native iOS & Android mobile apps (Capacitor) (#1954)
2026-07-01 09:55:41 +03:00
HANDOFF.md
feat: native iOS & Android mobile apps (Capacitor) (#1954)
2026-07-01 09:55:41 +03:00
package.json
feat: native iOS & Android mobile apps (Capacitor) (#1954)
2026-07-01 09:55:41 +03:00
README.md
feat: native iOS & Android mobile apps (Capacitor) (#1954)
2026-07-01 09:55:41 +03:00
tsconfig.json
feat: native iOS & Android mobile apps (Capacitor) (#1954)
2026-07-01 09:55:41 +03:00

README.md

OpenChamber Mobile

Capacitor shell for the dedicated OpenChamber mobile web surface.

The mobile package reuses the web build, then rewrites mobile.html to index.html in packages/mobile/dist so native iOS/Android always launch MobileApp instead of the hosted surface selector.

Runtime Model

  • The native app bundles the mobile UI only; it does not embed the OpenChamber web server or OpenCode server.
  • On first launch in Capacitor, the app shows a connection screen for an existing OpenChamber server.
  • Connections are saved locally in the app and can be managed from the mobile overflow menu under Instances.
  • The connection screen and Instances menu item are Capacitor-only. Hosted mobile.html in a normal browser keeps the regular web behavior.
  • Password-protected OpenChamber servers can be unlocked from the mobile app. The app stores the issued client token with the saved connection.

Commands

Run these from packages/mobile, or use the root mobile:* aliases.

  • bun run build: builds packages/web and prepares mobile web assets.
  • bun run sync: prepares assets and runs cap sync.
  • bun run add:ios: creates the native iOS project.
  • bun run add:android: creates the native Android project.
  • bun run build:android:debug: builds a debug Android APK without launching an emulator.
  • bun run build:ios:simulator: builds an iOS Simulator app without launching Xcode or Simulator.
  • bun run sim:run: boots a simulator if needed, installs the built iOS app, and launches it.
  • bun run sim:serve: starts serve-sim in detached JSON mode and prints the browser preview URL.
  • bun run sim:list: lists running serve-sim streams.
  • bun run sim:kill: stops running serve-sim streams.
  • bun run open:ios: opens the iOS project.
  • bun run open:android: opens the Android project.

Headless Quickstart

bun run build
bun run sync
bun run build:ios:simulator
bun run build:android:debug

These commands build and sync the native projects without launching Xcode, Android Studio, Simulator, or an emulator.

Local Tooling

The default scripts assume the local Homebrew/Xcode paths prepared for this workspace:

  • Xcode: /Applications/Xcode.app/Contents/Developer
  • JDK 21: /opt/homebrew/opt/openjdk@21
  • Android SDK: /opt/homebrew/share/android-commandlinetools

Override DEVELOPER_DIR, JAVA_HOME, ANDROID_HOME, or ANDROID_SDK_ROOT when using a different local setup.

Required local tools:

  • Xcode with iOS Simulator support.
  • CocoaPods for iOS dependency installation.
  • JDK 21 for Android Gradle builds.
  • Android SDK command-line tools with platform/build-tools 35.

Troubleshooting

  • If xcodebuild reports that the active developer directory is Command Line Tools, keep using the provided scripts or set DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer.
  • If Android builds fail with Unable to locate a Java Runtime or source release: 21, install/use JDK 21 and set JAVA_HOME accordingly.
  • If Android SDK packages are missing, install platform-tools, platforms;android-35, and build-tools;35.0.0, then accept SDK licenses.
  • If CocoaPods cannot find Capacitor pods after reinstalling dependencies, run bun install from the workspace root, then rerun bun run sync.
  • If connecting to a remote OpenChamber server fails from the app while /health works in curl, check that the server build includes the packaged-client CORS allowlist for capacitor://localhost and local dev origins.
  • If serve-sim preview says the stream is not producing frames, check the raw MJPEG stream before assuming the simulator stopped. In prior testing the raw stream worked while the browser preview UI stayed stale.

Generated Assets

The native projects currently use Capacitor-generated launcher and splash assets. Replace them before release branding work.

Reference in New Issue View Git Blame Copy Permalink