feat(voice): match local and macOS voices to the language of the text

Text-to-speech picked one voice regardless of what language a reply was in. A dependency-free language detector (script, marker letters, function words) now decides the language of the whole message once; with the new "Match the voice to the language of the text" setting the local provider switches to a catalog model for that language (Kokoro zh/en and Piper models for 12 languages, downloaded on first use like the existing model) and macOS say switches to an installed voice whose locale matches. The local voice picker lists voices of every installed model, and the settings show which language models are on disk. The Ukrainian Piper medium build is a character-level model that sherpa-onnx turns into noise, so the espeak-based Lada build is used instead. Claude-Session: https://claude.ai/code/session_017TK5JAYDfT3Fotc23UEg98
2026-08-30 02:24:22 +03:00
parent 49f0a9e62f
commit 391f938334
29 changed files with 949 additions and 115 deletions
@@ -11,12 +11,25 @@ live transcript costs O(n^2) work for a result the final decode replaces. The
 composer shows no text while recording and inserts the full transcript on
 stop.

-Local TTS (Kokoro via sherpa-onnx OfflineTts) runs in the same worker process
-and is exposed as `POST /api/dictation/tts/speak` (JSON `{text, speakerId?,
-speed?, model?}` → WAV bytes; 503 with `reasonCode` while the model is
-downloading). TTS models live in the same catalog/downloader as STT models
-(`local/model-catalog.js` `LOCAL_TTS_MODEL_CATALOG`) and are managed by the
-same status/download/delete routes.
+Local TTS (Kokoro and Piper/VITS via sherpa-onnx OfflineTts) runs in the same
+worker process and is exposed as `POST /api/dictation/tts/speak` (JSON
+`{text, speakerId?, speed?, model?, language?, languageSample?}` → WAV bytes; 503 with
+`reasonCode` while the model is downloading). TTS models live in the same
+catalog/downloader as STT models (`local/model-catalog.js`
+`LOCAL_TTS_MODEL_CATALOG`) and are managed by the same status/download/delete
+routes.
+
+Each TTS catalog entry declares the `languages` it speaks. With
+`language: 'auto'` the service detects the language of `languageSample` — the
+whole message the chunk belongs to, sent by the client with every chunk — or
+of `text` when no sample is given
+(`../tts/language-detect.js`, script plus function-word scoring, no
+dependencies) and keeps the caller's model when it speaks that language;
+otherwise it switches to the catalog model for the language, downloading it on
+first use like any other model, and starts from that model's default speaker
+(`defaultSpeakerByLanguage`) instead of the caller's speaker id. A language no
+catalog model covers keeps the caller's model, so text is always spoken. The
+response carries `X-Speech-Model` and `X-Speech-Language`.

 ## Ownership

@@ -68,9 +68,19 @@ export const LOCAL_STT_MODEL_CATALOG = {
 * Local text-to-speech models (sherpa-onnx OfflineTts). Downloaded and
 * managed through the same pipeline as the STT models.
 */
+/**
+ * Local text-to-speech models (sherpa-onnx OfflineTts). Downloaded and
+ * managed through the same pipeline as the STT models.
+ *
+ * `languages` lists the languages a model speaks well; the speech service
+ * uses it to pick a model for the language a text is written in. Kokoro
+ * models carry speaker ids (`voices`); a Piper model is one voice for one
+ * language. `lexicon` entries are joined with commas for sherpa-onnx.
+ */
 export const LOCAL_TTS_MODEL_CATALOG = {
  'kokoro-en-v0_19': {
    type: 'kokoro',
+    languages: ['en'],
    archiveUrl:
      'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/kokoro-en-v0_19.tar.bz2',
    extractedDir: 'kokoro-en-v0_19',
@@ -82,6 +92,188 @@ export const LOCAL_TTS_MODEL_CATALOG = {
    },
    description: 'Kokoro TTS (English, natural voices)',
  },
+  'kokoro-multi-lang-v1_1': {
+    type: 'kokoro',
+    languages: ['zh', 'en'],
+    // sherpa-onnx wires this Kokoro build for Chinese and English only;
+    // speakers 0-2 are English, 3-102 Chinese.
+    defaultSpeakerByLanguage: { en: 0, zh: 3 },
+    archiveUrl:
+      'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/kokoro-multi-lang-v1_1.tar.bz2',
+    extractedDir: 'kokoro-multi-lang-v1_1',
+    files: {
+      model: 'model.onnx',
+      voices: 'voices.bin',
+      tokens: 'tokens.txt',
+      espeakData: 'espeak-ng-data',
+      lexiconEnglish: 'lexicon-us-en.txt',
+      lexiconChinese: 'lexicon-zh.txt',
+    },
+    lexicon: ['lexiconEnglish', 'lexiconChinese'],
+    description: 'Kokoro TTS (Chinese and English, 103 voices)',
+  },
+  // The larger `ukrainian_tts-medium` build is a character-level model
+  // (`phoneme_type: text`); sherpa-onnx phonemizes every Piper model through
+  // espeak-ng, which turns that one into noise. `vits-coqui-uk-mai` sounds
+  // better but reads Cyrillic only and drops every Latin word (file names,
+  // product names), which is unusable in a coding chat. Lada is an espeak
+  // model: small, but it reads mixed text.
+  'piper-uk_UA-lada-x_low': {
+    type: 'vits',
+    languages: ['uk'],
+    archiveUrl:
+      'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-uk_UA-lada-x_low.tar.bz2',
+    extractedDir: 'vits-piper-uk_UA-lada-x_low',
+    files: {
+      model: 'uk_UA-lada-x_low.onnx',
+      tokens: 'tokens.txt',
+      espeakData: 'espeak-ng-data',
+    },
+    description: 'Piper TTS (Ukrainian)',
+  },
+  'piper-de_DE-thorsten-medium': {
+    type: 'vits',
+    languages: ['de'],
+    archiveUrl:
+      'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-de_DE-thorsten-medium.tar.bz2',
+    extractedDir: 'vits-piper-de_DE-thorsten-medium',
+    files: {
+      model: 'de_DE-thorsten-medium.onnx',
+      tokens: 'tokens.txt',
+      espeakData: 'espeak-ng-data',
+    },
+    description: 'Piper TTS (German)',
+  },
+  'piper-fr_FR-siwis-medium': {
+    type: 'vits',
+    languages: ['fr'],
+    archiveUrl:
+      'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-fr_FR-siwis-medium.tar.bz2',
+    extractedDir: 'vits-piper-fr_FR-siwis-medium',
+    files: {
+      model: 'fr_FR-siwis-medium.onnx',
+      tokens: 'tokens.txt',
+      espeakData: 'espeak-ng-data',
+    },
+    description: 'Piper TTS (French)',
+  },
+  'piper-es_ES-davefx-medium': {
+    type: 'vits',
+    languages: ['es'],
+    archiveUrl:
+      'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-es_ES-davefx-medium.tar.bz2',
+    extractedDir: 'vits-piper-es_ES-davefx-medium',
+    files: {
+      model: 'es_ES-davefx-medium.onnx',
+      tokens: 'tokens.txt',
+      espeakData: 'espeak-ng-data',
+    },
+    description: 'Piper TTS (Spanish)',
+  },
+  'piper-it_IT-paola-medium': {
+    type: 'vits',
+    languages: ['it'],
+    archiveUrl:
+      'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-it_IT-paola-medium.tar.bz2',
+    extractedDir: 'vits-piper-it_IT-paola-medium',
+    files: {
+      model: 'it_IT-paola-medium.onnx',
+      tokens: 'tokens.txt',
+      espeakData: 'espeak-ng-data',
+    },
+    description: 'Piper TTS (Italian)',
+  },
+  'piper-pt_BR-faber-medium': {
+    type: 'vits',
+    languages: ['pt'],
+    archiveUrl:
+      'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-pt_BR-faber-medium.tar.bz2',
+    extractedDir: 'vits-piper-pt_BR-faber-medium',
+    files: {
+      model: 'pt_BR-faber-medium.onnx',
+      tokens: 'tokens.txt',
+      espeakData: 'espeak-ng-data',
+    },
+    description: 'Piper TTS (Portuguese (Brazil))',
+  },
+  'piper-pl_PL-gosia-medium': {
+    type: 'vits',
+    languages: ['pl'],
+    archiveUrl:
+      'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-pl_PL-gosia-medium.tar.bz2',
+    extractedDir: 'vits-piper-pl_PL-gosia-medium',
+    files: {
+      model: 'pl_PL-gosia-medium.onnx',
+      tokens: 'tokens.txt',
+      espeakData: 'espeak-ng-data',
+    },
+    description: 'Piper TTS (Polish)',
+  },
+  'piper-ru_RU-irina-medium': {
+    type: 'vits',
+    languages: ['ru'],
+    archiveUrl:
+      'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-ru_RU-irina-medium.tar.bz2',
+    extractedDir: 'vits-piper-ru_RU-irina-medium',
+    files: {
+      model: 'ru_RU-irina-medium.onnx',
+      tokens: 'tokens.txt',
+      espeakData: 'espeak-ng-data',
+    },
+    description: 'Piper TTS (Russian)',
+  },
+  'piper-nl_NL-pim-medium': {
+    type: 'vits',
+    languages: ['nl'],
+    archiveUrl:
+      'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-nl_NL-pim-medium.tar.bz2',
+    extractedDir: 'vits-piper-nl_NL-pim-medium',
+    files: {
+      model: 'nl_NL-pim-medium.onnx',
+      tokens: 'tokens.txt',
+      espeakData: 'espeak-ng-data',
+    },
+    description: 'Piper TTS (Dutch)',
+  },
+  'piper-cs_CZ-jirka-medium': {
+    type: 'vits',
+    languages: ['cs'],
+    archiveUrl:
+      'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-cs_CZ-jirka-medium.tar.bz2',
+    extractedDir: 'vits-piper-cs_CZ-jirka-medium',
+    files: {
+      model: 'cs_CZ-jirka-medium.onnx',
+      tokens: 'tokens.txt',
+      espeakData: 'espeak-ng-data',
+    },
+    description: 'Piper TTS (Czech)',
+  },
+  'piper-tr_TR-dfki-medium': {
+    type: 'vits',
+    languages: ['tr'],
+    archiveUrl:
+      'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-tr_TR-dfki-medium.tar.bz2',
+    extractedDir: 'vits-piper-tr_TR-dfki-medium',
+    files: {
+      model: 'tr_TR-dfki-medium.onnx',
+      tokens: 'tokens.txt',
+      espeakData: 'espeak-ng-data',
+    },
+    description: 'Piper TTS (Turkish)',
+  },
+  'piper-sv_SE-nst-medium': {
+    type: 'vits',
+    languages: ['sv'],
+    archiveUrl:
+      'https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/vits-piper-sv_SE-nst-medium.tar.bz2',
+    extractedDir: 'vits-piper-sv_SE-nst-medium',
+    files: {
+      model: 'sv_SE-nst-medium.onnx',
+      tokens: 'tokens.txt',
+      espeakData: 'espeak-ng-data',
+    },
+    description: 'Piper TTS (Swedish)',
+  },
 };

 export const DEFAULT_LOCAL_STT_MODEL = 'parakeet-tdt-0.6b-v2-int8';
@@ -131,6 +323,34 @@ export function getLocalSttModelSpec(modelId) {
  };
 }

+/**
+ * The local TTS model to use for a language, preferring the model the user
+ * selected when it speaks that language. Returns null when no catalog model
+ * covers the language, in which case callers keep the selected model.
+ * @param {string} language BCP-47 primary subtag (`uk`, `zh`...)
+ * @param {string} [preferredModelId]
+ * @returns {string | null}
+ */
+export function resolveLocalTtsModelForLanguage(language, preferredModelId) {
+  const speaks = (modelId) => LOCAL_TTS_MODEL_CATALOG[modelId]?.languages?.includes(language) === true;
+  if (preferredModelId && speaks(preferredModelId)) return preferredModelId;
+  const candidate = LOCAL_TTS_MODEL_IDS.find(speaks);
+  return candidate ?? null;
+}
+
+/**
+ * The speaker id a model should use for a language when the caller's
+ * speaker was chosen for another language. `undefined` keeps the caller's
+ * speaker.
+ * @param {string} modelId
+ * @param {string} language
+ * @returns {number | undefined}
+ */
+export function getLocalTtsDefaultSpeaker(modelId, language) {
+  const speaker = LOCAL_TTS_MODEL_CATALOG[modelId]?.defaultSpeakerByLanguage?.[language];
+  return Number.isInteger(speaker) ? speaker : undefined;
+}
+
 /**
 * @param {string} modelsDir
 * @param {string} modelId
@@ -0,0 +1,42 @@
+import { describe, expect, it } from 'vitest';
+import {
+  DEFAULT_LOCAL_TTS_MODEL,
+  LOCAL_TTS_MODEL_CATALOG,
+  getLocalSttModelSpec,
+  getLocalTtsDefaultSpeaker,
+  resolveLocalTtsModelForLanguage,
+} from './model-catalog.js';
+
+describe('local TTS catalog', () => {
+  it('keeps the selected model when it speaks the language', () => {
+    expect(resolveLocalTtsModelForLanguage('en', DEFAULT_LOCAL_TTS_MODEL)).toBe(DEFAULT_LOCAL_TTS_MODEL);
+    expect(resolveLocalTtsModelForLanguage('zh', 'kokoro-multi-lang-v1_1')).toBe('kokoro-multi-lang-v1_1');
+  });
+
+  it('picks a catalog model for a language the selected model lacks', () => {
+    expect(resolveLocalTtsModelForLanguage('uk', DEFAULT_LOCAL_TTS_MODEL)).toBe('piper-uk_UA-lada-x_low');
+    expect(resolveLocalTtsModelForLanguage('zh', DEFAULT_LOCAL_TTS_MODEL)).toBe('kokoro-multi-lang-v1_1');
+  });
+
+  it('returns null for a language no model covers', () => {
+    expect(resolveLocalTtsModelForLanguage('xx', DEFAULT_LOCAL_TTS_MODEL)).toBeNull();
+  });
+
+  it('gives Chinese a Chinese speaker on the multi-language Kokoro', () => {
+    expect(getLocalTtsDefaultSpeaker('kokoro-multi-lang-v1_1', 'zh')).toBe(3);
+    expect(getLocalTtsDefaultSpeaker('kokoro-multi-lang-v1_1', 'en')).toBe(0);
+    expect(getLocalTtsDefaultSpeaker('piper-uk_UA-lada-x_low', 'uk')).toBeUndefined();
+  });
+
+  it('every TTS entry declares its languages and installable files', () => {
+    for (const [id, spec] of Object.entries(LOCAL_TTS_MODEL_CATALOG)) {
+      expect(spec.languages.length, id).toBeGreaterThan(0);
+      expect(spec.archiveUrl, id).toMatch(/^https:\/\/github\.com\/k2-fsa\/sherpa-onnx\/releases\/download\/tts-models\//);
+      const resolved = getLocalSttModelSpec(id);
+      expect(resolved.requiredFiles, id).toContain(spec.files.model);
+      for (const key of spec.lexicon ?? []) {
+        expect(spec.files[key], `${id} lexicon ${key}`).toBeTruthy();
+      }
+    }
+  });
+});
@@ -1,5 +1,5 @@
 /**
- * Sherpa-onnx offline TTS (Kokoro). Runs inside the dictation worker process
+ * Sherpa-onnx offline TTS (Kokoro and Piper/VITS). Runs inside the dictation worker process
 * only — never load the native addon in the main server process.
 */

@@ -23,20 +23,49 @@ function float32ToPcm16le(samples) {
  return Buffer.from(out.buffer, out.byteOffset, out.byteLength);
 }

+/**
+ * sherpa-onnx model config for one catalog entry. Kokoro carries a voices
+ * bank (speaker ids) and optional lexicons; a Piper/VITS model is a single
+ * voice with espeak-ng phonemization.
+ * @param {{ modelDir: string, type?: string, files: Record<string, string>, lexicon?: string[] }} config
+ */
+function buildModelConfig(config) {
+  const file = (key, label) => {
+    const filePath = path.join(config.modelDir, config.files[key]);
+    assertFileExists(filePath, label);
+    return filePath;
+  };
+  const modelPath = file('model', 'TTS model');
+  const tokensPath = file('tokens', 'TTS tokens');
+
+  if (config.type === 'vits') {
+    // Piper models phonemize through espeak-ng (`espeakData`); character
+    // models (Coqui) read the text directly and carry no espeak data.
+    const dataDir = config.files.espeakData ? file('espeakData', 'TTS espeak-ng dataDir') : '';
+    return { vits: { model: modelPath, tokens: tokensPath, ...(dataDir ? { dataDir } : {}), lengthScale: 1.0 } };
+  }
+
+  const dataDir = file('espeakData', 'TTS espeak-ng dataDir');
+  const voicesPath = file('voices', 'TTS voices');
+  const lexicon = (config.lexicon ?? []).map((key) => file(key, 'TTS lexicon')).join(',');
+  return {
+    kokoro: {
+      model: modelPath,
+      voices: voicesPath,
+      tokens: tokensPath,
+      dataDir,
+      lengthScale: 1.0,
+      ...(lexicon ? { lexicon } : {}),
+    },
+  };
+}
+
 export class SherpaTtsEngine {
  /**
-   * @param {{ modelDir: string, files: { model: string, voices: string, tokens: string, espeakData: string }, numThreads?: number }} config
+   * @param {{ modelDir: string, type?: string, files: Record<string, string>, lexicon?: string[], numThreads?: number }} config
   */
  constructor(config) {
-    const modelPath = path.join(config.modelDir, config.files.model);
-    const voicesPath = path.join(config.modelDir, config.files.voices);
-    const tokensPath = path.join(config.modelDir, config.files.tokens);
-    const dataDir = path.join(config.modelDir, config.files.espeakData);
-
-    assertFileExists(modelPath, 'TTS model');
-    assertFileExists(voicesPath, 'TTS voices');
-    assertFileExists(tokensPath, 'TTS tokens');
-    assertFileExists(dataDir, 'TTS espeak-ng dataDir');
+    const model = buildModelConfig(config);

    const sherpa = loadSherpaOnnxNode();
    if (typeof sherpa.OfflineTts !== 'function') {
@@ -44,15 +73,7 @@ export class SherpaTtsEngine {
    }

    this.tts = new sherpa.OfflineTts({
-      model: {
-        kokoro: {
-          model: modelPath,
-          voices: voicesPath,
-          tokens: tokensPath,
-          dataDir,
-          lengthScale: 1.0,
-        },
-      },
+      model,
      numThreads: config.numThreads ?? 2,
      provider: 'cpu',
      maxNumSentences: 1,
@@ -102,7 +102,9 @@ function getTtsEngine(modelsDir, modelId) {
  const spec = getLocalSttModelSpec(modelId);
  const created = new SherpaTtsEngine({
    modelDir: getLocalSttModelDir(modelsDir, modelId),
+    type: spec.type,
    files: spec.files,
+    lexicon: spec.lexicon,
    numThreads: 2,
  });
  ttsEngines.set(key, created);
@@ -63,6 +63,8 @@ export function createDictationRuntime({
        model: typeof req.body?.model === 'string' ? req.body.model : undefined,
        speakerId: Number.isInteger(req.body?.speakerId) ? req.body.speakerId : undefined,
        speed: typeof req.body?.speed === 'number' ? req.body.speed : undefined,
+        language: req.body?.language === 'auto' ? 'auto' : undefined,
+        languageSample: typeof req.body?.languageSample === 'string' ? req.body.languageSample.slice(0, 4000) : undefined,
      });
      if (result.error) {
        res.status(503).json({
@@ -73,6 +75,8 @@ export function createDictationRuntime({
        return;
      }
      res.setHeader('Content-Type', result.format || 'audio/wav');
+      res.setHeader('X-Speech-Model', result.modelId);
+      if (result.language) res.setHeader('X-Speech-Language', result.language);
      res.send(result.audio);
    } catch (error) {
      res.status(500).json({ error: error?.message || 'Failed to synthesize speech' });
@@ -1,3 +1,4 @@
+import { detectTextLanguage } from '../tts/language-detect.js';
 /**
 * Dictation service: resolves STT providers, tracks local model download
 * state, and exposes a readiness snapshot for the status route.
@@ -16,6 +17,8 @@ import { OpenAICompatibleTranscriptionSession } from './openai-compatible-sessio
 import {
  DEFAULT_LOCAL_STT_MODEL,
  DEFAULT_LOCAL_TTS_MODEL,
+  getLocalTtsDefaultSpeaker,
+  resolveLocalTtsModelForLanguage,
  LOCAL_STT_MODEL_CATALOG,
  LOCAL_STT_MODEL_IDS,
  LOCAL_TTS_MODEL_CATALOG,
@@ -220,10 +223,30 @@ export function createDictationService({ modelsDir }) {
  /**
   * Synthesize speech with the local TTS model. Returns WAV bytes, or a
   * readiness error while the model is missing/downloading.
-   * @param {{ text: string, model?: string, speakerId?: number, speed?: number }} options
+   *
+   * With `language: 'auto'` the text's language decides the model: the
+   * caller's model when it speaks that language, otherwise the catalog
+   * model for it (downloaded on first use, reported as in-progress until it
+   * lands). The caller's speaker id is kept only on the caller's model; a
+   * substitute model starts from its own default speaker for the language.
+   * A language no catalog model covers keeps the caller's model, so text is
+   * never silently dropped.
+   * `languageSample` is the whole message the chunk belongs to (or a prefix
+   * of it): the language is judged on that, never on a short chunk alone.
+   * @param {{ text: string, model?: string, speakerId?: number, speed?: number, language?: string, languageSample?: string }} options
   */
-  const synthesizeSpeech = async ({ text, model, speakerId, speed }) => {
-    const modelId = isLocalTtsModelId(model) ? model : DEFAULT_LOCAL_TTS_MODEL;
+  const synthesizeSpeech = async ({ text, model, speakerId, speed, language, languageSample }) => {
+    const requestedModelId = isLocalTtsModelId(model) ? model : DEFAULT_LOCAL_TTS_MODEL;
+    let modelId = requestedModelId;
+    let resolvedLanguage = null;
+    if (language === 'auto') {
+      resolvedLanguage = detectTextLanguage(languageSample || text).language;
+      const forLanguage = resolveLocalTtsModelForLanguage(resolvedLanguage, requestedModelId);
+      if (forLanguage && forLanguage !== requestedModelId) {
+        modelId = forLanguage;
+        speakerId = getLocalTtsDefaultSpeaker(modelId, resolvedLanguage);
+      }
+    }
    const installed = await isLocalSttModelInstalled(modelsDir, modelId);
    if (!installed) {
      const state = downloadStates.get(modelId);
@@ -251,7 +274,7 @@ export function createDictationService({ modelsDir }) {
      speakerId,
      speed,
    });
-    return { audio: result.audio, format: result.format };
+    return { audio: result.audio, format: result.format, modelId, language: resolvedLanguage };
  };

  /**
@@ -11,6 +11,7 @@ This module provides server-side Text-to-Speech services using OpenAI's TTS API.
 - `packages/web/server/lib/text/summarization.js`: Shared text summarization stub and sanitization utilities. It performs no external Zen calls.
 - `packages/web/server/lib/tts/stt.js`: STT proxy for OpenAI-compatible transcription endpoints.
 - `packages/web/server/lib/tts/base-url.js`: shared base URL validation and normalization for custom OpenAI-compatible endpoints.
+- `packages/web/server/lib/tts/language-detect.js`: dependency-free language detection for voice selection (`detectTextLanguage`, `pickVoiceForLanguage`, `languageOfLocale`). Used by the macOS `say` route (`language: 'auto'` switches to an installed voice whose locale matches the text; the response carries `X-Speech-Voice` and `X-Speech-Language`) and by the dictation module's local TTS model choice.

 ## Public exports

@@ -0,0 +1,210 @@
+/**
+ * Language detection for text-to-speech voice selection.
+ *
+ * Picks the language a piece of chat text is written in so a TTS provider
+ * can choose a matching voice or model. Deliberately small and dependency
+ * free: the writing system decides most cases outright, and Latin-script
+ * languages are told apart by function words and characteristic letters.
+ * The answer is a best effort for voice selection, not a linguistic claim —
+ * an unknown language falls back to English rather than failing.
+ */
+
+const SCRIPT_RANGES = [
+  ['hangul', /[가-힯ᄀ-ᇿ㄰-㆏]/g],
+  ['kana', /[぀-ヿ]/g],
+  ['han', /[一-鿿㐀-䶿]/g],
+  ['cyrillic', /[Ѐ-ӿ]/g],
+  ['greek', /[Ͱ-Ͽ]/g],
+  ['arabic', /[؀-ۿ]/g],
+  ['hebrew', /[֐-׿]/g],
+  ['thai', /[฀-๿]/g],
+  ['devanagari', /[ऀ-ॿ]/g],
+  ['latin', /[A-Za-zÀ-ɏ]/g],
+];
+
+const SCRIPT_LANGUAGE = {
+  hangul: 'ko',
+  greek: 'el',
+  arabic: 'ar',
+  hebrew: 'he',
+  thai: 'th',
+  devanagari: 'hi',
+};
+
+// Letters that only (or overwhelmingly) occur in one language of a script.
+const LATIN_MARKERS = {
+  pl: /[łęąńśźż]/i,
+  cs: /[řěůťďň]/i,
+  tr: /[ğışİ]/,
+  pt: /[ãõ]/i,
+  es: /[ñ¿¡]/,
+  de: /[ß]/,
+  fr: /[œ]/i,
+  sv: /[å]/i,
+};
+
+// Frequent function words per language. Scored by whole-word hits; every
+// list has the same length so scores stay comparable.
+const STOPWORDS = {
+  en: ['the', 'and', 'is', 'to', 'of', 'that', 'you', 'with', 'for', 'this', 'are', 'it', 'not', 'have', 'can', 'will', 'your', 'from', 'which', 'when'],
+  de: ['und', 'der', 'die', 'das', 'ist', 'nicht', 'mit', 'ein', 'eine', 'auch', 'sich', 'auf', 'für', 'wird', 'werden', 'oder', 'aber', 'wenn', 'sind', 'kann'],
+  fr: ['le', 'la', 'les', 'et', 'est', 'une', 'des', 'pour', 'que', 'qui', 'dans', 'pas', 'vous', 'sur', 'avec', 'sont', 'nous', 'cette', 'mais', 'plus'],
+  es: ['el', 'la', 'los', 'las', 'que', 'es', 'una', 'por', 'para', 'con', 'del', 'como', 'pero', 'más', 'este', 'esta', 'son', 'tiene', 'puede', 'también'],
+  it: ['il', 'la', 'che', 'di', 'è', 'una', 'per', 'non', 'con', 'del', 'della', 'come', 'sono', 'anche', 'questo', 'questa', 'gli', 'nel', 'più', 'essere'],
+  pt: ['o', 'a', 'os', 'as', 'que', 'é', 'uma', 'para', 'com', 'não', 'do', 'da', 'como', 'mas', 'também', 'este', 'esta', 'são', 'você', 'pode'],
+  pl: ['i', 'nie', 'jest', 'się', 'na', 'to', 'że', 'jak', 'ale', 'dla', 'oraz', 'przez', 'czy', 'tym', 'jego', 'można', 'jeśli', 'tego', 'które', 'także'],
+  nl: ['de', 'het', 'een', 'en', 'van', 'is', 'niet', 'dat', 'met', 'voor', 'ook', 'zijn', 'maar', 'als', 'wordt', 'deze', 'kan', 'naar', 'bij', 'dan'],
+  cs: ['a', 'je', 'se', 'na', 'to', 'že', 'jak', 'ale', 'pro', 'nebo', 'jsou', 'může', 'také', 'tento', 'když', 'jeho', 'které', 'být', 'aby', 'ještě'],
+  tr: ['ve', 'bir', 'bu', 'için', 'ile', 'de', 'da', 'ama', 'gibi', 'daha', 'var', 'olarak', 'çok', 'ne', 'her', 'kadar', 'sonra', 'değil', 'olan', 'ise'],
+  sv: ['och', 'att', 'det', 'är', 'en', 'som', 'för', 'inte', 'med', 'till', 'den', 'kan', 'har', 'ett', 'men', 'också', 'eller', 'från', 'när', 'vara'],
+  uk: ['і', 'та', 'що', 'це', 'не', 'як', 'для', 'він', 'вона', 'але', 'або', 'також', 'тільки', 'вже', 'якщо', 'його', 'цей', 'ця', 'бути', 'коли'],
+  ru: ['и', 'что', 'это', 'не', 'как', 'для', 'он', 'она', 'но', 'или', 'также', 'только', 'уже', 'если', 'его', 'этот', 'эта', 'быть', 'когда', 'чтобы'],
+};
+
+const LATIN_LANGUAGES = ['en', 'de', 'fr', 'es', 'it', 'pt', 'pl', 'nl', 'cs', 'tr', 'sv'];
+const CYRILLIC_LANGUAGES = ['uk', 'ru'];
+
+const countMatches = (text, pattern) => {
+  const matches = text.match(pattern);
+  return matches ? matches.length : 0;
+};
+
+const scoreStopwords = (words, languages) => {
+  const scores = {};
+  for (const language of languages) {
+    const list = new Set(STOPWORDS[language]);
+    let hits = 0;
+    for (const word of words) {
+      if (list.has(word)) hits += 1;
+    }
+    scores[language] = hits;
+  }
+  return scores;
+};
+
+const bestOf = (scores, fallback) => {
+  let best = fallback;
+  let bestScore = 0;
+  for (const [language, score] of Object.entries(scores)) {
+    if (score > bestScore) {
+      best = language;
+      bestScore = score;
+    }
+  }
+  return best;
+};
+
+const pickByMarkers = (text, markers) => {
+  for (const [language, pattern] of Object.entries(markers)) {
+    if (pattern.test(text)) return language;
+  }
+  return null;
+};
+
+/**
+ * @param {string} text
+ * @returns {{ language: string, script: string }} BCP-47 primary language subtag and the dominant script.
+ */
+export function detectTextLanguage(text) {
+  const source = typeof text === 'string' ? text : '';
+  const counts = SCRIPT_RANGES.map(([script, pattern]) => [script, countMatches(source, pattern)]);
+  const letters = counts.reduce((sum, [, count]) => sum + count, 0);
+  if (letters === 0) return { language: 'en', script: 'latin' };
+
+  // Kana settles Japanese even when Han dominates the character count.
+  const kana = counts.find(([script]) => script === 'kana')?.[1] ?? 0;
+  const han = counts.find(([script]) => script === 'han')?.[1] ?? 0;
+  if (kana > 0 && kana + han >= letters * 0.3) return { language: 'ja', script: 'kana' };
+  if (han > 0 && han >= letters * 0.3) return { language: 'zh', script: 'han' };
+
+  const [script] = counts.reduce((best, entry) => (entry[1] > best[1] ? entry : best));
+
+  if (script in SCRIPT_LANGUAGE) return { language: SCRIPT_LANGUAGE[script], script };
+
+  const words = source.toLowerCase().split(/[^\p{L}\p{M}']+/u).filter(Boolean);
+
+  if (script === 'cyrillic') {
+    const scores = scoreStopwords(words, CYRILLIC_LANGUAGES);
+    const ukMarkers = countMatches(source, /[іїєґ]/gi);
+    const ruMarkers = countMatches(source, /[ыэъё]/gi);
+    // Letters decide: the two alphabets differ in letters that occur in
+    // nearly every sentence. Function words only settle a text that shows
+    // neither set, and a text with no Russian-only letters is far more
+    // likely Ukrainian than the reverse, so that tie goes to Ukrainian.
+    if (ukMarkers !== ruMarkers) return { language: ukMarkers > ruMarkers ? 'uk' : 'ru', script };
+    if (scores.uk !== scores.ru) return { language: scores.uk > scores.ru ? 'uk' : 'ru', script };
+    return { language: ruMarkers > 0 ? 'ru' : 'uk', script };
+  }
+
+  const scores = scoreStopwords(words, LATIN_LANGUAGES);
+  const marked = pickByMarkers(source, LATIN_MARKERS);
+  // A characteristic letter outranks stopword counts unless another language
+  // clearly dominates the function words (a German text quoting "façade").
+  if (marked && scores[marked] * 2 >= scores[bestOf(scores, marked)]) {
+    return { language: marked, script };
+  }
+  return { language: bestOf(scores, 'en'), script };
+}
+
+/**
+ * Map a detected language onto the locales a voice list uses (`uk_UA`,
+ * `en_US`...). Returns the preferred locale prefixes in order.
+ * @param {string} language
+ * @returns {string[]}
+ */
+function localePrefixesForLanguage(language) {
+  const table = {
+    en: ['en_US', 'en_GB', 'en'],
+    uk: ['uk_UA', 'uk'],
+    ru: ['ru_RU', 'ru'],
+    de: ['de_DE', 'de'],
+    fr: ['fr_FR', 'fr_CA', 'fr'],
+    es: ['es_ES', 'es_MX', 'es'],
+    it: ['it_IT', 'it'],
+    pt: ['pt_BR', 'pt_PT', 'pt'],
+    pl: ['pl_PL', 'pl'],
+    nl: ['nl_NL', 'nl_BE', 'nl'],
+    cs: ['cs_CZ', 'cs'],
+    tr: ['tr_TR', 'tr'],
+    sv: ['sv_SE', 'sv'],
+    zh: ['zh_CN', 'zh_TW', 'zh_HK', 'zh'],
+    ja: ['ja_JP', 'ja'],
+    ko: ['ko_KR', 'ko'],
+    el: ['el_GR', 'el'],
+    ar: ['ar_001', 'ar_SA', 'ar'],
+    he: ['he_IL', 'he'],
+    th: ['th_TH', 'th'],
+    hi: ['hi_IN', 'hi'],
+  };
+  return table[language] ?? [language];
+}
+
+/**
+ * Choose a voice for a language from a `say`-style voice list.
+ * Prefers an enhanced/premium variant of a matching voice, then any voice of
+ * the exact locale, then any voice of the language. Returns null when the
+ * list has no voice for that language.
+ * @param {string} language
+ * @param {ReadonlyArray<{ name: string, locale: string }>} voices
+ * @returns {string | null}
+ */
+export function pickVoiceForLanguage(language, voices) {
+  const prefixes = localePrefixesForLanguage(language);
+  for (const prefix of prefixes) {
+    const matching = voices.filter((voice) => voice.locale === prefix || voice.locale.startsWith(`${prefix}_`) || (prefix === language && voice.locale.startsWith(`${language}_`)));
+    if (matching.length === 0) continue;
+    const enhanced = matching.find((voice) => /\((Enhanced|Premium)\)/i.test(voice.name));
+    return (enhanced ?? matching[0]).name;
+  }
+  return null;
+}
+
+/**
+ * Language of a voice, from its locale (`uk_UA` → `uk`).
+ * @param {string | null | undefined} locale
+ * @returns {string | null}
+ */
+export function languageOfLocale(locale) {
+  if (typeof locale !== 'string' || !locale) return null;
+  return locale.split(/[_-]/)[0].toLowerCase();
+}
@@ -0,0 +1,76 @@
+import { describe, expect, it } from 'vitest';
+import { detectTextLanguage, languageOfLocale, pickVoiceForLanguage } from './language-detect.js';
+
+describe('detectTextLanguage', () => {
+  it.each([
+    ['en', 'The build is green and the tests pass, so you can merge this now.'],
+    ['uk', 'Привіт! Це тестове повідомлення, і воно написане українською мовою.'],
+    ['ru', 'Привет! Это тестовое сообщение, и оно написано на русском языке.'],
+    ['de', 'Die Änderung ist fertig und die Tests laufen ohne Fehler durch.'],
+    ['fr', 'La modification est prête et les tests passent sans erreur.'],
+    ['es', 'El cambio está listo y las pruebas pasan sin errores.'],
+    ['it', 'La modifica è pronta e i test passano senza errori.'],
+    ['pt', 'A alteração está pronta e os testes passam sem erros, você pode continuar.'],
+    ['pl', 'Zmiana jest gotowa i testy przechodzą bez błędów.'],
+    ['nl', 'De wijziging is klaar en de tests slagen zonder fouten.'],
+    ['cs', 'Změna je hotová a testy procházejí bez chyb.'],
+    ['tr', 'Değişiklik hazır ve testler hatasız geçiyor.'],
+    ['sv', 'Ändringen är klar och testerna går igenom utan fel.'],
+    ['zh', '修改已经完成，所有测试都通过了。'],
+    ['ja', '変更が完了し、すべてのテストに合格しました。'],
+    ['ko', '변경이 완료되었고 모든 테스트를 통과했습니다.'],
+  ])('detects %s', (language, text) => {
+    expect(detectTextLanguage(text).language).toBe(language);
+  });
+
+  it.each([
+    ['uk', 'Готово. Запушено.'],
+    ['uk', 'Все ок'],
+    ['uk', 'Добре, давай так зробимо'],
+    ['ru', 'Хорошо, давай так и сделаем'],
+    ['ru', 'Готово, всё запушено.'],
+  ])('tells short %s phrases apart by letters', (language, text) => {
+    expect(detectTextLanguage(text).language).toBe(language);
+  });
+
+  it('falls back to English for text without letters', () => {
+    expect(detectTextLanguage('1234 ... !!!').language).toBe('en');
+    expect(detectTextLanguage('').language).toBe('en');
+  });
+
+  it('does not let a single quoted foreign word flip an English paragraph', () => {
+    const text = 'The façade of the building is the part that you see from the street, and it is not the same as the interior.';
+    expect(detectTextLanguage(text).language).toBe('en');
+  });
+});
+
+describe('pickVoiceForLanguage', () => {
+  const voices = [
+    { name: 'Samantha', locale: 'en_US' },
+    { name: 'Daniel', locale: 'en_GB' },
+    { name: 'Lesya', locale: 'uk_UA' },
+    { name: 'Lesya (Enhanced)', locale: 'uk_UA' },
+    { name: 'Milena', locale: 'ru_RU' },
+    { name: 'Anna', locale: 'de_DE' },
+  ];
+
+  it('prefers the enhanced variant of a matching voice', () => {
+    expect(pickVoiceForLanguage('uk', voices)).toBe('Lesya (Enhanced)');
+  });
+
+  it('prefers the primary locale of a language', () => {
+    expect(pickVoiceForLanguage('en', voices)).toBe('Samantha');
+  });
+
+  it('returns null when no voice speaks the language', () => {
+    expect(pickVoiceForLanguage('ja', voices)).toBeNull();
+  });
+});
+
+describe('languageOfLocale', () => {
+  it('reads the language subtag', () => {
+    expect(languageOfLocale('uk_UA')).toBe('uk');
+    expect(languageOfLocale('en-GB')).toBe('en');
+    expect(languageOfLocale(null)).toBeNull();
+  });
+});
@@ -2,6 +2,8 @@ import express from 'express';
 import { normalizeCustomOpenAIBaseURL } from './base-url.js';
 import { summarizeText, sanitizeForTTS, sanitizeForNote } from '../text/summarization.js';

+import { detectTextLanguage, languageOfLocale, pickVoiceForLanguage } from './language-detect.js';
+
 export function registerTtsRoutes(app, { sayTTSCapability }) {
  let ttsModulePromise = null;
  const getTtsModule = async () => {
@@ -154,7 +156,8 @@ export function registerTtsRoutes(app, { sayTTSCapability }) {
  // macOS 'say' command TTS speak endpoint
  app.post('/api/tts/say/speak', async (req, res) => {
    try {
-      const { text, voice = 'Samantha', rate = 200 } = req.body || {};
+      const { text, rate = 200, language, languageSample } = req.body || {};
+      let voice = typeof req.body?.voice === 'string' && req.body.voice.trim() ? req.body.voice.trim() : 'Samantha';
      
      if (!text || typeof text !== 'string' || !text.trim()) {
        return res.status(400).json({ error: 'Text is required' });
@@ -164,6 +167,23 @@ export function registerTtsRoutes(app, { sayTTSCapability }) {
      if (process.platform !== 'darwin') {
        return res.status(503).json({ error: 'macOS say command not available on this platform' });
      }
+
+      // `language: 'auto'`: keep the chosen voice while it speaks the text's
+      // language, otherwise switch to an installed voice that does. A
+      // language with no installed voice keeps the chosen voice — say still
+      // reads the text, just with an accent — rather than failing.
+      let resolvedLanguage = null;
+      if (language === 'auto') {
+        const capability = await sayTTSCapability;
+        const voices = Array.isArray(capability?.voices) ? capability.voices : [];
+        const sample = typeof languageSample === 'string' && languageSample.trim() ? languageSample.slice(0, 4000) : text;
+        resolvedLanguage = detectTextLanguage(sample).language;
+        const chosen = voices.find((entry) => entry.name === voice);
+        if (languageOfLocale(chosen?.locale) !== resolvedLanguage) {
+          const match = pickVoiceForLanguage(resolvedLanguage, voices);
+          if (match) voice = match;
+        }
+      }
      
      const { exec } = await import('child_process');
      const { promisify } = await import('util');
@@ -195,6 +215,8 @@ export function registerTtsRoutes(app, { sayTTSCapability }) {
      
      // Send audio response
      res.setHeader('Content-Type', 'audio/mp4');
+      res.setHeader('X-Speech-Voice', voice);
+      if (resolvedLanguage) res.setHeader('X-Speech-Language', resolvedLanguage);
      res.setHeader('Content-Length', audioBuffer.length);
      res.send(audioBuffer);
      
@@ -33,6 +33,32 @@ describe('tts routes', () => {
    });
  });

+  it('switches the say voice to the language of the text when asked to', async () => {
+    const capability = Promise.resolve({
+      available: true,
+      voices: [
+        { name: 'Samantha', locale: 'en_US' },
+        { name: 'Lesya', locale: 'uk_UA' },
+        { name: 'Lesya (Enhanced)', locale: 'uk_UA' },
+      ],
+    });
+    const app = createApp(capability);
+    const response = await request(app)
+      .post('/api/tts/say/speak')
+      .send({ text: 'Привіт! Це відповідь українською мовою, і вона досить довга.', voice: 'Samantha', language: 'auto' });
+
+    // On macOS the route synthesizes; elsewhere it refuses before running say.
+    // Either way the chosen voice must be the Ukrainian one when the platform
+    // allows the request to proceed.
+    if (process.platform === 'darwin') {
+      expect(response.status).toBe(200);
+      expect(response.headers['x-speech-voice']).toBe('Lesya (Enhanced)');
+      expect(response.headers['x-speech-language']).toBe('uk');
+    } else {
+      expect(response.status).toBe(503);
+    }
+  });
+
  it('returns local note fallback while model summarization is retired', async () => {
    const response = await request(createApp())
      .post('/api/text/summarize')