Switch the video guide narrator from espeak-ng to Piper (neural TTS)

espeak-ng's formant synthesis was reported as barely intelligible. Piper is a small
local neural TTS with the same offline, no-cloud-service properties but far more
natural speech; its default WAV output format already matched what the sync pipeline
assumed, so only the synthesis call itself needed to change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
larssand
2026-09-11 17:47:31 +02:00
parent 788319d098
commit a17054d8d0
5 changed files with 34 additions and 14 deletions

View File

@@ -8,6 +8,12 @@ The application version is maintained in `package.json` and displayed in the adm
No changes recorded yet.
## 1.18.2 - 2026-09-11
### Changed
- `scripts/record-video-guides.mjs` now narrates with Piper (a small local neural TTS) instead of espeak-ng - espeak-ng's formant synthesis was reported as barely intelligible ("robotröst från 80-talet"). Requires `pipx install piper-tts` plus a downloaded voice model; see the script's header comment.
## 1.18.1 - 2026-09-11
### Changed

View File

@@ -454,6 +454,15 @@ The user's original ask ("a video guide showing how to do the various things") m
Verified 2026-09-11: re-recorded all three clips and confirmed each now has both a video and an AAC audio stream (`ffprobe`). Extracted the volume envelope of the settings-tabs clip's audio and confirmed 5 distinct speech bursts separated by clean silence, matching its 5 narration lines. Cross-checked video frames against the audio timeline: at 8s into settings-tabs (mid second line, "Audio controls race sound announcements...") the Audio tab is the one on screen; at 20s into event-manager-tabs (the Schedule line) the Schedule tab/panel is on screen; at 12s into event-schedule (the "enter a start time" line) the form is visible filled in but not yet saved - narration and visuals line up correctly throughout, not just at the start.
### Second follow-up: espeak-ng's voice was reported barely intelligible
Reported 2026-09-11 by the user right after listening to the first narrated batch: "går ju knappt höra va den säger.. robot röst från 80 talet" (can barely make out what it's saying - a voice like a robot from the 80s). espeak-ng is formant synthesis (the same family as the original DECtalk/Speak & Spell era of TTS this compares it to) - accurate about what it is, not a bug, just the wrong tool for something meant to be listened to and understood.
- [x] Installed Piper (`pipx install piper-tts`, an isolated CLI install via pipx rather than a system-wide pip install) - a small local neural (VITS-based) TTS that produces natural-sounding speech offline, no cloud service involved. Downloaded the `en_US-lessac-medium` voice model from the `rhasspy/piper-voices` collection on Hugging Face to `~/.local/share/piper-voices/` (a one-time, machine-local setup step - the ~60MB model isn't part of the repo).
- [x] Swapped `synthesizeNarration` in `scripts/record-video-guides.mjs` from shelling out to `espeak-ng -w` to piping each line into `piper -m <model> -f <wav>` over stdin. Piper's default output format (22050Hz mono 16-bit PCM WAV) matches what the rest of the pipeline (the `adelay`/`amix` sync logic, `wavDurationMs`) already assumed, so no other part of the recorder needed to change.
Verified 2026-09-11: re-recorded all three clips with Piper and confirmed each still has a valid video+AAC-audio MKV (`ffprobe`). Re-checked the settings-tabs clip's volume envelope - still 5 clean speech bursts in the right places, now visibly fuller/louder (Piper's neural waveform vs. espeak-ng's thin formant one), confirming the sync logic itself didn't need to change, only the synthesis backend.
## Documentation maintenance
- [ ] Update the architecture's module/dependency inventory as modules change.

4
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{
"name": "rc-racecore",
"version": "1.18.1",
"version": "1.18.2",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "rc-racecore",
"version": "1.18.1",
"version": "1.18.2",
"hasInstallScript": true,
"dependencies": {
"better-sqlite3": "^12.11.1",

View File

@@ -1,6 +1,6 @@
{
"name": "rc-racecore",
"version": "1.18.1",
"version": "1.18.2",
"private": true,
"description": "RC RaceCore live timing and race control with AMMC ingest and SQLite persistence",
"engines": {

View File

@@ -4,23 +4,28 @@
// ever touches a real installation's data.
//
// Each clip is a list of steps ({ text, action }). For every step, the narration line's
// audio is synthesized up front (espeak-ng) so its real duration is known, the on-screen
// action runs, and the recording pauses for exactly that duration before moving to the
// next line - the actual wall-clock time each action took is recorded too, so the final
// narration track is built from real measured offsets (via ffmpeg's adelay/amix), not an
// idealized schedule that would drift out of sync with page-load/network latency.
// audio is synthesized up front (Piper, a small local neural TTS - far more intelligible
// than espeak-ng's formant synthesis, which is what this used before) so its real duration
// is known, the on-screen action runs, and the recording pauses for exactly that duration
// before moving to the next line - the actual wall-clock time each action took is recorded
// too, so the final narration track is built from real measured offsets (via ffmpeg's
// adelay/amix), not an idealized schedule that would drift out of sync with page-load/
// network latency.
//
// Usage: node scripts/record-video-guides.mjs [clip-name ...]
// With no arguments, records every clip below. Pass one or more names (e.g.
// "settings-tabs") to record only those. Output lands in video-guides/<name>.mkv
// (gitignored - these are sizeable binaries, not something to commit by default).
//
// Requires ffmpeg and espeak-ng on PATH.
// Requires ffmpeg and Piper (`pipx install piper-tts`) on PATH, plus a downloaded voice
// model - by default ~/.local/share/piper-voices/en_US-lessac-medium.onnx (get it from
// https://huggingface.co/rhasspy/piper-voices, the .onnx and its matching .onnx.json).
// Override the model with the PIPER_MODEL env var.
import { spawn, execFileSync } from "node:child_process";
import { existsSync, mkdtempSync, rmSync, mkdirSync, writeFileSync } from "node:fs";
import { createServer } from "node:net";
import { tmpdir } from "node:os";
import { homedir, tmpdir } from "node:os";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
@@ -28,10 +33,10 @@ const projectDir = dirname(dirname(fileURLToPath(import.meta.url)));
const outputDir = join(projectDir, "video-guides");
const chrome = [process.env.CHROME_PATH, "/usr/bin/google-chrome", "/usr/bin/chromium", "/usr/bin/chromium-browser"].find((p) => p && existsSync(p));
if (!chrome) throw new Error("Chrome/Chromium not found. Set CHROME_PATH to the browser executable.");
try { execFileSync("espeak-ng", ["--version"], { stdio: "ignore" }); } catch { throw new Error("espeak-ng not found on PATH (used for narration audio)."); }
try { execFileSync("piper", ["--help"], { stdio: "ignore" }); } catch { throw new Error("piper not found on PATH (used for narration audio). Install with: pipx install piper-tts"); }
const PIPER_MODEL = process.env.PIPER_MODEL || join(homedir(), ".local", "share", "piper-voices", "en_US-lessac-medium.onnx");
if (!existsSync(PIPER_MODEL)) throw new Error(`Piper voice model not found at ${PIPER_MODEL}. Download it from https://huggingface.co/rhasspy/piper-voices (both the .onnx and .onnx.json files), or set PIPER_MODEL.`);
const NARRATION_VOICE = "en-us";
const NARRATION_RATE = 160;
const LEAD_MS = 600; // silence before the first line, so the opening frame settles first
const GAP_MS = 450; // silence between narration lines
const TAIL_MS = 1400; // hold on the final frame after the last line finishes
@@ -63,7 +68,7 @@ function wavDurationMs(wavPath) {
function synthesizeNarration(steps, narrationDir) {
return steps.map((step, index) => {
const wavPath = join(narrationDir, `line_${String(index).padStart(2, "0")}.wav`);
execFileSync("espeak-ng", ["-v", NARRATION_VOICE, "-s", String(NARRATION_RATE), "-w", wavPath, step.text]);
execFileSync("piper", ["-m", PIPER_MODEL, "-f", wavPath], { input: step.text, stdio: ["pipe", "ignore", "ignore"] });
return { ...step, wavPath, durationMs: wavDurationMs(wavPath) };
});
}