Switch the video guide narrator from espeak-ng to Piper (neural TTS)
espeak-ng's formant synthesis was reported as barely intelligible. Piper is a small local neural TTS with the same offline, no-cloud-service properties but far more natural speech; its default WAV output format already matched what the sync pipeline assumed, so only the synthesis call itself needed to change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -8,6 +8,12 @@ The application version is maintained in `package.json` and displayed in the adm
|
||||
|
||||
No changes recorded yet.
|
||||
|
||||
## 1.18.2 - 2026-09-11
|
||||
|
||||
### Changed
|
||||
|
||||
- `scripts/record-video-guides.mjs` now narrates with Piper (a small local neural TTS) instead of espeak-ng - espeak-ng's formant synthesis was reported as barely intelligible ("robotröst från 80-talet"). Requires `pipx install piper-tts` plus a downloaded voice model; see the script's header comment.
|
||||
|
||||
## 1.18.1 - 2026-09-11
|
||||
|
||||
### Changed
|
||||
|
||||
@@ -454,6 +454,15 @@ The user's original ask ("a video guide showing how to do the various things") m
|
||||
|
||||
Verified 2026-09-11: re-recorded all three clips and confirmed each now has both a video and an AAC audio stream (`ffprobe`). Extracted the volume envelope of the settings-tabs clip's audio and confirmed 5 distinct speech bursts separated by clean silence, matching its 5 narration lines. Cross-checked video frames against the audio timeline: at 8s into settings-tabs (mid second line, "Audio controls race sound announcements...") the Audio tab is the one on screen; at 20s into event-manager-tabs (the Schedule line) the Schedule tab/panel is on screen; at 12s into event-schedule (the "enter a start time" line) the form is visible filled in but not yet saved - narration and visuals line up correctly throughout, not just at the start.
|
||||
|
||||
### Second follow-up: espeak-ng's voice was reported barely intelligible
|
||||
|
||||
Reported 2026-09-11 by the user right after listening to the first narrated batch: "går ju knappt höra va den säger.. robot röst från 80 talet" (can barely make out what it's saying - a voice like a robot from the 80s). espeak-ng is formant synthesis (the same family as the original DECtalk/Speak & Spell era of TTS this compares it to) - accurate about what it is, not a bug, just the wrong tool for something meant to be listened to and understood.
|
||||
|
||||
- [x] Installed Piper (`pipx install piper-tts`, an isolated CLI install via pipx rather than a system-wide pip install) - a small local neural (VITS-based) TTS that produces natural-sounding speech offline, no cloud service involved. Downloaded the `en_US-lessac-medium` voice model from the `rhasspy/piper-voices` collection on Hugging Face to `~/.local/share/piper-voices/` (a one-time, machine-local setup step - the ~60MB model isn't part of the repo).
|
||||
- [x] Swapped `synthesizeNarration` in `scripts/record-video-guides.mjs` from shelling out to `espeak-ng -w` to piping each line into `piper -m <model> -f <wav>` over stdin. Piper's default output format (22050Hz mono 16-bit PCM WAV) matches what the rest of the pipeline (the `adelay`/`amix` sync logic, `wavDurationMs`) already assumed, so no other part of the recorder needed to change.
|
||||
|
||||
Verified 2026-09-11: re-recorded all three clips with Piper and confirmed each still has a valid video+AAC-audio MKV (`ffprobe`). Re-checked the settings-tabs clip's volume envelope - still 5 clean speech bursts in the right places, now visibly fuller/louder (Piper's neural waveform vs. espeak-ng's thin formant one), confirming the sync logic itself didn't need to change, only the synthesis backend.
|
||||
|
||||
## Documentation maintenance
|
||||
|
||||
- [ ] Update the architecture's module/dependency inventory as modules change.
|
||||
|
||||
4
package-lock.json
generated
4
package-lock.json
generated
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "rc-racecore",
|
||||
"version": "1.18.1",
|
||||
"version": "1.18.2",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "rc-racecore",
|
||||
"version": "1.18.1",
|
||||
"version": "1.18.2",
|
||||
"hasInstallScript": true,
|
||||
"dependencies": {
|
||||
"better-sqlite3": "^12.11.1",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "rc-racecore",
|
||||
"version": "1.18.1",
|
||||
"version": "1.18.2",
|
||||
"private": true,
|
||||
"description": "RC RaceCore live timing and race control with AMMC ingest and SQLite persistence",
|
||||
"engines": {
|
||||
|
||||
@@ -4,23 +4,28 @@
|
||||
// ever touches a real installation's data.
|
||||
//
|
||||
// Each clip is a list of steps ({ text, action }). For every step, the narration line's
|
||||
// audio is synthesized up front (espeak-ng) so its real duration is known, the on-screen
|
||||
// action runs, and the recording pauses for exactly that duration before moving to the
|
||||
// next line - the actual wall-clock time each action took is recorded too, so the final
|
||||
// narration track is built from real measured offsets (via ffmpeg's adelay/amix), not an
|
||||
// idealized schedule that would drift out of sync with page-load/network latency.
|
||||
// audio is synthesized up front (Piper, a small local neural TTS - far more intelligible
|
||||
// than espeak-ng's formant synthesis, which is what this used before) so its real duration
|
||||
// is known, the on-screen action runs, and the recording pauses for exactly that duration
|
||||
// before moving to the next line - the actual wall-clock time each action took is recorded
|
||||
// too, so the final narration track is built from real measured offsets (via ffmpeg's
|
||||
// adelay/amix), not an idealized schedule that would drift out of sync with page-load/
|
||||
// network latency.
|
||||
//
|
||||
// Usage: node scripts/record-video-guides.mjs [clip-name ...]
|
||||
// With no arguments, records every clip below. Pass one or more names (e.g.
|
||||
// "settings-tabs") to record only those. Output lands in video-guides/<name>.mkv
|
||||
// (gitignored - these are sizeable binaries, not something to commit by default).
|
||||
//
|
||||
// Requires ffmpeg and espeak-ng on PATH.
|
||||
// Requires ffmpeg and Piper (`pipx install piper-tts`) on PATH, plus a downloaded voice
|
||||
// model - by default ~/.local/share/piper-voices/en_US-lessac-medium.onnx (get it from
|
||||
// https://huggingface.co/rhasspy/piper-voices, the .onnx and its matching .onnx.json).
|
||||
// Override the model with the PIPER_MODEL env var.
|
||||
|
||||
import { spawn, execFileSync } from "node:child_process";
|
||||
import { existsSync, mkdtempSync, rmSync, mkdirSync, writeFileSync } from "node:fs";
|
||||
import { createServer } from "node:net";
|
||||
import { tmpdir } from "node:os";
|
||||
import { homedir, tmpdir } from "node:os";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
@@ -28,10 +33,10 @@ const projectDir = dirname(dirname(fileURLToPath(import.meta.url)));
|
||||
const outputDir = join(projectDir, "video-guides");
|
||||
const chrome = [process.env.CHROME_PATH, "/usr/bin/google-chrome", "/usr/bin/chromium", "/usr/bin/chromium-browser"].find((p) => p && existsSync(p));
|
||||
if (!chrome) throw new Error("Chrome/Chromium not found. Set CHROME_PATH to the browser executable.");
|
||||
try { execFileSync("espeak-ng", ["--version"], { stdio: "ignore" }); } catch { throw new Error("espeak-ng not found on PATH (used for narration audio)."); }
|
||||
try { execFileSync("piper", ["--help"], { stdio: "ignore" }); } catch { throw new Error("piper not found on PATH (used for narration audio). Install with: pipx install piper-tts"); }
|
||||
const PIPER_MODEL = process.env.PIPER_MODEL || join(homedir(), ".local", "share", "piper-voices", "en_US-lessac-medium.onnx");
|
||||
if (!existsSync(PIPER_MODEL)) throw new Error(`Piper voice model not found at ${PIPER_MODEL}. Download it from https://huggingface.co/rhasspy/piper-voices (both the .onnx and .onnx.json files), or set PIPER_MODEL.`);
|
||||
|
||||
const NARRATION_VOICE = "en-us";
|
||||
const NARRATION_RATE = 160;
|
||||
const LEAD_MS = 600; // silence before the first line, so the opening frame settles first
|
||||
const GAP_MS = 450; // silence between narration lines
|
||||
const TAIL_MS = 1400; // hold on the final frame after the last line finishes
|
||||
@@ -63,7 +68,7 @@ function wavDurationMs(wavPath) {
|
||||
function synthesizeNarration(steps, narrationDir) {
|
||||
return steps.map((step, index) => {
|
||||
const wavPath = join(narrationDir, `line_${String(index).padStart(2, "0")}.wav`);
|
||||
execFileSync("espeak-ng", ["-v", NARRATION_VOICE, "-s", String(NARRATION_RATE), "-w", wavPath, step.text]);
|
||||
execFileSync("piper", ["-m", PIPER_MODEL, "-f", wavPath], { input: step.text, stdio: ["pipe", "ignore", "ignore"] });
|
||||
return { ...step, wavPath, durationMs: wavDurationMs(wavPath) };
|
||||
});
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user