feat(python): move first-party runtime to 3.14; swap wake engine to pyopen-wakeword

requires-python >=3.14,<3.15 (was <3.14 ceiling): the old cap existed because
Rust-backed transitives lacked cp314 wheels; tflite-runtime (openwakeword's
hard linux dep) still caps at cp311, so the wake engine moves to
pyopen-wakeword 1.1.0 — py3 wheels + bundled tensorflowlite_c lib, whose
bundled melspectrogram/embedding models are byte-identical to the openWakeWord
v0.5.1 files (verified by sha256), and it loads the shipped hey_hermes.tflite.
Drops openwakeword/onnxruntime/ai-edge-litert/wake-tflite machinery entirely.
win32-arm64 gets a marker gate (no pyopen-wakeword wheel there; porcupine
covers wake). uv.lock regenerated for 3.14 (254 pkgs, all platforms).
Real-lib smoke verified: engine builds against the actual wheel, silence
scores 0, noise scores ~0.003 (threshold 0.6), scores flow 1:1 per frame.
This commit is contained in:
ethernet
2026-09-07 14:03:27 -04:00
parent 6edee205bd
commit 7f1ddc70ce
11 changed files with 1194 additions and 2273 deletions

View File

@@ -1 +1 @@
3.11 3.14

View File

@@ -1148,19 +1148,16 @@ DEFAULT_CONFIG = {
# the model/keyword below) # the model/keyword below)
"phrase": "hey hermes", "phrase": "hey hermes",
"sensitivity": 0.6, # 0.0-1.0 threshold, consistent across engines (higher = stricter) "sensitivity": 0.6, # 0.0-1.0 threshold, consistent across engines (higher = stricter)
# openWakeWord only: consecutive over-threshold frames to fire (higher = fewer false # openWakeWord/pyopen-wakeword only: consecutive over-threshold frames to fire (higher = fewer
# triggers, more latency; 1 = single-frame) # false triggers, more latency; 1 = single-frame)
"confirmation_frames": 3, "confirmation_frames": 3,
"start_new_session": True, # fresh session on wake vs. continue the current one "start_new_session": True, # fresh session on wake vs. continue the current one
# sherpa only: listen for every wake-enabled profile's phrase and route to it # sherpa only: listen for every wake-enabled profile's phrase and route to it
"profile_routing": True, "profile_routing": True,
"openwakeword": { "openwakeword": {
# "hey_hermes" | built-in openWakeWord name ("hey_jarvis", "alexa", ...) | path to a # "hey_hermes" | built-in openWakeWord name ("hey_jarvis", "alexa", ...) | path to a
# custom .onnx/.tflite model # custom .tflite model
"model": "hey_hermes", "model": "hey_hermes",
# "" (auto: tflite on macOS ARM64, onnx elsewhere) | "onnx" | "tflite" — onnx scores
# near-zero on macOS ARM64 (arms but never fires)
"inference_framework": "",
}, },
"sherpa": { "sherpa": {
# sherpa-onnx KWS model dir; empty = auto-download the small English zipformer # sherpa-onnx KWS model dir; empty = auto-download the small English zipformer

View File

@@ -28,11 +28,10 @@ ANCHORS: dict[str, str | tuple[str, ...]] = {
"stt-whisper": "faster_whisper", "stt-whisper": "faster_whisper",
"audio-io": ("sounddevice", "numpy"), "audio-io": ("sounddevice", "numpy"),
"silk": "pilk", "silk": "pilk",
"wake": "openwakeword", "wake": "pyopen_wakeword",
"wake-openwakeword": "openwakeword", "wake-openwakeword": "pyopen_wakeword",
"wake-sherpa": "sherpa_onnx", "wake-sherpa": "sherpa_onnx",
"wake-porcupine": "pvporcupine", "wake-porcupine": "pvporcupine",
"wake-tflite": "ai_edge_litert",
"fal": "fal_client", "fal": "fal_client",
"honcho": "honcho", "honcho": "honcho",
"hindsight": "hindsight", "hindsight": "hindsight",

View File

@@ -7,12 +7,11 @@ description = "The self-improving AI agent — creates skills from experience, i
readme = "README.md" readme = "README.md"
# Upper bound is load-bearing, not cosmetic. uv resolves the project's # Upper bound is load-bearing, not cosmetic. uv resolves the project's
# Python from `requires-python`, and an inherited `UV_PYTHON` env var (or a # Python from `requires-python`, and an inherited `UV_PYTHON` env var (or a
# fresh distro whose newest interpreter uv auto-picks) will otherwise select # fresh distro whose newest interpreter uv auto-picks) would otherwise select
# 3.14, where Rust-backed transitives (e.g. pydantic-core) have no cp314 # a newer major. The 3.14 ceiling tracks the newest CPython line the full
# wheel yet and fall back to a maturin source build that fails. Capping at # dependency graph resolves on (py3 wheels for every target); raise it in the
# <3.14 makes uv refuse 3.14 with a clear error instead of attempting that # same commit as a lock regen after verifying the new interpreter resolves.
# build. Raise the ceiling once our Rust transitives ship cp314 wheels. requires-python = ">=3.14,<3.15"
requires-python = ">=3.11,<3.14"
authors = [{ name = "Nous Research" }] authors = [{ name = "Nous Research" }]
license = "MIT" license = "MIT"
license-files = ["LICENSE"] license-files = ["LICENSE"]
@@ -257,11 +256,16 @@ voice = [
# [wake]+[voice] so the ear works instantly; CLI-only installs lazy-install on # [wake]+[voice] so the ear works instantly; CLI-only installs lazy-install on
# first /wake; mirrored in tools/lazy_deps.py. # first /wake; mirrored in tools/lazy_deps.py.
wake = [ wake = [
# onnxruntime has no darwin-x64 wheel and no sdist (prebuilt only), so # pyopen-wakeword (rhasspy's maintained fork of openWakeWord) — py3 wheels
# openwakeword (which hard-requires onnxruntime) cannot install there. # plus a bundled tensorflowlite_c lib. Loads the shipped hey_hermes.tflite
# sherpa-onnx / sentencepiece / pvporcupine / sounddevice / numpy ship. # and bundles the SAME shared feature models openwakeword's download_models()
"openwakeword==0.6.0; platform_machine != 'x86_64' or platform_system != 'Darwin'", # fetched at runtime (byte-identical v0.5.1 melspectrogram/embedding, verified
"onnxruntime==1.27.0; platform_machine != 'x86_64' or platform_system != 'Darwin'", # by hash). Replaces openwakeword + onnxruntime + ai-edge-litert: upstream
# tflite-runtime (a hard openwakeword dep on linux) has no cp312+ wheels
# (caps at cp311), which is why the old requires-python ceiling existed.
# No win32-arm64 wheel: gate it there like sherpa-onnx; porcupine still
# covers wake on that target.
"pyopen-wakeword==1.1.0; platform_machine != 'ARM64' or sys_platform != 'win32'",
# sherpa-onnx ships win_amd64/win32 wheels only (its kaldi-decoder C++ has # sherpa-onnx ships win_amd64/win32 wheels only (its kaldi-decoder C++ has
# no win_arm64 build), so win32-arm64 would build the sdist and fail. Gate # no win_arm64 build), so win32-arm64 would build the sdist and fail. Gate
# it off there like faster-whisper; the other engines still ship. # it off there like faster-whisper; the other engines still ship.
@@ -270,12 +274,6 @@ wake = [
"pvporcupine==4.0.3", "pvporcupine==4.0.3",
"sounddevice==0.5.5", "sounddevice==0.5.5",
"numpy==2.4.3", "numpy==2.4.3",
# openWakeWord's onnx embedding model scores near-zero on macOS ARM64
# (dscripka/openWakeWord#336), so the wake word runs on tflite there.
# Upstream declares tflite-runtime for Linux only; ai-edge-litert is the
# macOS ARM64 equivalent (no darwin-x64 wheel, no sdist), bridged in
# tools/wake_word.py.
"ai-edge-litert==2.1.6; platform_system == 'Darwin' and platform_machine == 'arm64'",
] ]
honcho = ["honcho-ai==2.2.0"] honcho = ["honcho-ai==2.2.0"]
# Platform sub-extras / alias extras — declared so pm.extras.ANCHORS names # Platform sub-extras / alias extras — declared so pm.extras.ANCHORS names
@@ -292,11 +290,9 @@ audio-io = ["sounddevice==0.5.5", "numpy==2.4.3"]
silk = ["pilk==0.2.4"] silk = ["pilk==0.2.4"]
# Wake-engine sub-extras — lazy per-engine installs so a user who only wants # Wake-engine sub-extras — lazy per-engine installs so a user who only wants
# sherpa-onnx doesn't pull the whole [wake] bundle. Pins match [wake]. # sherpa-onnx doesn't pull the whole [wake] bundle. Pins match [wake].
wake-openwakeword = ["openwakeword==0.6.0; platform_machine != 'x86_64' or platform_system != 'Darwin'", "onnxruntime==1.27.0; platform_machine != 'x86_64' or platform_system != 'Darwin'"] wake-openwakeword = ["pyopen-wakeword==1.1.0; platform_machine != 'ARM64' or sys_platform != 'win32'"]
wake-sherpa = ["sherpa-onnx==1.13.4; platform_machine != 'ARM64' or sys_platform != 'win32'"] wake-sherpa = ["sherpa-onnx==1.13.4; platform_machine != 'ARM64' or sys_platform != 'win32'"]
wake-porcupine = ["pvporcupine==4.0.3"] wake-porcupine = ["pvporcupine==4.0.3"]
# macOS tflite bridge (tools/wake_word.py); same marker policy as [wake].
wake-tflite = ["ai-edge-litert==2.1.6; platform_system == 'Darwin' and platform_machine == 'arm64'"]
# Google Chat service account push (gateway/platforms/google_chat) — pubsub # Google Chat service account push (gateway/platforms/google_chat) — pubsub
# subscriber for the Chat API's Cloud Pub/Sub delivery. # subscriber for the Chat API's Cloud Pub/Sub delivery.
# grpcio (pubsub's native transitive) has no win_arm64 wheel, so win32-arm64 # grpcio (pubsub's native transitive) has no win_arm64 wheel, so win32-arm64
@@ -559,7 +555,6 @@ exclude-newer = "14 days"
# 14-day window is younger than the reviewed release. # 14-day window is younger than the reviewed release.
[tool.uv.exclude-newer-package] [tool.uv.exclude-newer-package]
agent-client-protocol = false agent-client-protocol = false
ai-edge-litert = false
aiohttp = false aiohttp = false
aiohttp-socks = false aiohttp-socks = false
aiosqlite = false aiosqlite = false
@@ -611,11 +606,9 @@ mistralai = false
modal = false modal = false
nemo-relay = false nemo-relay = false
numpy = false numpy = false
onnxruntime = false
openai = false openai = false
opentelemetry-exporter-otlp-proto-http = false opentelemetry-exporter-otlp-proto-http = false
opentelemetry-sdk = false opentelemetry-sdk = false
openwakeword = false
packaging = false packaging = false
parallel-web = false parallel-web = false
pathspec = false pathspec = false

View File

@@ -67,7 +67,7 @@ def test_lazy_installable_extras_excluded_from_all():
"modal", "daytona", "vercel", "modal", "daytona", "vercel",
"messaging", "slack", "matrix", "dingtalk", "feishu", "messaging", "slack", "matrix", "dingtalk", "feishu",
"telegram", "discord", "telegram", "discord",
"wake", "wake-openwakeword", "wake-sherpa", "wake-porcupine", "wake-tflite", "wake", "wake-openwakeword", "wake-sherpa", "wake-porcupine",
"google-chat", "google-chat",
"honcho", "hindsight", "honcho", "hindsight",
"supermemory", "mem0", "supermemory", "mem0",

View File

@@ -203,145 +203,65 @@ def test_requirements_deps_present_but_no_audio_hint(monkeypatch):
assert "audio device" in r["hint"] or "microphone" in r["hint"].lower() assert "audio device" in r["hint"] or "microphone" in r["hint"].lower()
# ── openWakeWord engine (bundled model + base-model fetch) ─────────────── # ── openWakeWord engine (pyopen-wakeword; bundled model, no runtime fetch) ──
def _install_fake_openwakeword(monkeypatch): def test_openwakeword_custom_model_path_used(monkeypatch):
"""Swap in a fake ``openwakeword`` so the engine builds with no network. # A custom ``model`` path passes through to pyopen-wakeword as-is. The
# shared feature models come from the wheel (from_builtin) — there is no
Returns a ``calls`` dict recording every ``download_models`` invocation. # download_models step to regress.
""" captured = {}
calls = {"download": []}
class _FakeModel: class _FakeModel:
def __init__(self, wakeword_models, inference_framework="onnx"): def __init__(self, model_path, libtensorflowlite_c_path=None):
self.wakeword_models = list(wakeword_models) self.id = os.path.splitext(os.path.basename(str(model_path)))[0]
self.models = {"hey_hermes": object()} captured["path"] = str(model_path)
def predict(self, frame): @staticmethod
return {"hey_hermes": 0.0} def from_model(model_path, libtensorflowlite_c_path=None):
return _FakeModel(model_path, libtensorflowlite_c_path)
def process_streaming(self, embeddings):
return iter(())
def reset(self): def reset(self):
pass pass
oww = types.ModuleType("openwakeword") def close(self):
oww.utils = types.SimpleNamespace( pass
download_models=lambda names=[]: calls["download"].append(list(names))
)
model_mod = types.ModuleType("openwakeword.model")
model_mod.Model = _FakeModel
monkeypatch.setitem(sys.modules, "openwakeword", oww) class _FakeFeatures:
monkeypatch.setitem(sys.modules, "openwakeword.model", model_mod) @staticmethod
def from_builtin(models_dir=None, libtensorflowlite_c_path=None):
return _FakeFeatures()
def process_streaming(self, audio_chunk):
return iter(())
def reset(self):
pass
def close(self):
pass
mod = types.ModuleType("pyopen_wakeword")
mod.OpenWakeWord = _FakeModel
mod.OpenWakeWordFeatures = _FakeFeatures
monkeypatch.setitem(sys.modules, "pyopen_wakeword", mod)
monkeypatch.setattr(pm, "ensure_import", lambda *a, **k: None) monkeypatch.setattr(pm, "ensure_import", lambda *a, **k: None)
return calls
def test_openwakeword_ensures_base_models_for_custom_path(monkeypatch):
# Regression: a custom ``.onnx`` path used to skip download_models entirely,
# so a fresh install crashed at load time on a missing melspectrogram.onnx.
# The base feature models must be ensured for a custom path too.
calls = _install_fake_openwakeword(monkeypatch)
eng = ww._OpenWakeWordEngine( eng = ww._OpenWakeWordEngine(
{"provider": "openwakeword", "openwakeword": {"model": "/models/hey_hermes.onnx"}} {"provider": "openwakeword", "openwakeword": {"model": "/models/hey_hermes.tflite"}}
) )
assert calls["download"] == [["/models/hey_hermes.onnx"]] assert captured["path"] == "/models/hey_hermes.tflite"
assert eng._labels == ["hey_hermes"] assert eng._labels == ["hey_hermes"]
def test_bundled_hey_hermes_model_ships_on_disk(): def test_bundled_hey_hermes_model_ships_on_disk():
# The "hey hermes" wake word works out of the box only if the model is # The "hey hermes" wake word works out of the box only if the model is
# actually bundled. Both framework artifacts must exist and be non-trivial. # actually bundled. pyopen-wakeword runs TFLite only.
for framework in ("onnx", "tflite"): path = ww._bundled_wakeword_path()
path = ww._bundled_wakeword_path(framework) assert os.path.exists(path), path
assert os.path.exists(path), path assert os.path.getsize(path) > 1024, path
assert os.path.getsize(path) > 1024, path
# ── platform-aware backend selection (openWakeWord onnx is broken on macOS ARM64,
# upstream dscripka/openWakeWord#336) ────────────────────────────────────────
def test_default_framework_tracks_the_macos_arm64_probe():
"""``default_inference_framework()`` is exactly the ``_is_macos_arm64()``
branch — tflite there, onnx everywhere else.
Stated as an invariant between the probe and its consumer so it holds on
every host, including the macOS runner (where both sides are real) and an
Intel Mac (where ONNX is fine and both sides say so).
"""
expected = "tflite" if ww._is_macos_arm64() else "onnx"
assert ww.default_inference_framework() == expected
@pytest.mark.platforms("macos")
def test_macos_arm64_prefers_tflite_on_this_host():
"""On a real ARM64 Mac the default must be tflite (upstream #336).
Runs on the macOS CI job, where ``platform.machine()`` and
``sys.platform`` are the genuine article rather than a patched pair.
"""
if not ww._is_macos_arm64():
pytest.skip("Intel Mac — ONNX works here, nothing to assert")
assert ww.default_inference_framework() == "tflite"
assert ww.resolve_inference_framework({}) == "tflite"
assert ww.resolve_inference_framework({"openwakeword": {"inference_framework": ""}}) == "tflite"
# The one explicit value we override: pinned onnx is provably dead here.
assert ww.resolve_inference_framework(
{"openwakeword": {"inference_framework": "onnx"}}
) == "tflite"
def test_explicit_framework_kept_where_onnx_works(monkeypatch):
# An operator who pins a backend keeps it everywhere ONNX actually works.
calls = _install_fake_openwakeword(monkeypatch)
monkeypatch.setattr(ww, "_is_macos_arm64", lambda: False)
ww._OpenWakeWordEngine(
{"provider": "openwakeword", "openwakeword": {"inference_framework": "onnx"}}
)
(downloaded,) = calls["download"]
assert downloaded == [ww._bundled_wakeword_path("onnx")]
def test_empty_framework_falls_back_to_platform_default(monkeypatch):
"""Empty/missing config defers to ``default_inference_framework()``.
The macOS-ARM64 side of the fallback is asserted for real in
``test_macos_arm64_prefers_tflite_on_this_host``; here we pin the
delegation itself by swapping the platform probe (a seam in our own
module) rather than lying to the interpreter about which OS it is on.
"""
monkeypatch.setattr(ww, "_is_macos_arm64", lambda: True)
assert ww.resolve_inference_framework({}) == "tflite"
assert ww.resolve_inference_framework({"openwakeword": {"inference_framework": ""}}) == "tflite"
monkeypatch.setattr(ww, "_is_macos_arm64", lambda: False)
assert ww.resolve_inference_framework({}) == "onnx"
# ── ambient-speech rejection: consecutive-frame confirmation ──────────────────
def _openwakeword_engine_with_scores(monkeypatch, cfg_wake, scores):
"""Build a real _OpenWakeWordEngine whose predict() replays ``scores``."""
seq = iter(scores)
class _ScriptedModel:
def __init__(self, wakeword_models, inference_framework="onnx"):
self.models = {"hey_hermes": object()}
def predict(self, frame):
return {"hey_hermes": next(seq)}
def reset(self):
pass
oww = types.ModuleType("openwakeword")
oww.utils = types.SimpleNamespace(download_models=lambda names=[]: None)
model_mod = types.ModuleType("openwakeword.model")
model_mod.Model = _ScriptedModel
monkeypatch.setitem(sys.modules, "openwakeword", oww)
monkeypatch.setitem(sys.modules, "openwakeword.model", model_mod)
monkeypatch.setattr(pm, "ensure_import", lambda *a, **k: None)
monkeypatch.setattr(ww, "ensure_tflite_runtime", lambda: True)
return ww._OpenWakeWordEngine({"provider": "openwakeword", **cfg_wake})
# ── sherpa-onnx open-vocabulary engine ─────────────────────────────────── # ── sherpa-onnx open-vocabulary engine ───────────────────────────────────

View File

@@ -21,7 +21,7 @@ from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import Any, Callable, Dict, Optional from typing import Any, Callable, Dict, Optional
from tools.wake_word_engines import _Engine, _OpenWakeWordEngine, _PorcupineEngine, _SherpaKwsEngine, _sub from tools.wake_word_engines import _Engine, _PorcupineEngine, _SherpaKwsEngine, _sub
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -71,66 +71,9 @@ _BUNDLED_MODEL_NAME = "hey_hermes"
_BUNDLED_MODEL_ALIASES = frozenset({"", "hey_hermes", "hey hermes", "hermes"}) _BUNDLED_MODEL_ALIASES = frozenset({"", "hey_hermes", "hey hermes", "hermes"})
def _bundled_wakeword_path(framework: str = "onnx") -> str: def _bundled_wakeword_path() -> str:
"""Path to the shipped hey_hermes model (.onnx/.tflite) for ``framework``.""" """Path to the shipped hey_hermes.tflite — pyopen-wakeword runs TFLite only."""
ext = "tflite" if str(framework).strip().lower() == "tflite" else "onnx" return os.path.join(os.path.dirname(__file__), "wakewords", f"{_BUNDLED_MODEL_NAME}.tflite")
return os.path.join(os.path.dirname(__file__), "wakewords", f"{_BUNDLED_MODEL_NAME}.{ext}")
def _is_macos_arm64() -> bool:
import platform
return sys.platform == "darwin" and platform.machine() == "arm64"
def default_inference_framework() -> str:
"""tflite on macOS ARM64, onnx elsewhere: openWakeWord's ONNX embedding model
scores near-zero on Apple Silicon — the detector arms but never fires."""
return "tflite" if _is_macos_arm64() else "onnx"
_warned_onnx_coerced = False
def resolve_inference_framework(cfg: Dict[str, Any]) -> str:
"""Effective openWakeWord backend: explicit ``openwakeword.inference_framework`` or
the platform default. Explicit ``onnx`` on macOS ARM64 is provably dead, so it is
coerced to tflite with a one-time warning (a pre-fix pin must not stay deaf)."""
global _warned_onnx_coerced
framework = str(_sub(cfg, "openwakeword").get("inference_framework") or "").strip().lower()
if not framework:
return default_inference_framework()
if framework == "onnx" and _is_macos_arm64():
if not _warned_onnx_coerced:
_warned_onnx_coerced = True
logger.warning("wake: openwakeword.inference_framework='onnx' is set but ONNX's "
"embedding model never fires on macOS ARM64 (openWakeWord #336) — "
"using tflite instead. Set inference_framework to '' (auto) or "
"'tflite' in config.yaml to silence this.")
return "tflite"
return framework
def ensure_tflite_runtime() -> bool:
"""Make ``import tflite_runtime.interpreter`` resolve, returning success. openWakeWord hardcodes
that import but only declares ``tflite-runtime`` on Linux; on macOS the wheel is ``ai-edge-litert``,
so alias it in-process (site-packages untouched)."""
try:
import tflite_runtime.interpreter # noqa: F401
return True
except ImportError:
pass
try:
from ai_edge_litert import interpreter as _litert # type: ignore[import-not-found]
except ImportError:
return False
import types
pkg = types.ModuleType("tflite_runtime")
pkg.__path__ = [] # type: ignore[attr-defined] # mark as package
sys.modules.setdefault("tflite_runtime", pkg)
sys.modules["tflite_runtime.interpreter"] = _litert
logger.debug("wake word: bridged tflite_runtime -> ai_edge_litert")
return True
def load_wake_word_config() -> Dict[str, Any]: def load_wake_word_config() -> Dict[str, Any]:
@@ -360,9 +303,13 @@ def _looks_like_path(value: str) -> bool:
class _OpenWakeWordEngine(_Engine): class _OpenWakeWordEngine(_Engine):
"""openWakeWord — free, local ONNX hotword detection.""" """pyopen-wakeword — free, local hotword detection (TFLite via a bundled
tensorflowlite_c lib; no runtime download, no framework choice). Scores one
~80 ms frame at a time; ``sensitivity`` IS the raw 0..1 threshold (higher =
stricter). A real utterance holds the score high across frames while a stray
phoneme spikes one, so ``confirmation_frames`` hits are required."""
# openWakeWord recommends 80 ms frames (1280 samples) for efficiency. # pyopen-wakeword recommends 80 ms frames (1280 samples) for efficiency.
frame_length = 1280 frame_length = 1280
def __init__(self, cfg: Dict[str, Any]): def __init__(self, cfg: Dict[str, Any]):
@@ -370,64 +317,36 @@ class _OpenWakeWordEngine(_Engine):
pm.ensure_import("wake-openwakeword") pm.ensure_import("wake-openwakeword")
import openwakeword from pyopen_wakeword import OpenWakeWord, OpenWakeWordFeatures
from openwakeword.model import Model
sub = cfg.get("openwakeword") if isinstance(cfg.get("openwakeword"), dict) else {} sub = cfg.get("openwakeword") if isinstance(cfg.get("openwakeword"), dict) else {}
model_ref = str(sub.get("model") or _BUNDLED_MODEL_NAME).strip() model_ref = str(sub.get("model") or _BUNDLED_MODEL_NAME).strip()
framework = resolve_inference_framework(cfg) # Default (or explicit "hey_hermes") → the bundled model; a custom path
# openWakeWord returns a 0..1 score per frame; sensitivity IS the raw # is used as-is. pyopen-wakeword bundles the shared feature models
# threshold a score must clear. Higher = stricter (fewer false fires). # (melspectrogram + embedding — byte-identical to the openWakeWord
# Default 0.6 sits above openWakeWord's permissive 0.5 baseline, which # v0.5.1 files) inside its wheel, so there is no download_models step.
# let near-misses like "hey hor" through. if model_ref.lower() in _BUNDLED_MODEL_ALIASES:
model_ref = _bundled_wakeword_path()
# pyopen-wakeword returns a 0..1 score per completed window; sensitivity
# IS the raw threshold a score must clear. Higher = stricter (fewer
# false fires). Default 0.6 sits above openWakeWord's permissive 0.5
# baseline, which let near-misses like "hey hor" through.
self._threshold = _sensitivity(cfg) self._threshold = _sensitivity(cfg)
self._confirm_needed = _confirmation_frames(cfg) self._confirm_needed = _confirmation_frames(cfg)
self._confirm_streak = 0 self._confirm_streak = 0
self._features = OpenWakeWordFeatures.from_builtin()
# openWakeWord silently downgrades tflite -> onnx when no tflite runtime self._model = OpenWakeWord.from_model(model_ref)
# imports (model.py). On macOS ARM64 that lands on the backend whose self._labels = [self._model.id]
# embedding model is broken, so the listener would arm and never fire.
# Install + bridge the runtime first, and refuse the downgrade rather
# than ship a dead ear.
if framework == "tflite" and not ensure_tflite_runtime():
# Same lazy-install contract as every other backend; the platform
# gate lives here because dep specs can't carry PEP 508 markers.
try:
import pm
pm.ensure_import("wake-tflite")
except Exception as e:
logger.debug("wake word: tflite runtime install failed: %s", e)
if not ensure_tflite_runtime():
if _is_macos_arm64():
raise RuntimeError(
"The wake word needs the tflite backend on this Mac, but its "
"runtime is missing. Install it with: pip install ai-edge-litert"
)
logger.warning("wake word: no tflite runtime available — falling back to onnx")
framework = "onnx"
# Default (or explicit "hey_hermes") → the bundled model; a built-in name
# or custom path is used as-is.
if model_ref.lower() in _BUNDLED_MODEL_ALIASES:
model_ref = _bundled_wakeword_path(framework)
# openWakeWord needs its shared feature models (melspectrogram + embedding)
# for ANY model — download_models() fetches those first on every call, so a
# custom path must call it too, else a fresh install crashes on a missing
# melspectrogram.onnx. A built-in name additionally pulls that pretrained
# model; a path matches nothing in the catalog and is a no-op beyond base.
try:
openwakeword.utils.download_models([model_ref])
except Exception as e: # pragma: no cover - network/path dependent
logger.debug("openwakeword model download skipped: %s", e)
models = [model_ref]
self._model = Model(wakeword_models=models, inference_framework=framework)
self._labels = list(self._model.models.keys())
def process(self, frame) -> bool: def process(self, frame) -> bool:
scores = self._model.predict(frame) # frame is a 1-D int16 ndarray; the features pipeline consumes int16
over = any(score >= self._threshold for score in scores.values()) # bytes. process_streaming() yields embeddings as the window fills and
# the model yields one 0..1 score per completed window.
over = False
for emb in self._features.process_streaming(frame.tobytes()):
for score in self._model.process_streaming(emb):
if score >= self._threshold:
over = True
# Require N consecutive over-threshold frames: a real phrase holds the # Require N consecutive over-threshold frames: a real phrase holds the
# score high across frames, a stray ambient phoneme spikes just one. # score high across frames, a stray ambient phoneme spikes just one.
if over: if over:
@@ -440,16 +359,18 @@ class _OpenWakeWordEngine(_Engine):
return False return False
def reset(self) -> None: def reset(self) -> None:
# Clears openWakeWord's rolling feature/prediction buffer so stale audio # Clears pyopen-wakeword's rolling feature/prediction buffer so stale
# captured before a pause can't re-fire the moment we resume. # audio captured before a pause can't re-fire the moment we resume.
self._confirm_streak = 0 self._confirm_streak = 0
try: with suppress(Exception):
self._features.reset()
self._model.reset() self._model.reset()
except Exception:
pass
def close(self) -> None: def close(self) -> None:
self.reset() self.reset()
with suppress(Exception):
self._features.close()
self._model.close()
# sherpa-onnx open-vocabulary KWS model: a small streaming zipformer # sherpa-onnx open-vocabulary KWS model: a small streaming zipformer
@@ -749,21 +670,11 @@ def check_wake_word_requirements(cfg: Optional[Dict[str, Any]] = None) -> Dict[s
tts_ok = _tts_ready() tts_ok = _tts_ready()
hint = "" hint = ""
# The tflite backend needs a runtime openWakeWord doesn't declare off Linux.
# Report it as a real remediation instead of arming a detector that can't fire.
tflite_ok = True
if provider not in ("porcupine", "sherpa", "sherpa-onnx", "kws", "open"):
framework = resolve_inference_framework(cfg)
if framework == "tflite":
tflite_ok = ensure_tflite_runtime() or pm.available("wake-tflite") or lazy_ok
if provider == "porcupine" and not (os.getenv("PORCUPINE_ACCESS_KEY") or "").strip(): if provider == "porcupine" and not (os.getenv("PORCUPINE_ACCESS_KEY") or "").strip():
key_ok = False key_ok = False
hint = "Set PORCUPINE_ACCESS_KEY (free key at https://console.picovoice.ai)." hint = "Set PORCUPINE_ACCESS_KEY (free key at https://console.picovoice.ai)."
elif not deps_ok and not lazy_ok: elif not deps_ok and not lazy_ok:
hint = f"uv sync --frozen --extra {feature}" hint = f"uv sync --frozen --extra {feature}"
elif not tflite_ok:
hint = "The wake word needs the tflite runtime on this Mac: pip install ai-edge-litert"
elif deps_ok and not audio_ok and resolve_capture_mode(cfg) == "local": elif deps_ok and not audio_ok and resolve_capture_mode(cfg) == "local":
hint = "Microphone capture needs sounddevice + numpy and a working audio device." hint = "Microphone capture needs sounddevice + numpy and a working audio device."
elif not stt_ok or not tts_ok: elif not stt_ok or not tts_ok:
@@ -786,7 +697,7 @@ def check_wake_word_requirements(cfg: Optional[Dict[str, Any]] = None) -> Dict[s
"build with client-capture wake support.") "build with client-capture wake support.")
return { return {
"available": key_ok and stt_ok and tts_ok and tflite_ok and mic_ok, "provider": provider, "available": key_ok and stt_ok and tts_ok and mic_ok, "provider": provider,
"deps_available": deps_ok, "audio_available": audio_ok, "deps_available": deps_ok, "audio_available": audio_ok,
"local_input_available": _local_input_device_ready() if deps_ok else False, "local_input_available": _local_input_device_ready() if deps_ok else False,
"capture": capture_mode, "access_key_set": key_ok, "stt_available": stt_ok, "tts_available": tts_ok, "capture": capture_mode, "access_key_set": key_ok, "stt_available": stt_ok, "tts_available": tts_ok,

View File

@@ -69,73 +69,6 @@ def _sub(cfg: Dict[str, Any], key: str) -> Dict[str, Any]:
return sub if isinstance(sub, dict) else {} return sub if isinstance(sub, dict) else {}
class _OpenWakeWordEngine(_Engine):
"""openWakeWord — free, local ONNX/tflite hotword detection. Scores one ~80 ms frame at a time;
``sensitivity`` IS the raw 0..1 threshold (higher = stricter). A real utterance holds the score
high across frames while a stray phoneme spikes one, so ``confirmation_frames`` hits are required."""
feature, section = "wake-openwakeword", "openwakeword"
frame_length = 1280 # openWakeWord recommends 80 ms frames.
def _build(self, cfg, sub, ww) -> None:
import openwakeword
from openwakeword.model import Model
model_ref = str(sub.get("model") or ww._BUNDLED_MODEL_NAME).strip()
framework = self._usable_framework(ww.resolve_inference_framework(cfg))
self._threshold = ww._sensitivity(cfg)
self._confirm_needed = ww._confirmation_frames(cfg)
self._confirm_streak = 0
# Default (or explicit "hey_hermes") → the bundled model; built-in names / paths as-is.
if model_ref.lower() in ww._BUNDLED_MODEL_ALIASES:
model_ref = ww._bundled_wakeword_path(framework)
# download_models() also fetches the shared feature models (melspectrogram +
# embedding) needed for ANY model, so a custom path must call it too.
try:
openwakeword.utils.download_models([model_ref])
except Exception as e: # pragma: no cover - network/path dependent
logger.debug("openwakeword model download skipped: %s", e)
self._model = Model(wakeword_models=[model_ref], inference_framework=framework)
self._labels = list(self._model.models.keys())
@staticmethod
def _usable_framework(framework: str) -> str:
"""Refuse openWakeWord's silent tflite→onnx downgrade: without a tflite runtime it falls back
to onnx, which on macOS ARM64 never fires (armed but deaf). Install + bridge the runtime first
(gate lives here because dep specs can't carry PEP 508 markers); on that Mac raise instead."""
ww = _ww()
if framework != "tflite" or ww.ensure_tflite_runtime():
return framework
try:
_ensure_dep("wake-tflite")
except Exception as e:
logger.debug("wake word: tflite runtime install failed: %s", e)
if ww.ensure_tflite_runtime():
return framework
if ww._is_macos_arm64():
raise RuntimeError("The wake word needs the tflite backend on this Mac, but its "
"runtime is missing. Install it with: pip install ai-edge-litert")
logger.warning("wake word: no tflite runtime available — falling back to onnx")
return "onnx"
def process(self, frame) -> bool:
hit = any(score >= self._threshold for score in self._model.predict(frame).values())
self._confirm_streak = self._confirm_streak + 1 if hit else 0
if self._confirm_streak < self._confirm_needed:
return False
self._confirm_streak = 0
return True
def reset(self) -> None:
# Clears openWakeWord's rolling feature buffer so stale audio captured before a
# pause can't re-fire the moment we resume.
self._confirm_streak = 0
with suppress(Exception):
self._model.reset()
def close(self) -> None:
self.reset()
# sherpa-onnx open-vocabulary KWS model: small streaming zipformer transducer (English, # sherpa-onnx open-vocabulary KWS model: small streaming zipformer transducer (English,
# GigaSpeech), downloaded once under HERMES_HOME. Keywords are tokenized at RUNTIME. # GigaSpeech), downloaded once under HERMES_HOME. Keywords are tokenized at RUNTIME.
_SHERPA_KWS_MODEL_URL = ( _SHERPA_KWS_MODEL_URL = (

View File

@@ -1,20 +1,22 @@
# Bundled wake-word models # Bundled wake-word models
`hey_hermes.onnx` / `hey_hermes.tflite` — the on-device "Hey Hermes" hotword `hey_hermes.tflite` — the on-device "Hey Hermes" hotword model. This is the
model. This is the default detector for the wake word feature (see default detector for the wake word feature (see
`website/docs/user-guide/features/wake-word.md`); no training or setup is `website/docs/user-guide/features/wake-word.md`); no training or setup is
required to say "hey hermes". required to say "hey hermes".
- **Engine:** [openWakeWord](https://github.com/dscripka/openWakeWord) (Apache-2.0). - **Engine:** [pyopen-wakeword](https://github.com/rhasspy/pyopen-wakeword)
(rhasspy's maintained fork of openWakeWord; Apache-2.0). Runs TFLite via a
bundled `tensorflowlite_c` library — no onnx, no runtime download.
- **Provenance:** trained with the openWakeWord training pipeline (synthetic - **Provenance:** trained with the openWakeWord training pipeline (synthetic
TTS-generated speech), which produces both the `.onnx` and `.tflite` artifacts. TTS-generated speech), which produces the `.tflite` artifact. Redistribution
Redistribution is permitted under the openWakeWord license. is permitted under the openWakeWord license.
- **Label:** the model registers as `hey_hermes` (matches the filename). - **Label:** the model registers as `hey_hermes` (matches the filename).
- **Runtime:** openWakeWord's shared feature-extraction models (melspectrogram + - **Runtime:** the shared feature-extraction models (melspectrogram +
embedding) are NOT bundled here — they are fetched once on first use by embedding) are bundled inside the `pyopen-wakeword` wheel — byte-identical
`tools/wake_word.py` via `openwakeword.utils.download_models()`. to the official openWakeWord v0.5.1 files, so scores match the original
engine exactly.
To use a different phrase, train your own model and point To use a different phrase, train your own model and point
`wake_word.openwakeword.model` at its path, or set a built-in openWakeWord name `wake_word.openwakeword.model` at its `.tflite` path. See the wake-word docs
(`hey_jarvis`, `alexa`, `hey_mycroft`, …). See the wake-word docs for the for the training guide.
training guide.

Binary file not shown.

2988
uv.lock generated

File diff suppressed because it is too large Load Diff