A local pipeline can transcribe speech in about 100 ms and decide what it means in roughly 250 ms when the only model in the loop is making a decision. The Home Assistant agents and voice tools here converge on that division of labor: model decides, code executes, generation held in reserve. The best keep the model advisory, behind a policy check.
One call per utterance
The custom conversation agent in ha-conversation-jev makes one system_one call per utterance on jev-latest, with a ten-second timeout; it classifies each utterance into a router kind — fast_service, grok, or reject. Only the fast path executes device services directly — light, climate, and cover operations — while the rest hand the conversation to a SpaceXAI Grok agent or turn it down. The boundaries are explicit: FAST_MIN_CONFIDENCE=0.80, NOUL_YES_THRESHOLD=0.55, NOUL_UNSURE_LOW=0.40. State is a JSON object of utterance, language, entities, and areas, and multi-area requests stay a fast_service union of the named rooms' exposed lights even when Jev marks them compound. Caveats: no license listed, German and English only, the Grok hand-off needs a companion agent, and the router tests mock Jev.
home-assistant-typesafe pushes the premise into entity binding: Choice and Noul against registered intent schemas, lexical or exhaustive candidate retrieval with domain filtering, then confidence gating and area-aware disambiguation. Low-confidence and multi-action utterances escalate to a fallback agent; cloud routing needs a TypeSafe key and network access. The routing article covers the same split outside the home.
Sensors, actions, and a daily budget
Jev for Home Assistant is the most complete integration here: entity-state questions become sensors whose answer is a probability, one of your options with its distribution, or a number that can land between levels. Automations call four actions — jev.noul, jev.choice, jev.score, jev.ask — and get a response variable back. The budget is the part to copy: it reports calls, input tokens, and estimated daily cost, and halts when the budget is spent — the Assist agent's spoken commands count against the same cap. MIT-licensed but unaffiliated with TypeSafe; it needs Home Assistant 2026.9 or newer, a TypeSafe key, and a custom HACS install. See Choice, Score, or Noul for picking primitives.
The sentinel pattern: advisory by design
Jev Home Assistant Sentinel fences the model: a bounded case goes to Jev, a deterministic policy check runs next, and state verification is recorded separately from the command result, so even a confident recommendation stays advisory. Four modes set where judgment runs — api_only, local_only, api_with_local_fallback, local_with_api_fallback — across Jev on TypeSafe or OpenRouter, Clef and Clef Flash on Cloudflare Workers AI, and open-weight Laya locally, with local_model selecting whichever engine answers the /v1/systemone shape (laya by default). The README is truncated and publishes no benchmarks, so copy the design, not the claims.
Voice: transcribe locally, decide fast
Jev Voice states the pipeline cleanly: mic, energy VAD, whisper.cpp on Metal in about 100 ms, one roughly 250 ms Jev request that selects a typed action plus arguments, then macOS actions and spoken replies. It handles app launches, typing, about 45 keyboard shortcuts, media keys, and compound commands; multi-step tasks port the jev-ultrafast agent loop from the DOM to the macOS Accessibility tree. Before acting, the executor re-reads the target's role, label, value, and enabled state and hit-tests its centre. MIT-licensed; Apple Silicon macOS, whisper-cpp, ffmpeg, uv, a key, and Microphone, Accessibility, and Input Monitoring permissions.
Hey Jev folds the idea into a wake phrase — say "Hey Jev" or hold right Option — where every command is one Jev call the README prices at $0.00004, and only text after the wake phrase reaches TypeSafe. Whisper runs locally via faster-whisper (small.en on first launch); Fish Audio S2.1 Pro speaks replies during a README-claimed free window through November 2026; non-commands go to Claude Haiku via OpenRouter. Dictation is the one feature that sends audio off the Mac, and no license is listed.
voice-browser moves the decision into Chromium: one Jev request per partial transcript, 9 to 11 typed questions deciding intent, target, and completeness in about 250 to 350 ms. Ambiguous targets get numbered overlays — a spoken number picks one with no model call — and destructive clicks need a spoken confirmation. Chrome or Edge for the Web Speech API, Node 20+, a TypeSafe key. The browser-agents article has more.
| Tool | Transcription | Decision per event |
|---|---|---|
| Jev Voice | whisper.cpp on Metal, ~100 ms | one ~250 ms Jev request |
| Hey Jev | faster-whisper, local | one Jev call, $0.00004 claimed |
| voice-browser | Web Speech API | 9–11 typed questions, 250–350 ms |
Narrower jobs: readings, videos, and inboxes
voicevox-jev-proxy fixes Japanese readings before synthesis: Sudachi splits sentences, one Jev request decides word boundaries, accent-phrase splits, readings, and whether www or 笑 signal laughter, and answers below 0.6 confidence (0.8 for phrase height under --intonation) are discarded. Output is corrected WAVs or a VOICEVOX-compatible server on 127.0.0.1:50121. Intonation correction is off by default, dictionaries are not bundled, and the proxy has no authentication.
jevmeter scores every sentence of a video and exports a 16:9 edit with flags and a scoreboard, with debate, earnings-call, podcast, and pitch presets. Its demo debate took 1,191 Jev calls and 5,955 yes/no answers for $0.0497; the presets scored 99% on 200 held-out sentences per the repo's results file — probabilities, not fact-checking. typesafe-jev-workflow is the smallest repeat: a LangGraph node sends a mocked email to Jev, receives a typed invoice-or-general Choice, and branches, stopping on API failure instead of inventing an intent. OpenWhisper, a dictation app in the same listings, documents no Jev integration; the connection is the listing.
What to copy
- One decision call per utterance or command, execution in code, thresholds as visible constants — 0.80, 0.55, 0.40, 0.6.
- A daily token budget every surface shares, voice included.
- The generative agent as fallback, not default.
- Local transcription and a local checkpoint fallback where the network cannot be trusted.
