User repository on GitHub
Siri-style push-to-talk voice orb for autonomous browser control with Jev and Metal Whisper
- Forks
- 0
- Last pushed
- Updated Sep 2026
Repo stats from the GitHub API, cached Oct 2026.
CommunityJev Demos & Experiments
Jev Orb
A native macOS push-to-talk voice orb that runs a direct Jev choice loop over the Accessibility tree: pure SwiftUI and AppKit, local Metal Whisper transcription, and Accessibility plus microphone permissions held by JevOrb.app instead of your terminal.
- Stars
- 0
- Language
- Makefile
- Published by
- Community
- Author
- bottlebrushes
- Last verified
- Oct 2026
About this project
A lightweight, native macOS push-to-talk floating voice orb that interprets arbitrary spoken commands at runtime and operates applications through macOS Accessibility (AX), using Jev and local Metal Whisper speech-to-text. Built with pure SwiftUI and AppKit, isolating all OS-level permissions into a standalone application (JevOrb.app). Push-to-Talk Siri Orb: Powered by a pure SwiftUI port of metasidd/Orb. Fluid wavy blobs, rotating depth glows, and floating particles.
Quoted from the project's README. Read the full README.
Works forJev for Classification
Used inJev for Software & DevTools
Highlights
- The dispatcher runs a direct TypeSafe Jev choice loop over the available semantic operations, executes one choice, then re-observes the AX state.
- AX-only control of roles, labels, values, relationships and supported actions - no screenshots, OCR, image matching, coordinates or CDP.
- Accessibility and microphone permissions belong to JevOrb.app, not the invoking terminal, and Screen Recording is never requested.
- Transcription prefers the local inference endpoint and falls back to the Metal-accelerated whisper-cli with the local model.
Watch out
The present loop supports application activation and launch, AX press, focus and scroll, and literal text quoted in the command. It cannot generate new prose, choose arbitrary keyboard chords or unquoted values, prove multi-part goals comprehensively, or handle clarification and high-stakes external commits. Missing Accessibility permission, unavailable windows, inaccessible content, ambiguous or stale targets and unsupported actions are blockers rather than reasons to guess, and there is no screenshot, OCR, image-matching or coordinate fallback. Login, MFA, consent, permission and anti-automation barriers are not bypassed, and Cancel cannot undo an action already sent to macOS. The build/JevOrb.app files tracked in the repository are an older binary, not a release of this implementation, so build from source. Copy reviewed against the repository README on 2026-10-04, not from running it.
Questions about this entry
Short answers drawn from the linked project or page.
No. Control is AX-only: accessible roles, labels, values, relationships and supported actions, with no screenshots, OCR, YOLO, image matching or screen coordinates, and no CDP, browser extensions or browser automation harness. Screen Recording is not required, and VoiceOver does not need to be enabled. Accessibility and microphone permissions are granted to JevOrb.app itself, not to the terminal that launches it.
It stops and reports a blocker code. Missing Accessibility permission, unavailable windows, inaccessible content, ambiguous or stale targets and unsupported actions are blockers, not reasons to guess a target. Pixel-only canvases and unlabeled controls without enough semantic context cannot be operated reliably, there is no visual fallback, and login, MFA, consent and permission barriers are not bypassed either.
The current loop covers application activation and launch, AX press, focus and scroll, and literal text quoted in the command. It cannot generate new prose, choose arbitrary keyboard chords or unquoted values, prove multi-part goals comprehensively, or handle clarification and high-stakes external commits. A completion report confirms recorded local effects rather than every interpretation of a broad command, and a research request does not authorize a purchase, booking or message.
Build it from source: ./scripts/build_app.sh installs JevOrb.app into ~/Applications. The build/JevOrb.app files tracked in the repository are an older binary and not a release of this implementation. Set TYPESAFE_API_KEY in the app environment or in ~/.config/jev-orb/credentials.env at mode 0600, because the direct Jev choice service is required for navigation and the former OpenRouter planner is no longer used. There is also a --command entry point that writes a report to /tmp/jevorb-last-report.json.
JevDispatcher runs a direct TypeSafe Jev choice loop over the available semantic operations, executes one choice, then re-observes the resulting AX state and repeats. Speech is transcribed locally, first through the configured inference endpoint and falling back to the Metal-accelerated whisper-cli.
From the signed app bundle, open -a "$HOME/Applications/JevOrb.app" --args --command 'Open Safari' runs a typed command and writes a structured report to /tmp/jevorb-last-report.json. Run it only in the logged-in GUI session with Accessibility permission for JevOrb.
Cancel stops the current typed task before its next action, but an action already sent to macOS cannot be undone. A completion report confirms the recorded local effects, not every possible interpretation of a broad command.
More Jev resources like this
45GitHub stars
Drop-in OpenAI-compatible cache proxy for repeatable agent workflows: exact hits stay local, and with a TypeSafe key Jev judges whether a cached response is semantically reusable for the current request.
Jev Demos & Experiments#community#jevbest#catalog-import
46GitHub stars
Open 150M encoder that reproduces the typed noul, choice, and score interface in one forward pass, trainable on a free GPU and runnable locally for nothing.
Jev Demos & Experiments#community#jevbest#catalog-import
941GitHub stars
Library that adds schema-guaranteed typed outputs to existing autoregressive LLMs without changing weights, served through SGLang or a hosted API.
Jev Demos & Experiments#community#jevbest#catalog-import
Community
TypeLLM/TypeLLMPython
TypeLLM/TypeLLMPython45GitHub stars
Drop-in service that exposes Upstage Solar models through the System One shape, accepting state plus noul, choice, or score questions.
Jev Demos & Experiments#community#jevbest#catalog-import
Community
hunkim/solar-mini4-jevPython
hunkim/solar-mini4-jevPython98GitHub stars
Self-hosted Docker service and web UI that packages Laya structured-decision inference behind a compatible System One endpoint.
Jev Demos & Experiments#community#jevbest#catalog-import
Community
1Panel-dev/laya-serverPython
1Panel-dev/laya-serverPython38GitHub stars
Research recipe that fuses two frozen open readers into Jev-style yes/no judgments with a fit-free router, confidence gate, and candidate sets.
Jev Demos & Experiments#community#jevbest#catalog-import
Community
FeiLiuEM/open-medical-jevPython
FeiLiuEM/open-medical-jevPythonFrom the Jev guides
Original write-ups on what this entry is for.
A game tick is a state plus a legal action list. How Clash Royale, Minecraft, and the earlier Mario, Pokémon, StarCraft, and drone loops ask Jev one step at a time.
Jev Demos & Experiments#gaming#agents#control
Read article
From the Jev community
Posts from builders shipping with Jev right now.