Why drive the accessibility tree instead of screenshots?

Speed and stability. Screenshot guessing is slow, fragile, and steals the mouse; here each element carries a stable ref, role, label and geometry, diff-only re-observation keeps traffic small, and every action separates action_sent from verified with retry, reobserve, or never guidance.

How painful is setup across different agent clients?

One command: pipx install then macos-cu setup for your client, with idempotent JSON merging and a .bak backup. Claude Code can also install as a plugin, and --dry-run plus --read-only modes let you preview safely.

What stops it from typing my passwords or locking the machine?

A built-in deny list refuses password managers, Keychain, login window and system auth prompts, secure-field values are redacted, lock and log-out chords are blocked, typed text is audit-logged as length only, and everything runs locally except the optional Jev guard.

Where does Jev actually fit in this kit?

An optional pre-step check before irreversible actions, configured only when you supply a key and call macos_jev_guard. Screenshots with numbered marks and on-device Apple Vision OCR are also fallbacks reserved for canvas and custom-drawn UI.

What do I need to grant before it will work?

macOS 12 or later and Python 3.10 or later. Grant Accessibility (for reading the tree and posting input) and Screen Recording (for screenshots and OCR, without which every frame is black) to the app that runs your agent. macos-cu doctor tells you exactly which app is missing which permission.

How does input happen without hijacking my cursor?

Clicks, keys, scrolls, drags and Unicode typing are posted to the target process, so you can keep working, CJK, emoji and accents arrive intact, and the clipboard is left alone. Every result separates action_sent from verified and says whether it is safe to retry, whether to reobserve, or whether to never retry.

Does it log anything or send data anywhere?

Everything runs locally and the kit makes no network calls, except the optional Jev guard, which only runs when you configure a key and call it. The audit log is one JSON line per mutating action with typed text logged as its length only, and dry-run mode resolves targets but posts no events.

CommunityJev Demos & Experiments

Macos Computer Use Kit

macOS computer-use kit exposing an 18-tool stdio MCP server plus CLI that drives apps through the accessibility tree, posting input in the background without moving the user cursor.

See highlights

Open github.com

Stars: 5
Language: Python
License: MIT
Published by: Community
Author: Sur-Cai
Last verified: Oct 2026

Sur-Cai/macos-computer-use-kit

User repository on GitHub

View on GitHub

AX-first computer use for AI agents on macOS with optional Jev (TypeSafe System One) semantic guards: calibrated target/input judgments before an irreversible action, decisions kept in code. Accessibility-tree targeting, window-scoped input, clipboard-safe paste, read-back verification. Ships a pip CLI, a pi package and a DeepSeek Harness plugin.

Forks: 1
Last pushed: Updated Oct 2026

Repo stats from the GitHub API, cached Oct 2026.Homepage

About this project

AX-first macOS computer use for AI agents: an MCP server and a CLI. Agents read the accessibility tree instead of guessing coordinates from screenshots. Input is posted to the target app in the background, so your cursor never moves, and every action is verified. One command per client. Works with Claude Code, Claude Desktop, Codex, Cursor, Gemini CLI, opencode, pi and DeepSeek Harness. It is a stdio MCP server with no runtime dependencies beyond pyobjc and Pillow.

Quoted from the project's README. Read the full README.

Tagscommunity awesomejev catalog-import

Works forJev for Classification Jev for Moderation Jev for Tool calling

Used inJev for Software & DevTools

Languages: Python

Highlights

Background clicks, typing, scrolls and drags go to the target process, leaving clipboard and cursor alone.
Unicode-safe typing keeps CJK, emoji and accents intact where clipboard paste would corrupt them.
Verification-first results distinguish sent from verified instead of assuming success.
macos-cu doctor reports permissions, display layout with Retina scale, OCR and policy in one place.

Watch out

Requires macOS 12 or later with Accessibility and Screen Recording granted to the agent host app, and Electron trees need extra handling the kit enables automatically. Copy reviewed against the repository README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

More Jev resources like this

jev-kit

Claude Code kit that guards PreToolUse calls with a free pre-filter plus one Jev question for ambiguous cases, and adds sub-agent sizing, rechecks, search, daemon speedups, and monitoring.

Jev Demos & Experiments#community#awesomejev#catalog-import

Community

jonathanavis96/jev-kitPython

Footwork

A computer-use agent for the browser and for native macOS apps that puts a Jev guard in front of the driver: claimed completions are checked against observed state and the trajectory, irreversible actions pass a gate, and Jev can take the mechanical steps itself.

Jev Demos & Experiments#community#awesomejev#catalog-import

Community

Tom-R-Main/FootworkPython

robodiag-harness

Terminal harness that checks ROS 2 robots with deterministic health tests, capability descriptions, SQLite history, and optional Jev evidence routing.

Jev Demos & Experiments#community#jevbest#catalog-import

Community

YueBit/robodiag-harnessPython

jev-ultrafast-mcp

MCP server where one browser_goal call hands a URL, a goal, and code-checked assertions to a server-side loop driven by a decision model over live page elements.

Jev Demos & Experiments#community#jevbest#catalog-import

Community

jiawei686/jev-ultrafast-mcpPython

jev-style

Local Qwen3.5 decision models in 0.8B and 2B sizes with fitted temperatures, systemone-shaped server, playground, guard, MCP tools, and eval harness.

Jev Demos & Experiments#community#jevbest#catalog-import

Community

lawrence3699/jev-stylePython

aside-jev

Browser workflow that keeps one Aside REPL session open, compacts page observations into allowed candidates, and has Jev pick the next action with confidence checks.

Jev Demos & Experiments#community#awesomejev#catalog-import

Community

himomohi/aside-jevPython

From the Jev guides

Original write-ups on what this entry is for.

Per-Tick Decisions in Games

Sep 28, 2026

A game tick is a state plus a legal action list. How Clash Royale, Minecraft, and the earlier Mario, Pokémon, StarCraft, and drone loops ask Jev one step at a time.

Jev Demos & Experiments#gaming#agents#control

Read article

Back to all resources

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai