Do I need a Mac to use it?

For the accessibility-tree computer use, yes: macOS 13 or newer with Accessibility permission, plus Screen Recording permission for screenshots. Releases also ship Linux and Windows binaries with the C-ABI library.

How is this different from screenshot-based computer use?

It reads the operating system's accessibility tree and acts on stable element references instead of guessing from pixels, so retries act on the same element. Skeleton traversal cuts tokens 78-96% on dense apps like Slack.

What does Jev actually decide here?

Through the jev-desktop skill, Jev picks the UI operation and its target from accessibility-tree candidates. The runtime handles observation, guarded execution, and re-verification around that decision.

Can I call it from Python without subprocess overhead?

Yes. Every release ships a C-ABI shared library, so you can dlopen it and call in-process instead of fork-exec per command. The README shows the ctypes pattern including an ABI version check.

What does agent-desktop use instead of pixels to see the UI?

OS accessibility trees; it exposes refs that stay stable and actions that are safe to retry, rather than guessing from screenshots.

How much do snapshots cost in tokens, and how is that reduced?

A regular Slack snapshot was 30,743 tokens versus 383 for a skeleton overview; progressive skeleton traversal gives 78-96% token reduction on dense apps via a shallow overview plus targeted drill-down.

What language bindings does the FFI library offer?

A prebuilt C-ABI cdylib ships for macOS, Linux and Windows so it can be dlopen'd from Python, Swift, Go, Ruby, Node or C for in-process calls instead of fork-exec per command.

CommunityJev Demos & Experiments

Agent Desktop

Rust CLI that gives agents computer use through OS accessibility trees instead of screenshots: snapshot a live UI, act on stable element refs, then re-verify. Ships a jev-desktop skill where Jev picks the operation and target from accessibility-tree candidates.

See highlights

Open github.com

Stars: 1.8k
Language: Rust
License: APACHE-2.0
Published by: Community
Author: lahfir
Last verified: Oct 2026

lahfir/agent-desktop

User repository on GitHub

View on GitHub

Agent Desktop gives any agent reliable computer use on the desktop. Built with Rust, it sees any app's real UI structure through OS accessibility trees and operates it — refs stay stable and actions stay safe to retry, instead of guessing from pixels.

Forks: 127
Last pushed: Updated Oct 2026

Repo stats from the GitHub API, cached Oct 2026.

About this project

agent-desktop gives any agent reliable computer use on the desktop. Built with Rust, it sees any app's real UI structure through OS accessibility trees and operates it — refs stay stable and actions stay safe to retry, instead of guessing from pixels. Native Rust CLI: Fast, single binary, no runtime dependencies. C-ABI cdylib (libagentdesktopffi): Load once from Python / Swift / Go / Ruby / Node / C instead of forking the CLI per call.

Quoted from the project's README. Read the full README.

Tagscommunity jevbest

Works forJev for Classification Jev for Tool calling

Used inJev for Software & DevTools

Languages: Rust

Highlights

Reads live UI structure from accessibility trees and acts by stable element refs, so retries act instead of re-guessing from pixels.
Progressive skeleton traversal cuts tokens 78-96% on dense apps: a shallow overview first, then drill into regions of interest.
Ships a jev-desktop skill plus 54 operational commands with structured JSON output, error codes, and recovery hints.
Installs with npm as a single binary; every release also ships a C-ABI library for in-process calls without forking the CLI.

Watch out

The accessibility path needs macOS 13+ with Accessibility permission; Chromium interop goes through a verified CDP port. Copy reviewed against the repo README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

More Jev resources like this

celesto

Sandboxed virtual machines for agents: each sandbox is a microVM that starts in about 500 ms, keeps files and state between sessions, and runs locally or in Celesto Cloud behind a CLI and Python SDK.

Jev Demos & Experiments#community#jevbest

Community

CelestoAI/celestoPython

webctl

156

A CLI search wrapper that fans out to up to three providers, dedupes the merged results, has Jev score them for relevance against a stated goal, and optionally scrapes and chunk-filters pages so an agent reads less. Saves context, at the cost of a Jev call per batch.

Jev Demos & Experiments#community#jevbest

Community

dorkitude/webctlGo

jimothy

A Node package that distils Jev-style typed answers into small local classifiers, trained either from saved labels or from a Jev teacher, then served in Node or a browser worker.

Jev Demos & Experiments#community#jevbest

Community

AndrewPrifer/jimothyTypeScript

jevk5

147

Apache-2.0 open-weight stand-in for a hosted decision model: Qwen3.5-4B with a distilled adapter scores noul, choice and score questions through a systemone-shaped server, with no generated tokens.

Jev Demos & Experiments#community#jevbest

Community

allebee/jevk5Python

ghosthands

macOS agent loop that moves a real USB HID mouse and keyboard under a vision model, with an experimental decision-model lane picking the next operation and screen target from an element table.

Jev Demos & Experiments#community#jevbest

Community

affirmitv/ghosthandsPython

jev-browser-skill

Standalone CLI skill that drives isolated Playwright Chromium with Jev choosing scoped in-page actions, defaulting to Vercel AI Gateway with TypeSafe as an option.

Jev Demos & Experiments#community#jevbest#catalog-import

Community

hqman/jev-browser-skillTypeScript

From the Jev guides

Original write-ups on what this entry is for.

Per-Tick Decisions in Games

Sep 28, 2026

A game tick is a state plus a legal action list. How Clash Royale, Minecraft, and the earlier Mario, Pokémon, StarCraft, and drone loops ask Jev one step at a time.

Jev Demos & Experiments#gaming#agents#control

Read article

Back to all resources

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai