TypeSafe Computer Use

Not quite. The screen is read deterministically with macOS Vision OCR on a crop of the frontmost window plus labelled accessibility-tree controls, one TypeSafe request answers up to three Choices, and a writing model is called only when a text field needs free text or the screen must be read. The README's own caveat is that everything a frontier model does for free must be rebuilt here as code.

They are the author's own measurement: one decision each on the same screenshot and goal, with roughly equal input tokens, comparing the classifier against a bare screenshot call to Claude Opus 5. The honest caveat printed next to the table is that the big model read event dates off the pixels and compared them unaided, while the classifier needed extra date-parsing code to match.

Only with --act. Without it, a dry run prints the single step it would take; with it, the loop drives the real mouse and keyboard for up to 100 steps. The project is marked beta with settings and behavior expected to change between 0.x releases. Without Accessibility permission synthetic clicks are silently dropped and --act refuses to start; without Screen Recording you capture wallpaper.

No. The OSWorld setup needs a Linux host with KVM, and the docs also point at a one-command cloud machine for it. On macOS the agent itself runs, and a browser backend offers DOM perception over Chrome DevTools with no OCR and no screen permission if you would rather not grant those.

macOS 14 or newer, Python 3.12 or newer and uv are the base install. TYPESAFE_API_KEY is required for every decision; ANTHROPIC_API_KEY is optional and only used for type_text, writer-proposed URLs and the final answer. CLICKER_EMAIL enables type_email and CLICKER_BROWSER defaults to Google Chrome. Your terminal also needs Screen Recording and Accessibility permission in System Settings.

Yes. The writer is only called when a text field needs free text or the screen must be read, and the docs describe running it on another model or endpoint such as LM Studio, Ollama or any OpenAI-compatible server. The classifier still uses TypeSafe's model through TYPESAFE_API_KEY, so that key stays required regardless of which writer you choose.

The loop stops on done, confidence below 0.4, a stall, the step limit of 100 by default, Ctrl-C, or slamming the mouse into the top-left corner. Every run writes runs/<timestamp>/, which the docs say can be replayed offline, so a stall is something you can inspect rather than re-run blind.