Question 1

The bench claims 100% on 15 states. Is that a real result or a convenient fixture?

Accepted Answer

Real but narrow: five frozen observations of a real window, three repeats each, with no capture and no actuation, so it measures the decision alone and nothing about whether the click worked. Fifteen states is fifteen states. The README also walks back its own headline, noting the competing model's three misses were one repeated transport failure on the hardest state, so latency is the column that matters most here. You can rerun it against your own fixtures with jev-pilot bench.

Question 2

How is this different from asking a chat model what to click?

Accepted Answer

The model answers one bounded question over ids your code minted and returns a confidence you route on. It cannot write the selector, produce the string to type or decide it succeeded, and postconditions are evaluated in code before and after every action, so its done is a report rather than a proof. The measured difference is speed and the shape of the confidence: hundreds of milliseconds against seconds for the same pick.

Question 3

Can prompt injection make it type something or visit a bad host?

Accepted Answer

It cannot invent content, because it never generates text or URLs. Hosts are declared up front in a SafetyPolicy for the URL you are driving, allowed actions are listed, typing is off by default and dry run exists. What comes back is one id from the candidates your own perception code built, so the residual risk is a wrong pick among your own candidates rather than an invented one.

Question 4

What happens when the model is not confident?

Accepted Answer

Two explicit paths. A low confidence escalates with the pick attached, so your code decides what that means rather than the model. And where the next step is not derivable from the current observation the answer is a stuck sentinel instead of a guess, because the design has no lookahead. The README's own advice is to put a planner above it.

Question 5

How do I install and try it without a key?

Accepted Answer

pip install jev-browser-pilot gives the core, and the [jev] extra adds the TypeSafe provider. jev-pilot selftest proves the loop offline with no browser, network, or credentials, and jev-pilot decide --provider mock returns JSON without a key.

Question 6

What is the loop and who does what?

Accepted Answer

Your code observes the surface, builds the candidate list, executes one action, and verifies the result; the model makes exactly one typed choice per step. It never sees a screenshot, writes a selector, or judges whether it succeeded.

Question 7

What safety controls and exit codes does it offer?

Accepted Answer

A safety policy declares the allowed hosts and actions, typing is off by default, and dry run is available. Exit code 0 means every goal reached its postcondition, 1 means a goal did not, and 2 is a hard error such as no browser or a safety refusal.

Jev Browser Pilot

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev guides

From the Jev community