Question 1

Are the Flights and Wikipedia speed claims actually checked?

Accepted Answer

The digest reports a Flights run in 5.63s with 10 model calls and 15,071 input tokens versus 7.09s and 17 calls for the Jev-based baseline, plus a Wikipedia run in about 1.3s with 2 calls. Both were code-checked on the final page and traces are kept under bench/final.

Question 2

Am I locked into one model provider to use it?

Accepted Answer

No. OpenRouter is the default and any OpenAI-compatible endpoint works, and the comparison table includes Groq, Cerebras, DeepInfra and Gemini paths with different time, reliability and cost tradeoffs. You still need Python 3.12 or newer, uv, Chrome and one API key.

Question 3

How does it avoid wasting calls on fixed sleeps?

Accepted Answer

It waits for the DOM to go quiet and for requests the action started to finish, lists below-the-fold elements instead of adding a scroll decision, and explains bad answers back to the model for one retry. The Wikipedia run is described as costing a fraction of a cent.

Question 4

What stops the model from clicking the wrong element?

Accepted Answer

The model only names an element number and the page resolves and rechecks visibility, state, geometry and overlays before input, with DONE accepted only on a settled page. Password, file and hidden inputs are never listed and the browser uses a throwaway profile unless you attach your own.

Question 5

Which models actually work, and which ones fail?

Accepted Answer

Seven of eight finish the Wikipedia task in one typed action, and three pass every hold-out task. The nine-step Flights form is what separates them: llama-3.3-70b on Groq passes every run, gpt-oss-120b on Cerebras passes some of the time, and smaller models declare victory inside the calendar. gemini-2.5-flash, gemini-2.5-flash-lite and qwen3-next-80b stop too early on Flights, deepseek-v3.1 takes about 3 s per call, and gpt-4.1-nano loops.

Question 6

Can I compare this against Jev on my own machine?

Accepted Answer

Yes, with --backend typesafe and TYPESAFE_API_KEY, which runs the Jev arrangement of one operation head plus target heads plus a text helper so the head-to-head happens on one machine. The README is straight that it is untested there because no TypeSafe key was available when it was written, so treat the switch as available for comparison rather than as a supported path.

Question 7

What are the hard limits, and what platforms are covered?

Accepted Answer

The DOM reader covers common HTML and ARIA controls, not the full accessible name algorithm. Closed shadow roots, cross-origin frames, canvas, uploads and new tabs are not handled, and guards accept unrelated changes elsewhere on the page by design. It was tuned while watching llama-3.3-70b on the two reference tasks, so other models were measured rather than tuned for. Linux is covered by CI and Windows is untested.

Dejevu

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev guides

From the Jev community