Vercel fx Safety Reviewer on Jev

Vercel, on its own safety classifier. The figure appears first as a CEO claim, then a Vercel engineer replies that the benchmark showed roughly 5 to 18x faster and more accurate than gpt-f-luna, the previous top choice. Neither post publishes the dataset, the labels, or how accuracy was scored.

A single command. fx auto mode runs a safety reviewer over each command, which is a bounded yes-or-no decision on one string rather than a judgement about what the agent is trying to accomplish. Neither post describes multi-step or trajectory-level reviewing, and no limit on command length is given.

Undocumented in both posts. They cover the gateway model id typesafe-ai/jev with a state plus questions call, and that Jev is likely the new default, but nothing explains the low-confidence path or the fallback when the reviewer call itself fails, which is the failure mode a gate has to define.

Yes, through the gateway now. Vercel's changelog shows an await evaluate call with model typesafe-ai/jev, a state block, and questions. That gives you the same classifier on your own commands, though reproducing the accuracy claim needs a labelled command set that has not been published.

Guillermo Rauch, Vercel's chief executive, quote-posting a benchmark from a Vercel engineer and adding that Jev is likely to become the new default reviewer. So both the announcement and the claim come from the company that ships fx, not an outside tester.

It is a tail-latency figure: 18 times faster at the 95th percentile, the speed of the slowest five percent of calls. That is more meaningful than an average for a reviewer that runs on every command, because the worst cases are what stall an agent, but no distribution or percentile table is published.

Its developers account says you can build agents that decide, route, score, and stop in milliseconds, with the evaluate call shown as the interface. That is the same classifier decision shape as the safety reviewer, generalized to agent control, and it is the clearest statement of how Vercel expects people to use Jev on the gateway.