fx auto mode benchmarked with Jev

Not in the public post. It is a screenshot with a one-line summary: roughly 5 to 18x faster and more accurate than gpt-5.6-luna, the previous top choice. No dataset, harness, latency percentiles, or precision and recall table is published, and the replies only ask what the chart means and how much cheaper it is.

The post only labels it a safety classifier that screens tool calls in fx auto mode. It does not give the label set, the cost of a false negative, or whether a flagged call is blocked or escalated, so you cannot use it to argue that Jev is safe enough for your own gate.

You cannot tell from the post. It does not state whether the baseline was evaluated with logit access, whether prompts matched, or whether accuracy was measured on the same examples. The replies questioning the setup went unanswered in the thread.

Not on this evidence. It is one engineer's summary of an internal benchmark, published as a screenshot with no code and no dataset. Take the direction as encouraging and the magnitude as unverified until someone releases the evaluation harness.

The post gives a spread rather than one figure, which normally means the gain varied across slices of the evaluation, but it publishes no per-slice breakdown, no latency percentiles and no accuracy table to show where the low and high ends come from. The range is unexplained variance in an otherwise bare summary.

gpt-5.6-luna, described as the company's current top choice for the fx auto-mode safety classifier. The comparison is therefore against the incumbent in this exact role rather than an arbitrary model, though the post does not say how long it held that position or on what prior evidence it was chosen.

Only how fast it decides, as far as the post says. It describes a classifier swap for the safety reviewer, claiming better speed and accuracy, and says nothing about changing what auto mode allows, blocks or escalates. The intended reading is the same gate running on a different model, not a new safety policy.