Testing TypeSafe Jev, Mistral and Gemini for local event validation

Not blind. The study retains cases where a model rejected a listing, so the sample skews toward rejections, and the expected decisions were assistant-written rather than drawn from user behaviour. It is also an events company testing its own use case, not a general model ranking. The page returned an access error when fetched, so all of this is the entry's own summary.

Unclear, and it is the biggest catch. Prompts were tuned separately per provider across 132 cases, including 32 written specifically to probe policy boundaries. That is fair to each model, but it means the headline gap mixes model quality with tuning effort. The entry notes each provider got its own pass; the source does not document how much effort each received.

They come from a separate 21-record timing run, which the entry flags as not a blind test, so treat them as indicative. Even so the gap is wide: a 0.58s median and $0.000904 total for Jev against 3.44s and $0.052421 for Gemini, which works out to roughly $0.043 versus $2.496 per 1,000 decisions at list prices.

On this evidence it is not, though 13 valid events is a small sample to draw a false-positive rate from. Falsely rejecting good listings is the failure mode that hurts most in a moderation gate, and it is where Mistral Small 4 did badly at 5 rejections and Gemini rejected 1. The other two misses out of 50 went the other way, but do not treat zero as a guarantee.

Jev, Mistral Small 4, and Gemini 3.5 Flash-Lite. Jev matched 48 of 50 expected decisions, Gemini 3.5 Flash-Lite matched 43, and Mistral Small 4 matched 42.

At list prices: Jev $0.043, Mistral Small 4 $0.370, and Gemini 3.5 Flash-Lite $2.496. The timing figures come from the study's separate 21-record run.

It matched 42 of 50 expected decisions and rejected 5 of the 13 expected-valid events, the most conservative of the three. Its cost was $0.370 per 1,000 decisions at list prices.