Jev in Search: Three Practical Evaluations

On that specific job, yes. Across 100 multi-hop questions both Jev and DeepSeek V4 Flash reached 93.25 percent Recall@5 with nearly identical search rounds, while decision latency fell from 2.23 s to 0.55 s and estimated decision cost dropped to roughly a seventh. The author's own caveat: it is a binary decision and it did not stretch Jev far.

No. On 2,172 MemSearch questions Jev lifted Recall@5 from 74.71 to 79.41 percent, but Voyage rerank-3 reached 81.87 percent and was better at surfacing the correct result, with the gap appearing in both languages. Jev had no cost advantage there either. The author suspects domain shift and implicit multi-step reasoning.

Not fully, and the author flags it: baselines reuse published 1,000-question results while Jev ran 500-question samples, so these are unpaired comparisons rather than same-questions tests. He also works on the projects being evaluated, which is worth keeping in mind when reading the wins.

The cheap win is high-frequency, low-stakes gating such as should-I-keep-searching, where equal recall at a quarter of the latency compounds over many calls. Relationship reranking and multi-hop graph paths are where dedicated rerankers and stronger models still win, so those need evaluating against your own data.

On Vector Graph RAG, Jev beat GPT-4o-mini but trailed GPT-5-mini by about 1 point on MuSiQue and 4.13 on HotpotQA. The author reads this as useful relationship reranking but a reasoning gap, since some paths need implicit multi-step reasoning Jev does not yet handle as well, and names domain shift and those implicit steps as the suspected causes.

About $3.15 per 1,000 questions for Jev against $6 to $9 for GPT-5-mini in the recalculated cost figures. In the DeepSearcher test the decision cost estimate fell to roughly one-seventh of the DeepSeek V4 Flash service it replaced while matching its recall and average search rounds. Treat these as the write-up's estimates, not billed totals.

Cheney Zhang, whose team works on the evaluated projects: DeepSearcher, MemSearch and Vector Graph RAG are their open-source work, and the post says so. He flags the unpaired sample sizes himself and frames the domain-shift explanation as a hypothesis, since Jev's training data is not known. The page also notes the animations are illustrative replays, not live API calls.