Does this change DeepSearcher's default retrieval agent?

No. The Jev stopping policy lives only in evaluation/jev_stopping. Installing or running DeepSearcher does not call Jev or change ChainOfRAG; a production integration would need a configurable stopping policy with explicit timeout and failure behavior in the agent.

What did the stopping-policy comparison find?

On 100 paired queries both policies reached the same 93.25% Recall@5 at nearly identical depth (2.96 vs 3.00 rounds). Jev's median decision response was 0.55s vs 2.23s and its estimated decision cost was lower, but the repo states Jev did not reduce search depth.

What is the setup behind those numbers?

A paired replay over 100 sampled 2WikiMultiHopQA queries with a frozen manifest, seed 20260922 and jev-1.13.0. Decisions agreed on 92.57% of the 700 shared states, and the estimated decision cost was $0.0059 per 100 queries against $0.0446 for the DeepSeek reflection call it replaced.

What license is the project under, and what does it normally do?

Apache-2.0. DeepSearcher combines LLMs such as OpenAI o3, Qwen3, DeepSeek, Grok 4, Claude 4 Sonnet, Llama 4 and QwQ with vector databases such as Milvus and Zilliz Cloud to search, evaluate and reason over private data, aimed at enterprise knowledge management, internal Q&A and information retrieval.

How do I install DeepSearcher itself?

A Python 3.10 virtualenv with `pip install deepsearcher`, or a clone plus `uv sync`, which the project recommends for speed. Optional extras come as packages such as `pip install "deepsearcher[ollama]"`, and providers are configured in code with set_provider_config for the LLM and the embedding model.

How do I load my own data into DeepSearcher?

Load local files with load_from_local_files, or crawl the web with load_from_website, which requires a FIRECRAWL_API_KEY environment variable. Web crawling is under development, and both loading paths run before query() using the configured embedding model.

Who maintains DeepSearcher?

Zilliz, the company behind Milvus, maintains it under Apache-2.0. The repository has about 8,300 stars and runs a Discord for support. The Jev stopping experiment is a separate evaluation harness inside that repo rather than a maintained product feature.

CommunityJev Benchmarks & Evaluations

DeepSearcher Jev stopping experiment

Agentic search evaluation inside Zilliz DeepSearcher: a standalone harness replaces the ChainOfRAG reflection call with one Jev Noul per round to decide whether to stop within a seven-round budget, then compares stopping behavior, evidence recall and decision cost on 100 sampled 2WikiMultiHopQA queries.

See highlights

Open github.com zilliztech/deep-searcher

Stars: 8.3k
Language: Python
License: APACHE-2.0
Published by: Community
Last verified: Oct 2026

zilliztech/deep-searcher

Organization repository on GitHub

View on GitHub

Open Source Deep Research Alternative to Reason and Search on Private Data. Written in Python.

Forks: 811
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.Homepage

About this project

DeepSearcher combines cutting-edge LLMs (OpenAI o3, Qwen3, DeepSeek, Grok 4, Claude 4 Sonnet, Llama 4, QwQ, etc.) and Vector Databases (Milvus, Zilliz Cloud etc.) to perform search, evaluation, and reasoning based on private data, providing highly accurate answer and comprehensive report. This project is suitable for enterprise knowledge management, intelligent Q&A systems, and information retrieval scenarios.

Quoted from the project's README. Read the full README.

Tagscommunity rag evaluation deep-research

Works forJev for Evaluation Jev for Decision gating

Used inJev for Data & Analytics

Languages: Python
Platforms: API
Capabilities: Open sourceRequires API key

Highlights

Paired replay on 100 sampled 2WikiMultiHopQA queries with frozen manifest, seed 20260922 and jev-1.13.0.
Both policies score 93.25% Recall@5 at 2.96 vs 3.00 mean rounds; decisions agree on 92.57% of 700 shared states.
Median stop-decision response 2.23s for DeepSeek vs 0.55s for Jev; estimated cost $0.0446 vs $0.0059 per 100 queries.
Installing DeepSearcher does not enable Jev; the integration lives only in evaluation/jev_stopping.

Watch out

Apache-2.0 licensed. Jev is called only by the evaluation runner, not DeepSearcher's default ChainOfRAG agent, and the experiment is explicitly negative on search depth: Jev did not reduce mean rounds in this sample.

Questions about this entry

Short answers drawn from the linked project or page.

Related terms

Glossary definitions related to this entry.

Precision and recall

More Jev resources like this

hev reranker

A BEIR reranking recipe that asks up to 30 parallel Noul relevance questions in one Jev state, with RESULTS.md against cross-encoders and hosted rerankers.

Jev Benchmarks & Evaluations#community#evaluation#python

Community

hev/rerankerPython

jev vs. open alternatives

Notebook benchmark that compares Jev with open models and specialised tools on five document-pipeline chores: language detection (Choice(12)), page orientation (Choice(4)), RVL-CDIP classification (Choice(16)), bundle splitting and parse-tier triage, with liteparse doing all PDF rendering and OCR work.

Jev Benchmarks & Evaluations#community#evaluation#document-processing

Community

run-llama/jev_vs_ossJupyter Notebook

LangWatch

4.9k

LangWatch Instant Evals ships a TypeSafe Jev classifier that batches Noul, Score, and Choice judgments on production traces in one POST per text.

Jev Benchmarks & Evaluations#community#typescript#evaluation

Community

langwatch/langwatchTypeScript

jev-oncall

On-call tooling where Jev answers four typed questions per alert - actionable, severity, owning team and duplicate-of - and plain code routes, pages or drops. Includes a root-cause benchmark where Jev and code investigate recorded incidents with no general-purpose LLM in the loop, enforcing budgets and verification.

Jev Integrations & Adapters#community#alerting#incident-response

Community

mingleiw/jev-oncallHTML

jev-medhallu-benchmark

Pre-registered benchmark adding Jev 1.13 to Stanford MedHELM's MedHallu results: one Noul asks whether a proposed answer misrepresents its PubMed abstract. Jev scored 92.9% on 1,000 test items at a 204 ms median and USD 0.03 per 1,000 checks through OpenRouter.

Jev Benchmarks & Evaluations#community#medical#hallucination

Community

stperic/jev-medhallu-benchmarkPython

DecisionBench

An open benchmark runtime for document-grounded decision models: it scores Jev Noul, Choice and Score answers on pinned tasks and counts malformed probability distributions as misses so model comparisons stay reproducible.

Jev Benchmarks & Evaluations#community#benchmark#evaluation

Community

Hanno-Labs/decision-benchPython

Back to all resources

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai