Article
CommunityJev Practices & Patterns
Jev handed our LLM the wrong page a tenth as often as our embeddings
Sayora's AI receptionist replaced production RAG with Jev for page picking and measured the change on 168 caller questions: wrong pages handed to the LLM fell from 33% to 3%, and the right page ranked first 88% of the time against 70%.
- Format
- Article
- Reading time
- 2 min
- Published by
- Community
- Author
- Jacob Edgar-Anderson
- Last verified
- Oct 2026
Highlights
- On 168 caller questions across four websites, Jev handed the LLM a wrong page 3% of the time against the embeddings' 33%.
- Jev put the right page first 88% of the time versus 70%, and returned none when the site had no answer.
- Production baseline: 1,200-character chunks with 200-character overlap, embedded with text-embedding-3-large at 768 dimensions.
- Jev got whole pages with ~100-token descriptions written by Claude Haiku 4.5, one choice question plus a none option.
- Sites ran from 22 to 232 pages; Claude Opus 5.5 wrote 168 caller-style questions, 40 of them unanswerable.
Watch out
Grounded in the sayora.ai post captured on 2026-10-15; the article is truncated mid-results, so the pooled-judging detail and the other pickers' final tables were not captured, and the comparison is one company's test on its own receptionist rather than an independent benchmark.
Questions about this entry
Short answers drawn from the linked project or page.
Its AI receptionist's retrieval: production embeddings on 1,200-character chunks against Jev picking whole pages from a described list, run over 168 caller questions across four small-business websites.
Search handed the LLM a wrong page on a third of questions (33%) while Jev did so 3% of the time; Jev also put the right page first 88% of the time against 70%, and knew when no page held the answer.
Each page keeps a short description—title, path, a 30-word factual summary and likely caller questions—and Jev answers one multiple-choice question over up to 255 options, with none-of-these offered as an option too.
The post cites Jev's pricing of $0.042 per million input tokens with output free, which works out to about a twentieth of a cent per lookup on a 100-page site.
Chunks of 1,200 characters with 200 characters of overlap, embedded with OpenAI's text-embedding-3-large at 768 dimensions, injecting the closest chunks scoring above 0.36 cosine similarity on every caller turn.
Nine pickers ran every question: Jev with full and thinner descriptions, embeddings then Jev, GPT-6 Luna, embeddings on the page descriptions, production chunk embeddings, and BM25 on descriptions and on page text.
No. It is Sayora's own test on its own receptionist with pooled judging, published as a company blog post; the method is described in detail, but the questions, websites and grading all come from one vendor.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: A RAG loop where Jev decides what is real
X: Measuring what Jev does in a RAG pipeline
X: Jev-powered reranking for RAG. Likitd_’s jev-ranker uses Jev to rerank results in a RAG pipeline.
X: Coverage is not product value
More Jev resources like this
Tenuo's engineering write-up of safe-upgrade, a dependency-upgrade agent built on Jev, LangGraph and Tenuo: Jev turns repository evidence into typed probabilistic choices, the graph routes on confidence, and each action gets a short-lived task-scoped warrant.
Jev Practices & PatternsArticle#community#catalog-import#yibie
Community
Framework-agnostic guard layer that puts Jev in front of an agent's tool calls: a Choice risk tier, a Noul destructiveness check and a Score blast radius decide allow, deny or ask.
Jev Practices & Patterns#community#awesomejev#catalog-import
Community
lgy1027/jevshieldPython
lgy1027/jevshieldPythonA DSH plugin, published to npm, that registers one agent tool, jev_decide, so a coding agent can ask TypeSafe's Jev a typed question and get a calibrated probability instead of guessing from intuition.
Jev Practices & Patterns#community#awesomejev#catalog-import
Community
nanami-0713/dsh-jev-decideJavaScript
nanami-0713/dsh-jev-decideJavaScriptAn open lab for using Jev correctly next to Claude Code: a patched jev-router with per-turn model routing, a health-gated fallback with alerts, and write-ups of two silent bugs and of where tokens actually go.
Jev Practices & Patterns#community#catalog-import#logicrw
Community
pasblinn/jev-labTypeScript
pasblinn/jev-labTypeScriptRunnable benchmark putting Jev, jev-preview and Claude Sonnet 5 through one real agent task — classifying a tool call as readonly, destructive, privileged or exfiltration — on the same 60 labelled cases, with raw per-call results committed.
Jev Practices & Patterns#community#awesomejev#catalog-import
Community
themsquared/jev-benchmarkPython
themsquared/jev-benchmarkPythonpydecide is one Python client for typed System One decisions — Choice, Score and Noul — over hosted Jev, OpenRouter's Decisions endpoint, local laya on PyTorch or MLX, a sentence-transformers CrossEncoder, or a JSON-prompted LLM fallback.
Jev Practices & Patterns#community#awesomejev#catalog-import
Community
gopaljigaur/decidePython
gopaljigaur/decidePythonFrom the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.