CommunityJev Guides & Articles
Does an open-weight decision model beat a hosted one?
Discussion of ASTGL's held-out routing replay that chose local Laya over hosted Jev for one Mac Studio deployment.
- Format
- Post
- Published by
- Community
- Author
- Jmeg8r
- Last verified
- Sep 2026
Used inJev for Software & DevTools
Highlights
- 15 points on 22 September 2026.
- Story links the ASTGL article cataloged as astgl-laya-vs-jev.
- Commenters note the blog post reads like LLM-generated prose; numbers should be checked in the article body.
Watch out
Thread tone is skeptical of the writing, not necessarily the replay math. Point totals move.
Questions about this entry
Short answers drawn from the linked project or page.
No, and the author says so himself. The 37-of-40 against 33-of-40 result is versus his own deterministic router; he never ran the hosted model on those cases. Read it as local model versus rules, not as a competitive benchmark.
Real, and it cuts against the local path. The author reports p95 rising from 143 ms to 467 ms on a busy host with identical decisions, which is why the 300 ms timeout carries the design. Forty correlated decisions is a thin basis for that conclusion.
Not zero-shot, per one commenter: it tracks the hosted model on a few labels and falls off a cliff at 77, giving 38% against 76% on banking77. What closed the gap for them was a small head trained on the task's own rows, with fallback to the provider for the rest.
Arithmetic over fields. A frozen encoder learns what text says, not how to compute: one commenter's risk rule scored 0.42 phrased as an amount comparison and 0.94 when restated in words. That kind of thing passes a smoke test and then fails silently in production.
One compared Jev with DeepSeek as the decider for cutting live speech into sentences on 226 segments: with Jev the longest held segment was 17 s, with DeepSeek 40 s. Another reported Jev 98% versus Laya 15% on a 1,000-item synthetic set.
The 37/40 versus 33/40 result is against the author's own deterministic router, not against Jev, which was never run on these cases. The 40 held-out decisions are correlated, and p95 went from 143 ms to 467 ms on a busy host.
Zero-shot Laya tracks Jev on 2-4 labels and falls off a cliff on 77, at 38% versus 76% on banking77 in JevBench. Training a small head on the task's own rows over a frozen encoder closed much of that gap, but the encoder learns what the text says, not arithmetic over fields.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
Hacker News: ASTGL replay chose local Laya routing over hosted Jev on one Mac Studio setup.
Jmeg8r15 pointsDoes an open-weight decision model beat a hosted one? Jev vs. LayaRead the thread on Hacker NewsReddit: A walkthrough of routing between models with Jev.
Facebook: Intake Router keeps diagnosis out of the answer space at the front desk.
FacebookClinicnotes
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
More Jev resources like this
James Cruce replays held-out routing decisions with local Laya against deterministic ordering, not a full Laya-versus-Jev accuracy bake-off.
Jev Guides & ArticlesArticle#community#article#open-models
Community
632GitHub stars
A curated list of System One and typed decision models: hosted APIs, open weights, runtimes, SDKs, apps, benchmarks, and papers, in English and Chinese.
Jev Guides & Articles#community#awesome-list#open-models
The r/LocalLLaMA thread arguing that Jev's constrained probabilities are ordinary zero-shot classification, and that the comparison should be other classifiers.
Jev Guides & ArticlesPost#community#open-models#evaluation
Community
Firecrawl's September 2026 explainer on Jev primitives, Vercel AI Gateway access, openjev/logit reproductions, and skeptical Reddit threads about classification prior art.
Jev Guides & ArticlesArticle#community#tutorial
Community
Opper hosts Kev 4B and typesafe/jev-latest on the same /v1/systemone compat route and compares accuracy, tokens, and latency on 362 post-release questions.
Jev Guides & ArticlesArticle#community#article#open-models
Community
The Hacker News thread for Opper's hosted comparison of typesafe/jev-latest and opper/kev-4b on the same wire format.
Jev Guides & ArticlesPost#community#open-models#evaluation
Community
From the Jev guides
Original write-ups on what this entry is for.
The typed-answer shape removes a category of data risk, not all of it. Zero data retention per surface, the legal set to read, and the controls that remain yours to run.
Jev Guides & Articles#privacy#compliance#legal
Read article
TypeSafe serves Jev from one API. A second tier of independent sites wraps it with their own keys, credits, and playgrounds. How to read the disclaimers before you paste a key.
Jev Guides & Articles#ecosystem#trust#hosting
Read article
The first ten days produced three receptions that rarely meet: a Product Hunt launch, a 620-point Hacker News thread, and a r/LocalLLaMA that doubted the framing. Reading them together.
Jev Guides & Articles#launch#community#reception
Read article
From the Jev community
Posts from builders shipping with Jev right now.