CommunityJev Guides & Articles
JevBench on Hacker News
The Show HN thread for a public benchmark of typed decision models, including a comment on the launch terms that discouraged benchmarks.
- Format
- Post
- Published by
- Community
- Author
- florianstandhar
- Last verified
- Sep 2026
Works forJev for Evaluation
Used inJev for Software & DevTools
Highlights
- 150 points and 39 comments on 22 September 2026.
- The author says the bench exists to compare open and lookalike decision models with Jev.
- A commenter says the no-benchmarks clause was meant for the limited preview and was left in at launch by accident.
Watch out
That terms explanation is one commenter's account. The benchmark site is the existing JevBench entry.
Questions about this entry
Short answers drawn from the linked project or page.
With care, and the author lists the caveats himself: English only, latency from one server in Germany, and local or demo latency gets a disclosed doubling plus 150 ms, which he calls an informed assumption. He also says 1-point gaps can be noise.
The posted leaderboard puts the model at 74.4 against SemIf at 73.1, which the author says can be noise. A commenter reads it the other way, noting SemIf runs in a browser on raw Qwen with no fine-tuning, and questions the funding behind the gap.
That is the sharpest objection in the thread. A commenter argues the bench misses the space the model was trained for and points at the Doom demo as an example; the author agrees coverage is a real gap and is working on screen and computer-use cases.
One commenter's account, per the entry copy: the clause was written for the limited preview and left in at launch by accident. It is a single explanation offered in the thread, not something the vendor confirmed there.
It combines chance-corrected Intelligence, Calibration, Speed and Cost in a weighted way, and the weighting is configurable. A full run asks 534 English decisions.
Two: who-is-right.app.mintapis.com and is-it-ai-slop.app.mintapis.com. The harness, public items, frozen artifacts, scoring code and per-task outcomes are on GitHub.
Today it covers text decisions such as intent, routing, extraction, scoring and harder multi-constraint cases. The author is working on screen and computer-use and image decisions, with a noindex preview already existing.
More Jev resources like this
958GitHub stars
An independent, community-maintained catalog of open-source projects built with Jev, citing public evidence behind each entry it lists.
Jev Guides & Articles#community#typescript#awesome-list
Community
heyjunpenn/awesome-jevAstro
heyjunpenn/awesome-jevAstroThe r/LocalLLaMA thread arguing that Jev's constrained probabilities are ordinary zero-shot classification, and that the comparison should be other classifiers.
Jev Guides & ArticlesPost#community#open-models#evaluation
Community
A searchable companion site to the heyjunpenn awesome-jev list: 962 GitHub projects with stars, languages, and one-line Jev decision summaries per repo.
Jev Guides & Articles#community#awesome-list#evaluation
Community
The Hacker News thread for frodi-karlsson/onesie, covering stdin questions, shell-safety gates, and calibration over labelled commands.
Jev Guides & ArticlesPost#community#integration#evaluation
Community
The Hacker News thread for enigma-jev, a software Bombe plus live Enigma demo where Jev picks cribs and judges German plaintext.
Jev Guides & ArticlesPost#community#evaluation#games
Community
The Hacker News thread for Selmar's calibration notes on Jev comment rules, batching traps, and ordering.py comparisons.
Jev Guides & ArticlesPost#community#evaluation#code-review
Community
From the Jev guides
Original write-ups on what this entry is for.
The typed-answer shape removes a category of data risk, not all of it. Zero data retention per surface, the legal set to read, and the controls that remain yours to run.
Jev Guides & Articles#privacy#compliance#legal
Read article
TypeSafe serves Jev from one API. A second tier of independent sites wraps it with their own keys, credits, and playgrounds. How to read the disclaimers before you paste a key.
Jev Guides & Articles#ecosystem#trust#hosting
Read article
The first ten days produced three receptions that rarely meet: a Product Hunt launch, a 620-point Hacker News thread, and a r/LocalLLaMA that doubted the framing. Reading them together.
Jev Guides & Articles#launch#community#reception
Read article
From the Jev community
Posts from builders shipping with Jev right now.