Post
CommunityJev Practices & Patterns
Jev isn't tricked by ordering, and the others are.
Kyle Wild's October 13, 2026 post testing whether decision models inherit the primacy and recency effects of the LLMs underneath: Jev was not tricked by option ordering, while the other decision models he tested were. Reproduction code is linked on GitHub.
- Format
- Post
- Reading time
- 3 min
- Published by
- Community
- Author
- Kyle Wild
- Last verified
- Oct 2026
Works forJev for Evaluation
Used inJev for Software & DevTools
Highlights
- Claim: Jev is order-blind in the author's testing while most copycat 'decision models' are not.
- Hypothesis in the post: the others appear built on fast LLMs, so they show primacy and recency effects - 'and they sure do'.
- A chart compares Jev and other decision models across order-bias and accuracy measures (values not recovered in the capture).
- Published October 13, 2026, with results and reproduction code in a linked GitHub repository and discussion on Hacker News.
Watch out
Page text fetched 2026-10-17; the capture includes the post's prose, date and links but not the comparison chart's data, so no scores, other-model names or methodology details could be verified from this digest. The characterization that the other decision models are built on fast LLMs is the author's own inference. The post invites reproduction via its linked GitHub repository.
Questions about this entry
Short answers drawn from the linked project or page.
Kyle Wild's October 13, 2026 write-up argues that TypeSafe's Jev was not tricked by ordering in his tests, while the other 'decision models' he compared it with were. The headline claim is that Jev is order-blind and the others are not.
Known weaknesses of LLMs where the position of an option in the prompt changes the answer - earlier items (primacy) or later ones (recency) get favoured. The author hypothesized that decision models built on fast LLMs would inherit these effects, then reported that they do.
The captured text does not give the methodology. The post says the author tested whether decision models show primacy and recency effects, and hosts results plus reproduction code in a linked GitHub repository, but model names, tasks, sample sizes and prompts were not recoverable.
Only Jev is named in the captured prose; the other models are described as copycats that call themselves decision models. The comparison chart was not recovered with its values, so neither the participants nor the scores can be quoted from this digest.
Kyle Wild published it on dorkitude.com on October 13, 2026, under the writing section of his personal site. The page footer carries a 2025 copyright line, and the post links to a Hacker News thread for continuing the discussion.
The post says both are hosted in a GitHub repository and links to it as 'this GitHub repo'; the capture does not include the link target, so the repository URL and its contents are not recorded in this digest.
It is one author's test on his own tasks: the digest captured the prose but not the chart, sample sizes or prompts, so the claim cannot be independently checked from this page. Treat it as a promising signal to validate on your own option sets rather than a general guarantee.
More Jev resources like this
A self-hosted implementation of TypeSafe's Jev System One API powered by the 400M-parameter GLiFormer encoder: it serves choice, score, and noul and drops into the official typesafe-sdk via TYPESAFE_BASE_URL, but trails Jev on reasoning-heavy tasks.
Jev Practices & Patterns#community#python#open-models
Community
logan-markewich/jeffPython
logan-markewich/jeffPythonA personal-assistant agent with 100 mocked tools that measures how many steps a task takes when the LLM picks the tool versus when Jev picks it before every model step.
Jev Practices & Patterns#community#typescript#agents
Community
vinilana/jev-eval-agentHTML
vinilana/jev-eval-agentHTMLA comparison arena that runs the same batch of review comments through Jev and DeepSeek, showing processing time, cost, and per-label results with CSV/Excel import, replay, and offline reports.
Jev Practices & Patterns#community#javascript#benchmarks
Community
NanmiCoder/jev-arenaJavaScript
NanmiCoder/jev-arenaJavaScriptIndependent cross-model benchmark for Jev-class decision models, running 534 frozen cases per complete entrant with scoring code and a four-axis score of accuracy, calibration, latency and cost.
Jev Practices & Patterns#community#python#benchmarks
Community
fstandhartinger/jevbenchPython
fstandhartinger/jevbenchPythonAn experiment comparing Jev with GPT-5.6 Luna, GPT-5.6 Terra, and Claude Sonnet 4.6 as evaluators of five frozen weather-agent runs, measuring pass-or-fail accuracy against human labels plus variance, cost, and latency.
Jev Practices & Patterns#community#python#evaluation
Community
danielgshea/jev-as-a-judgePython
danielgshea/jev-as-a-judgePythonSide-by-side benchmark of TypeSafe Jev, Qwen 3.8 27B on Cerebras, and a local Needle 3 across seven synthetic workloads, recording validated outputs, mistakes, latency, tokens, and estimated cost. Raw exports and per-scene limitations are published.
Jev Practices & Patterns#community#typescript#benchmarks
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.