Skip to content

Jev directory

Jev Practices & Patterns

296 curated Jev entries. How to design Jev states, questions, and confidence gates the right way, from the official patterns to the failure modes worth designing around.

By format

296
  • Docs18
  • Repos & sites199
  • Videos7
  • Posts16
  • Articles56

Updated

All 296 Jev Practices & Patterns

Search the full category, or filter by format and tag.

Showing 241–288 of 296 resources

A public benchmark comparing Jev with Claude Haiku 4.5 on whether an email agent should click a link, over 2,000 PhishNChips emails, with calibration, latency, cost, and five signal questions.
Jev Practices & Patterns#community#python#security
A measurement harness that sends each file snippet to Jev as one Noul question asking whether the content contains a usable secret credential, then scores the returned probability against the expected label.
Jev Practices & Patterns#community#python#security
26GitHub stars
A probability-aware evaluation harness that compares TypeSafe Jev with GLiNER2.5 on zero-shot single-label text classification, measuring calibration, coverage at a fixed error budget, latency, and token cost.
Jev Practices & Patterns#community#python#benchmarks
6GitHub stars
A toy archipelago air-traffic demo where every in-flight judgment - divert or continue, declare an emergency, give way, and who lands first - is made live by Jev from each plane's local state in about 150 ms.
Jev Practices & Patterns#community#html#demo
2GitHub stars
An independent calibration study of Jev against a frontier fallback on Banking77 and Web of Science, packaged as the janus-decide tool that measures a routing threshold on your own data instead of shipping a default one.
Jev Practices & Patterns#community#python#routing
2GitHub stars
A reproducible harness that scores Jev Ultrafast research-browser runs over 11 baseline cases plus human and quant stress suites with CoS-locked QC grades, regenerating the field note and trace notebooks offline.
Jev Practices & Patterns#community#python#evaluation
2GitHub stars
A Next.js playground that benchmarks Jev against GPT-5.6 Luna, Claude Haiku 4.5, Gemini 3.5 Flash Lite, and GPT-6 Astra in tic-tac-toe and Connect Four, giving every model the same legal-move Choice options through the Vercel AI Gateway.
Jev Practices & Patterns#community#typescript#evaluation
295GitHub stars
A ModernBERT-based open decision model that evaluates Choice, Score, and Noul schemas in one non-autoregressive pass; Verdict 2.0 reports 77.10% accuracy on 2,000 held-out enterprise decisions.
Jev Practices & Patterns#community#python#open-models
16GitHub stars
Unofficial companion to dspy-typesafeify that records and replays TypeSafe calls to measure calibration, selective risk, confidence-gated abstention, latency, tokens and modeled cost for Jev decisions.
Jev Practices & Patterns#community#python#evaluation
0GitHub stars
Independent measurement of whether ORDER BY over a Jev probability is defensible, pre-registering gates on pairwise inversion, Score ordinality, calibration, and negation and paraphrase invariants.
Jev Practices & Patterns#community#python#benchmarks
282GitHub stars
Independent cross-model benchmark for Jev-class decision models, running 534 frozen cases per complete entrant with scoring code and a four-axis score of accuracy, calibration, latency and cost.
Jev Practices & Patterns#community#python#benchmarks
0GitHub stars
A pre-registered, independently rescored audit of Jev's calibration and type safety: calibrated on CLINC150 (ECE 0.0204), systematically overconfident on Banking77 (ECE 0.0936), and zero type errors across 8,576 responses.
Jev Practices & Patterns#community#python#calibration
15GitHub stars
A from-scratch one-pass option scorer that maps a context and a changing list of text options to one probability per option, adding a hashed n-gram encoder, rival-aware attention, a gated head, and temperature scaling; JSONL data is drop-in compatible with jevlike.
Jev Practices & Patterns#community#python#calibration
134GitHub stars
One-pass option scoring with a local Gemma 3 4B on Apple silicon via MLX: it prefills the context once, expands that KV cache across the option batch, and scores every option in a single padded forward pass with no decoding.
Jev Practices & Patterns#community#python#open-models
110GitHub stars
An encoder-side reproduction of Jev built on a retrained GLiClass ModernBERT base (151M): it evaluates multiple typed questions in one non-autoregressive forward pass, returning choices, ordinal scores, and probabilities under 35 ms.
Jev Practices & Patterns#community#python#open-models
169GitHub stars
An open re-creation of Jev's decision model on an open-weights Qwen3.5-4B: it answers a list of typed questions in one forward pass over POST /v1/systemone in about 200 ms, never writing free text.
Jev Practices & Patterns#community#python#open-models
An experiment comparing Jev with GPT-5.6 Luna, GPT-5.6 Terra, and Claude Sonnet 4.6 as evaluators of five frozen weather-agent runs, measuring pass-or-fail accuracy against human labels plus variance, cost, and latency.
Jev Practices & Patterns#community#python#evaluation
123GitHub stars
A comparison arena that runs the same batch of review comments through Jev and DeepSeek, showing processing time, cost, and per-label results with CSV/Excel import, replay, and offline reports.
Jev Practices & Patterns#community#javascript#benchmarks
303GitHub stars
A self-hosted implementation of TypeSafe's Jev System One API powered by the 400M-parameter GLiFormer encoder: it serves choice, score, and noul and drops into the official typesafe-sdk via TYPESAFE_BASE_URL, but trails Jev on reasoning-heavy tasks.
Jev Practices & Patterns#community#python#open-models
28GitHub stars
A bilingual field map of where Jev fits and where it fails, separating the author's raw API suites from cited third-party results and TypeSafe's own claims.
Jev Practices & Patterns#community#python#evaluation
75GitHub stars
An independent CUDA and vLLM take on parallel typed decisions: it scores verified single-token labels with the model's own head and assembles JSON in code.
Jev Practices & Patterns#community#python#javascript
10GitHub stars
An open, local-first alternative to Jev that reproduces the Choice, Score, and Noul interface on commodity zero-shot NLI models with temperature scaling and conformal abstention, and runs offline with no API key.
Jev Practices & Patterns#community#python#calibration
Every's head of evals ran Jev across 27 of his articles plus 10 AI-styled fakes, asking 21 AI-tell questions about each and reporting what the judgments caught and where he would not trust them yet.
Jev Practices & PatternsArticle#community#evaluation#writing
Community
An independent measurement spanning 5,477 test rows and 34.1M input tokens, comparing one direct Jev question per row with 12-14 scored dimensions fitted to local labels.
Jev Practices & PatternsArticle#community#benchmarks#evaluation
Community
Sixteen thousand calls against gpt-5.4-mini and gpt-5.6-luna on four public datasets plus production pipeline decisions, with a threshold procedure and an explicit filter-not-replacement verdict.
Jev Practices & PatternsArticle#community#benchmarks#evaluation
Community
An independent trial running Jev on 24 Norwegian resource-tax hearing responses with eleven typed questions each, then comparing agreement, calibration, latency and cost against DeepSeek V4.1 Flash.
Jev Practices & PatternsArticle#community#evaluation#calibration
Community
A use-case study comparing Jev, Mistral Small 4, and Gemini 3.5 Flash-Lite on rejecting unsuitable local-event listings, with individually tuned prompts, accuracy, response times, and costs.
Jev Practices & PatternsArticle#community#article#benchmarks
Community
A 2026 technical SEO audit price survey plus a measured case where a Jev pipeline crawled and judged 1,204 pages, producing 4,816 typed judgments in under three minutes for under $1 in model cost.
Jev Practices & PatternsArticle#community#article#pricing
Community
An independent benchmark of hosted Jev against six LLMs on PubMedQA, Banking77, and HelpSteer2, scoring decisions against human labels and reporting per-board results for each model.
Jev Practices & PatternsArticle#community#benchmarks#evaluation
Community
A 706,000-parameter model fills a whole form for a fraction of a cent, presented as the specialist case for System One models on narrow tasks.
Jev Practices & PatternsPost#community#x#open-models
Community
An argument that security work is full of bounded engineering decisions that belong to Jev rather than a chat model, with examples of the calls that fit.
Jev Practices & PatternsPost#community#x#security
Community
A measured search reranking run over 33,047 catalog entries, 164 real queries, and 9,831 labelled pairs, reporting how Jev compares with BM25 and bge-m3.
Jev Practices & PatternsPost#community#x#search
Community
Notes from a technical analysis that inferred Jev's internals from roughly 1,000 API calls, probing how probabilities and confidence behave.
Jev Practices & PatternsPost#community#x#analysis
Community
How to shape what Jev evaluates: a string, a named object, or an array of messages. Every question in a request sees the same state and is evaluated independently.
Jev Practices & PatternsDocs#official#docs#state
Official
A maintained list of jev-1.13's known failure modes, literal reading, unreliable counting, dates as text, indirection, context rot, and contradictory criteria, each with a guardrail.
Jev Practices & PatternsDocs#official#docs#evaluation
Official
Pick one option from a set you define. Returns the option, the full probability distribution, and 0-1 confidence, with up to 255 options and parallel questions that barely add latency.
Jev Practices & PatternsDocs#official#primitives#choice
Official
Rate content against 2 to 10 ordered levels. Returns a probability-weighted score that can land between levels, per-level probabilities, and confidence, with the arithmetic left to code.
Jev Practices & PatternsDocs#official#primitives#score
Official
The yes/no primitive: one probability from 0 to 1, where the value is the answer and no separate confidence is needed. True and false criteria pin down subtle boundaries.
Jev Practices & PatternsDocs#official#primitives#noul
Official
Instructions, option descriptions, score levels, and noul criteria all accept JSON, so you can label question parts, pass schemas and taxonomies, and keep candidate paths alive.
Jev Practices & PatternsDocs#official#primitives#architecture
Official
Break a judgment into atomic Score questions, normalize each by its top level, and combine them with weights you control in code, as in the resume example's four scored dimensions.
Jev Practices & PatternsDocs#official#patterns#score
Official
Use the answer to decide what to do and confidence to decide whether to act: the voice-banking example routes below 0.6 to a human and needs 0.85 or more to auto-approve a transfer.
Jev Practices & PatternsDocs#official#patterns#confidence
Official
Ask everything the system might need in one request, then let code throw away what it does not use. The ticket-triage example sends a category plus four speculative questions in parallel.
Jev Practices & PatternsDocs#official#patterns#batching
Official
Classify intent with a Choice and complexity with a Score, then send each branch to deterministic code, a specialist model, or a person, with a 0.5 intent-confidence floor.
Jev Practices & PatternsDocs#official#patterns#routing
Official
155GitHub stars
An agent skill for writing and improving programs that call Jev: question design, state structure, answer composition, confidence thresholds, and how to diagnose a question that keeps coming back wrong.
Jev Practices & Patterns#community#skills#claude-code
A map of documented Jev use cases, routing, guardrail verification, citation checking, reranking, rubric scoring, bulk classification, and semantic linting in CI, each with a confidence policy.
Jev Practices & PatternsArticle#community#routing#evaluation
Community
An engineering note that places Jev as a typed micro-decision layer inside an agent harness rather than a planner, with preconditions for confidence gates and fan-out and a runtime split into five parts.
Jev Practices & PatternsArticle#community#agents#routing
Community
A comparative review of Jev and the open-source Laya encoder: interfaces, deployment, disclosed internals, calibration claims, and the limits of agreement-based workflow evals.
Jev Practices & PatternsArticle#community#evaluation#calibration
Community
Yuchen Jin's take on Jev for LLM-as-a-judge work: 20-200x faster and 40-400x cheaper, with a screenshot of the model picking which candidate is AGI after a prompt sweep under $0.10.
Jev Practices & PatternsPost#community#x#evaluation
Community

Index of all 296 entries

Start here with the Jev AI model

Featured entries in practices & patterns.

The box question type: one bounding box per question, returned as four edges that are fractions of the image, plus confidence, with no criteria field and no not-found value.
Jev Practices & PatternsDocs#official#docs#vision
Official
Published evaluations of four automation workflows, security incidents, agent trace observability, invoice processing, and customer service, comparing Jev and frontier LLMs as structured workflows versus single prompts.
Jev Practices & PatternsDocs#official#benchmarks#evaluation
Official
9.3kGitHub stars
Apache-licensed, locally runnable Jev-style decision models (0.8B, 4B, 9B on Qwen3.5) with released weights, training code, a System One-compatible server, frozen eval suites, and a playground.
Jev Practices & Patterns#community#python#open-models

Jev AI model articles

Original guides that go deeper on Jev practices & patterns.

Jev's compaction tools keep or drop tool calls instead of summarizing them — the thresholds they fire at, the savings they report, and why 1.14's opt-in 262k context doesn't settle the choice.
Jev Practices & Patterns#compaction#long-context#coding-agents
Read article
Per-tick buy/sell calls on Monad, a five-sleeve Hyperliquid desk, Nifty-50 rankers, and Polymarket confidence gates: how the catalog's trading repos wire Jev in without asking it to predict prices.
Jev Practices & Patterns#trading#markets#gates
Read article
How the box question type turns a crop into a decision: four fractional edges, a presence Noul for missing targets, the cut-off problem, and two patterns worth building.
Jev Practices & Patterns#vision#box#patterns
Read article

From the Jev community

Watch walkthroughs, read the threads, and see what builders are posting.

YouTube: A podcast segment on TypeSafe AI, its $40M raise, and the Jevons paradox behind the name.

YouTube: Gary Explains the catch: Jev understands natural language but answers with structured values and confidence.

YouTube: Sam Witteveen on System 1 thinking, then Choice, Score, and Noul demos, and chained actions.

Hacker News: Pipeable onesie CLI with shell-safety exit codes and a calibrate mode over labelled commands.

frodikarlsson3 pointsShow HN: onesie – An expressive Unix-pipeable CLI for System One models like JevRead the thread on Hacker News

Hacker News: Spoken-language learning with video scenes branching on TypeSafe Jev.

j_rcht1 pointShow HN: Worklaude – learn by speaking; video scenes branch on TypeSafe JEVRead the thread on Hacker News

Hacker News: OpenAI's Luna Decisions API framed as a fast follow to TypeSafe's Jev wave.

matt_d6 pointsOpenAI Answers TypeSafe's Jev with a Decision API Built on LunaRead the thread on Hacker News

Hacker News: Nikola Balić on what made the Jev launch easy to try and easy to share.

nkko2 pointsWhat TypeSafe Got Right with the Jev LaunchRead the thread on Hacker News