Jev Evaluation

• Typed Decisions Benchmark

  

  The 400-case, 2,000-decision benchmark open models train and report against: four workflows, state plus typed questions, and gold distributions averaged from three teacher samples.

  Jev Benchmarks & EvaluationsDocs#community#benchmarks#evaluation

  Community
• Tasksource Jev Typed-Decisions

  

  2.5 million typed decisions recast from 670 sources: choice, score, and noul rows with per-row licenses, soft annotator targets, and Jev-audited mix weights.

  Jev Benchmarks & EvaluationsDocs#community#evaluation#calibration

  Community
• Procedural Typed Decisions

  

  572,000 procedurally generated decision problems where labels are computed exactly from rules in the state: 13 task families, 4–60 options, and difficulty calibrated against Jev 1.13.

  Jev Benchmarks & EvaluationsDocs#community#evaluation#calibration

  Community
• PT-BR Typed Decisions Benchmark

  

  A native Brazilian Portuguese typed-decisions benchmark: seven choice and noul tasks over legislative, legal, scientific, and fact-checking text, with a tuned 4B model at 72.9% mean.

  Jev Benchmarks & EvaluationsDocs#community#evaluation#calibration

  Community
• Jev Distill Corpus v3

  

  The 741,000-row distillation corpus behind AutoTrust's Jev students: choice, score, and noul rows with full target distributions across yuri and openjev source streams, Apache-2.0.

  Jev Benchmarks & EvaluationsDocs#community#evaluation#distillation

  Community
• Typed Decision Leaderboard

  

  An independent 2,018-item answer-verifier benchmark: the same items and labels for every entrant, each run on its own maker's code, with Jev, ZTC-Judge, Darwin, and open models ranked by weighted AUC.

  Jev Benchmarks & EvaluationsPlayground#community#evaluation#benchmarks

  Community
• S1MB Leaderboard

  

  The System One Mosaic Benchmark leaderboard: about 100 decision models compared across 100+ specialized tasks, with results regenerated from a public dataset rather than estimated.

  Jev Benchmarks & EvaluationsPlayground#community#evaluation#benchmarks

  Community
• DeepSearcher Jev stopping experiment

  8.3kGitHub stars

  Agentic search evaluation inside Zilliz DeepSearcher: a standalone harness replaces the ChainOfRAG reflection call with one Jev Noul per round to decide whether to stop within a seven-round budget, then compares stopping behavior, evidence recall and decision cost on 100 sampled 2WikiMultiHopQA queries.

  Jev Benchmarks & Evaluations#community#rag#evaluation

  Communityzilliztech/deep-searcherPython
• jev-oncall

  4GitHub stars

  On-call tooling where Jev answers four typed questions per alert - actionable, severity, owning team and duplicate-of - and plain code routes, pages or drops. Includes a root-cause benchmark where Jev and code investigate recorded incidents with no general-purpose LLM in the loop, enforcing budgets and verification.

  Jev Integrations & Adapters#community#alerting#incident-response

  Communitymingleiw/jev-oncallHTML
• jev-acento

  0GitHub stars

  Independent pre-registered audit of Jev on Spanish: 19,200 calls over 3,200 paired items on XNLI, PAWS-X, MASSIVE and Belebele. A Spanish state costs 3.0-6.4 accuracy points and roughly doubles ECE on the two hardest tasks, while Spanish instructions change nothing; a CLI reruns the same comparison on your own data.

  Jev Benchmarks & Evaluations#community#evaluation#calibration

  Communitymarcosmartinez/jev-acentoPython
• jev vs. open alternatives

  7GitHub stars

  Notebook benchmark that compares Jev with open models and specialised tools on five document-pipeline chores: language detection (Choice(12)), page orientation (Choice(4)), RVL-CDIP classification (Choice(16)), bundle splitting and parse-tier triage, with liteparse doing all PDF rendering and OCR work.

  Jev Benchmarks & Evaluations#community#evaluation#document-processing

  Communityrun-llama/jev_vs_ossJupyter Notebook
• S1MB

  2GitHub stars

  System One Mosaic Benchmark: a Python evaluator and Next.js viewer comparing Jev and open decision models across 137 English Choice, Noul and Score benchmarks from 106 dataset subsets, with recorded evaluation conditions and a public Hugging Face leaderboard.

  Jev Benchmarks & Evaluations#community#benchmark#leaderboard

  Communityhotchpotch/S1MBPython
• Social Risk Qwen Jev

  6GitHub stars

  Chinese-language research project for multimodal social-risk moderation: a local Qwen3-VL-32B with language-side LoRA emits structured evidence JSON, and a fixed Jev policy turns that evidence into risk bands, with frozen test leaderboards and pure-Jev ablations.

  Jev Benchmarks & Evaluations#community#moderation#multimodal

  CommunityTianJinWeiBoss520/social-risk-qwen-jevPython
• DecisionBench

  3GitHub stars

  An open benchmark runtime for document-grounded decision models: it scores Jev Noul, Choice and Score answers on pinned tasks and counts malformed probability distributions as misses so model comparisons stay reproducible.

  Jev Benchmarks & Evaluations#community#benchmark#evaluation

  CommunityHanno-Labs/decision-benchPython
• judgekit

  4GitHub stars

  CLI and Python library that runs declarative YAML judgment tasks - classify, route, score, verify - natively on Jev Choice/Score/Noul, on any OpenAI-compatible backend, or on local keyword rules, with confidence gates, fallbacks and per-decision cost records.

  Jev Tools & CLIs#community#yaml#classification

  Communitylexingtonhibiki/judgekitPython
• Lies, Damned Lies, and Benchmarks

  

  TypeSafe's 11 September 2026 essay against benchmaxxing: a public eval can select the model even when nobody trains on that test.

  Jev Guides & ArticlesArticle#official#benchmarks#evaluation

  Official
• WorkflowEvals

  16GitHub stars

  The code behind evals.typesafe.ai. It runs four published workflows against Jev and other providers and scores them on reference labels.

  Jev Benchmarks & Evaluations#official#evaluation#python

  Officialtypesafe-ai/WorkflowEvalsPython
• Scoring support calls for policy breaks

  

  Sean Brownell scores sample call transcripts in a harness, including a case where the agent reads out card details, and compares that pass with a frontier-model estimate.

  Jev Media & TalksVideo#community#video#evaluation

  Community
• A playground pass from early access

  

  Turing Post TV tests Jev from early access in the TypeSafe playground, starting from the constraint that it does not write essays or code.

  Jev Media & TalksVideo#community#video#evaluation

  Community
• Why the launch chart is not a benchmark

  

  ML Visualized retells Diogo Almeida's Latent Space point that a launch chart with the new model on top of every row is not something you can trust as a benchmark.

  Jev Media & TalksVideo#community#video#evaluation

  Community
• Jev Decision Index

  

  An unofficial Hugging Face space that tracks open Jev reproductions and scores the ones that finished a frozen benchmark suite.

  Jev Benchmarks & EvaluationsPlayground#community#evaluation#open-models

  Community
• Knowledge Signal

  0GitHub stars

  A Show HN prototype that turns study notes into a confidence-aware knowledge profile by scoring them with Jev.

  Jev Tools & CLIs#community#evaluation

  Communitysumant1122/knowledge-signalJavaScript
• LocalLLaMA: Jev as a classifier

  

  The r/LocalLLaMA thread arguing that Jev's constrained probabilities are ordinary zero-shot classification, and that the comparison should be other classifiers.

  Jev Guides & ArticlesPost#community#open-models#evaluation

  Community
• Best Jev Projects (jevbest.com)

  

  A searchable companion site to the heyjunpenn awesome-jev list: 962 GitHub projects with stars, languages, and one-line Jev decision summaries per repo.

  Jev Guides & Articles#community#awesome-list#evaluation

  Community
• LangWatch

  4.9kGitHub stars

  LangWatch Instant Evals ships a TypeSafe Jev classifier that batches Noul, Score, and Choice judgments on production traces in one POST per text.

  Jev Benchmarks & Evaluations#community#typescript#evaluation

  Communitylangwatch/langwatchTypeScript
• OpenWork

  24kGitHub stars

  An open Claude Cowork-style workspace whose eval testkit can call Jev as a typed verification judge on agent-produced artifacts.

  Jev Benchmarks & Evaluations#community#typescript#agents

  Communitydifferent-ai/openworkTypeScript
• Jeeves

  420GitHub stars

  PostHog's reasoning Jev-style classifier with a diffusion drafter, reporting higher accuracy than Kev-9B and hosted Jev on held-out and JevBench tiers.

  Jev Models & Reimplementations#community#evaluation#open-models

  CommunityPostHog/jeevesPython
• enigma-jev

  3GitHub stars

  A browser Enigma I you can type on, plus live Bletchley-style breaking where Jev ranks cribs and judges plaintext, with research pages and CLI backtests.

  Jev Demos & ExperimentsPlayground#community#playground#evaluation

  Communityagodoy21/enigma-jevTypeScript
• typesafe-jev-calibrate-for-code-review

  3GitHub stars

  A calibration write-up and scripts for Jev comment and code rules on a Unity project, with ordering.py to compare rule versions on real blocks.

  Jev Benchmarks & Evaluations#community#evaluation#code-review

  CommunitySelmar/typesafe-jev-calibrate-for-code-reviewPython
• TypeSafe's Jev Can't See. I Made It Guess What I Drew Anyway

  

  A preregistered Quick, Draw! study: 400 SVG-sketch classifications with Jev versus Sonnet on the same text, plus base64 and image baselines.

  Jev Benchmarks & EvaluationsArticle#community#article#evaluation

  Community
• hev reranker

  15GitHub stars

  A BEIR reranking recipe that asks up to 30 parallel Noul relevance questions in one Jev state, with RESULTS.md against cross-encoders and hosted rerankers.

  Jev Benchmarks & Evaluations#community#evaluation#python

  Communityhev/rerankerPython
• onesie

  2GitHub stars

  A Unix-pipeable CLI that asks Jev (or Berget System One models) one question over stdin, with shell-safety exit codes and a calibrate subcommand.

  Jev Integrations & Adapters#community#cli#shell

  Communityfrodi-karlsson/onesieGo
• Show HN: onesie – pipeable CLI for System One models

  

  The Hacker News thread for frodi-karlsson/onesie, covering stdin questions, shell-safety gates, and calibration over labelled commands.

  Jev Guides & ArticlesPost#community#integration#evaluation

  Community
• Show HN: Jev as the Bletchley Park Analyst

  

  The Hacker News thread for enigma-jev, a software Bombe plus live Enigma demo where Jev picks cribs and judges German plaintext.

  Jev Guides & ArticlesPost#community#evaluation#games

  Community
• Calibrating Jev as a Code Reviewer

  

  The Hacker News thread for Selmar's calibration notes on Jev comment rules, batching traps, and ordering.py comparisons.

  Jev Guides & ArticlesPost#community#evaluation#code-review

  Community
• TypeSafe's Jev Can't See. I Made It Guess What I Drew Anyway

  

  The Hacker News thread for Bartosz Mikulski's 400-drawing Jev versus Sonnet evaluation on SVG text.

  Jev Guides & ArticlesPost#community#evaluation

  Community
• OpenAI answers TypeSafe's Jev with a Decision API built on Luna

  

  The New Stack reports OpenAI's limited-preview Decisions API on GPT-6 Luna: predefined answers with confidence scores in about 150 ms, framed as a response to Jev's decision-model wave.

  Jev Guides & ArticlesArticle#community#evaluation

  Community
• OpenAI Answers TypeSafe's Jev with a Decision API Built on Luna

  

  The Hacker News thread for The New Stack's write-up on OpenAI's Luna-based Decisions API and how it compares to TypeSafe's Jev launch.

  Jev Guides & ArticlesPost#community#evaluation

  Community
• Jev is the fish at the poker table

  

  An independent poker eval against TexasSolver: Jev matches solver top actions on easy spots but shoves the nuts into traps and needs explicit hand-rank text to fold to shown flushes.

  Jev Practices & PatternsArticle#community#evaluation#calibration

  Community
• Typesafe's Jev is the fish at the poker table

  

  The Hacker News thread for backnotprop's TexasSolver poker evaluation of Jev action choices, trap spots, and decomposed question shapes.

  Jev Guides & ArticlesPost#community#evaluation

  Community
• OpenAI's Jev clone could help the frontier lab stop its swarming agents

  

  TechCrunch coverage of OpenAI's DevDay Decisions API beside TypeSafe's Jev, including agent-monitoring cost comparisons and CEO responses on X.

  Jev Guides & ArticlesArticle#community#evaluation#agents

  Community
• Jev vs Kev on Opper

  

  Opper hosts Kev 4B and typesafe/jev-latest on the same /v1/systemone compat route and compares accuracy, tokens, and latency on 362 post-release questions.

  Jev Guides & ArticlesArticle#community#article#open-models

  Community
• I built non-autoregressive decision models with RL a year ago

  

  A long thread on launch language, whether constrained routing is novel, and links to open Jev-like work including OpenDecision.

  Jev Guides & ArticlesPost#community#open-models#evaluation

  Community
• Jev vs Kev tested side by side

  

  The Hacker News thread for Opper's hosted comparison of typesafe/jev-latest and opper/kev-4b on the same wire format.

  Jev Guides & ArticlesPost#community#open-models#evaluation

  Community
• Jev and System One Models: Calibration Beats Accuracy

  

  Thread on Kartik Pansuriya's essay arguing calibrated probabilities matter more than headline accuracy for production classifiers.

  Jev Guides & ArticlesPost#community#calibration#evaluation

  Community
• We tested Jev on 100 real agent tool calls

  

  Hacker News link to Archestra's write-up comparing Jev with a constant benign baseline on production Claude Code tool calls.

  Jev Guides & ArticlesPost#community#agents#evaluation

  Community
• jev-harness-lab

  3GitHub stars

  Evaluation scripts, MCP tools, and a skill router built to measure where jev-1.13.0 fits in agent harnesses, with raw results under eval/.

  Jev Benchmarks & Evaluations#community#evaluation#agents

  CommunityAitejiu/jev-harness-labPython
• Jev AI Model in Action: 5 Practical QA Use Cases + Hands-On Demo

  

  Hands-on QA automation examples with Jev: Playwright failure classification, guardrails, RAG, API calls from Postman, and decision-focused testing.

  Jev Practices & PatternsVideo#community#video#tutorial

  Community
• This AI Can't Write a Single Word. That's the Point.

  

  Heroic steps on typed decisions with confidence scores, parallel output speed, the Doom demo, and citations to TypeSafe, The Register, and Every.to.

  Jev Media & TalksVideo#community#video#explainer

  Community
• Jev on r/AmItheAsshole

  3GitHub stars

  A 770-post benchmark that scores Jev, Sonnet 5, GPT-5 nano, and two local models on Reddit verdicts with weighted Brier scores.

  Jev Benchmarks & Evaluations#community#python#evaluation

  Communitydchristopoulos/jev-aitaPython
• Jev calibration study

  3GitHub stars

  A sentiment study of whether Jev's stated confidence matches accuracy, comparing raw scores with Platt scaling and isotonic regression.

  Jev Benchmarks & Evaluations#community#python#evaluation

  CommunityAnthusAI/Jev-CalibrationPython
• Using Jev for evals in Datadog

  

  Datadog's walkthrough of one Jev rubric that scores every criterion in a single request, then runs as online evals on live spans and as offline experiments.

  Jev Guides & ArticlesArticle#community#article#evaluation

  Community
• Typed judgments or agentic loops?

  

  Sam Reghenzi's product-taxonomy benchmark: a Jev Choice at every level versus a gpt-5.2 agent that descends the tree and can be sent back by a judge.

  Jev Guides & ArticlesArticle#community#article#evaluation

  Community
• mini-Jev

  59GitHub stars

  A preregistered measurement of a Jev-style interface on frozen Qwen3-4B: read the option letter's logits instead of generating grammar-constrained JSON.

  Jev Models & Reimplementations#community#python#local

  Communityr-ms/mini-jevPython
• Jev can't be calibrated

  

  Alex Molas argues that a probability fit on TypeSafe's data is not calibrated on yours, so Jev's numbers should be treated as scores.

  Jev Guides & ArticlesArticle#community#article#calibration

  Community
• GLM-5.3-Flash as a decision model

  

  Privatemode's write-up of typed decisions from GLM-5.3-Flash in one forward pass, with latency, cost, and accuracy set next to Jev and Laya.

  Jev Guides & ArticlesArticle#community#article#evaluation

  Community
• Jev by TypeSafe AI, a System-1 decision model

  

  CampusX on Jev as a classifier over schema-constrained choices, with a live benchmark and an e-commerce review demo.

  Jev Media & TalksVideo#community#video#explainer

  Community
• Turning GLM-5.3-Flash into a Jev-like model

  

  The Hacker News thread on Privatemode's GLM-5.3-Flash write-up, focused on whether a base model plus a classifier head can match Jev.

  Jev Guides & ArticlesPost#community#open-models#evaluation

  Community
• Jev can't be calibrated

  

  The Hacker News thread on Alex Molas's calibration argument, including a commenter who reran the fair-coin and fair-die prompts.

  Jev Guides & ArticlesPost#community#calibration#evaluation

  Community
• Jev vs Claude Opus 5 on Banking77

  

  OpenRouter's Banking77 run: 3,080 support utterances, 77 intents, Jev 1.13 against Claude Opus 5 on the same one-line criteria, with latency, cost, and a confidence cascade.

  Jev Guides & ArticlesArticle#community#article#evaluation

  Community
• Jev vs an LLM on support triage

  

  OpenRouter times Jev 1.13, GPT Luna, and Claude Opus on 60 support tickets: intent plus an escalation flag, then says the reply still belongs to a generative model.

  Jev Guides & ArticlesArticle#community#article#routing

  Community
• Jev hands-on against Sonnet 5

  

  Five self-funded experiments on jev-1.13.0 from a Mac mini, comparing direct HTTP with claude-sonnet-5 after an earlier test mistook CLI startup for model speed.

  Jev Guides & ArticlesArticle#community#article#evaluation

  Community
• What Jev can do inside an agent harness

  

  A black-box pass over ten public datasets with jev-1.13.0: injection detection, reranking, intent, tool routing, and three tasks the author says do not work.

  Jev Guides & ArticlesArticle#community#article#evaluation

  Community
• Jev on 100 real agent tool calls

  

  Archestra labels 100 production Claude Code tool calls and compares Jev with a constant benign baseline, Sonnet 5, Laya, and several open decoders.

  Jev Guides & ArticlesArticle#community#article#evaluation

  Community
• Moderating a site with Jev

  

  Tom Nick's write-up before jevdit.com: irony detection on 100 German soccer tweets, with Jev 1.13.0 next to Opus, Astra, and Laya.

  Jev Guides & ArticlesArticle#community#article#moderation

  Community
• TypeSafe Jev fact-check

  

  Prism Labs checks the launch demos against TypeSafe's own footnotes: what they keep from the 70 ms claim, and what they drop from the speed multiples.

  Jev Media & TalksVideo#community#video#evaluation

  Community
• What the headline multiples compare

  

  Nimrod on what the 200× headline is comparing, where a decision model fits, and a DSPy result he treats as the practical number.

  Jev Media & TalksVideo#community#video#evaluation

  Community
• Four synthetic ticket tests

  

  AI Test Bench runs four tests on synthetic support tickets in the official TypeSafe playground and states what that small set does not prove.

  Jev Media & TalksVideo#community#video#evaluation

  Community
• JevBench on Hacker News

  

  The Show HN thread for a public benchmark of typed decision models, including a comment on the launch terms that discouraged benchmarks.

  Jev Guides & ArticlesPost#community#evaluation

  Community
• Jev on 770 AITA posts

  

  The Hacker News thread for the Am I the Asshole comparison, including a moderator note about generated text.

  Jev Guides & ArticlesPost#community#evaluation

  Community
• jevals

  103GitHub stars

  OpenLayer's eval and guardrail library that replaces LLM judges with Jev-style decisions, packing every check for an agent trace into one request.

  Jev Repos & SDKs#community#python#evaluation

  Communityopenlayer-ai/jevalsPython
• JevHarness

  536GitHub stars

  A Python framework where an authoring LLM writes a task-specific Jev harness, then optionally evolves it using reward reflection and GEPA over full trajectories.

  Jev Tools & CLIs#community#python#agents

  CommunityTianyuCodings/JevHarnessPython
• TypeSafe's Jev Explained: Why AI Decisions May Not Need a Chatbot

  

  The Neuron's plain-English explainer on why Jev replaces the chatbot interface with parallel typed decisions, plus the vendor-reported speed and cost claims.

  Jev Guides & ArticlesArticle#community#explainer#news

  Community
• Jev: The Language Model That Won't Talk

  

  A critical technical read of Jev's launch claims that digs into the calibration and eval numbers and asks what the architecture keeps secret.

  Jev Practices & PatternsArticle#community#evaluation#calibration

  Community
• Jev in Search: Three Practical Evaluations

  

  An independent evaluation of Jev in three retrieval stacks, with recalculated recall, latency, and cost against DeepSeek, GPT-4o-mini, and GPT-5-mini baselines.

  Jev Practices & PatternsArticle#community#search#reranking

  Community
• Is Jev actually good at computer use?

  

  A skeptical computer-use test of Jev with a public test repository, from Builder.io's Steve.

  Jev Practices & PatternsVideo#community#video#computer-use

  Community
• TypeSafe AI Jev: 50 Financial Jobs, Verifiable Demos & Honest Limits

  

  A two-hour recorded run of 50 financial workflows with saved outputs, baselines, and failures, from ledger integrity to AML triage and sanctions matching.

  Jev CookbooksVideo#community#video#finance

  Community
• TypeSafe Jev: the First Decision-Only Model Class, Benchmarked and Priced

  

  A release guide with eval tables, pricing, curl and Python examples, and fit boundaries for Jev's early access.

  Jev Guides & ArticlesArticle#community#article#pricing

  Community
• Typed Decisions, Not Chat

  

  A heavily footnoted claims audit that separates documented Jev facts from marketing, including the seed round, third-party tests, and missing transparency.

  Jev Guides & ArticlesArticle#community#article#analysis

  Community
• A/B testing hooks against 100 personas

  

  Near-real-time A/B testing of TikTok and Instagram hooks against 100 rich personas, replacing a $1 to $5 per-run panel with a few cents of Jev.

  Jev CookbooksPost#community#x#video

  Community
• Workflow evals

  

  Published evaluations of four automation workflows, security incidents, agent trace observability, invoice processing, and customer service, comparing Jev and frontier LLMs as structured workflows versus single prompts.

  Jev Practices & PatternsDocs#official#benchmarks#evaluation

  Official
• TypeSafe AI Rust SDK (community)

  0GitHub stars

  A community Rust port of the official Python SDK with the same primitives, wire format, retries, and error semantics, plus derive macros for typed questions and answers.

  Jev Repos & SDKs#community#rust#sdk

  Communitycommunity-ports/typesafeai-sdk-rust-communityRust
• Supercov

  152GitHub stars

  A CLI that scores code quality by asking Jev yes/no questions about each file, runs your existing test command, and turns uncovered paths into small actionable queries for agents.

  Jev Tools & CLIs#community#rust#cli

  Communitysupercorp-ai/supercovRust
• jev-plays-pokemon-red

  8GitHub stars

  Pokémon Red under PyBoy where ordinary Python owns the route and arithmetic and Jev is asked only at branches, with each battle turn's faint prediction scored by Brier.

  Jev Cookbooks#community#python#games

  Communityvalentynkit/jev-plays-pokemon-redPython
• typesafe-ai-benchmark

  38GitHub stars

  Side-by-side benchmark of TypeSafe Jev, Qwen 3.8 27B on Cerebras, and a local Needle 3 across seven synthetic workloads, recording validated outputs, mistakes, latency, tokens, and estimated cost. Raw exports and per-scene limitations are published.

  Jev Practices & Patterns#community#typescript#benchmarks

  Communityiammrduncan/typesafe-ai-benchmarkTypeScript
• jevcal

  10GitHub stars

  CLI that fits a per-question confidence threshold to a target accuracy on your own labeled data, verifies it on a held-out split, estimates how much traffic still needs an LLM, and re-checks locked thresholds in CI. It publishes no Jev results of its own.

  Jev Practices & Patterns#community#python#calibration

  Communityabhixhek/jevcalPython
• jev-align

  307GitHub stars

  An experimental Sutro CLI that evaluates CSV, Parquet, and JSONL data with Jev, asks you to label ambiguous and random audit rows, and uses GEPA to propose improved function definitions.

  Jev Tools & CLIs#community#python#cli

  Communitysutro-sh/jev-alignPython
• jev-eval-agent

  106GitHub stars

  A personal-assistant agent with 100 mocked tools that measures how many steps a task takes when the LLM picks the tool versus when Jev picks it before every model step.

  Jev Practices & Patterns#community#typescript#agents

  Communityvinilana/jev-eval-agentHTML
• slop-grader

  33GitHub stars

  A CLI that grades markdown and text files against plain-Markdown rulesets with Jev, running every rule against every line in parallel and caching unchanged lines to avoid repeat API calls.

  Jev Tools & CLIs#community#typescript#cli

  Communitylukstei/slop-graderTypeScript
• jev-sec-bench

  3GitHub stars

  Blind security benchmarks for Jev on public corpora: 662 prompt-injection messages and 200 matched vulnerable-code pairs, with raw per-sample results and a TUI dashboard.

  Jev Practices & Patterns#community#go#benchmarks

  CommunityGaurav-Gosain/jev-sec-benchGo
• Zero-shot email classification with TypeSafe's Jev

  3GitHub stars

  An exploratory study of zero-shot ham, spam, and phishing classification with Jev: enriched email context raised main-set accuracy from 93.62% to 98.64%, within 0.23 points of a trained TF-IDF baseline.

  Jev Practices & Patterns#community#python#evaluation

  Communitybitnovus/jev-spam-evalJupyter Notebook
• jev-agent-failure-benchmark

  3GitHub stars

  A benchmark that runs Jev on all 6,257 text traces of Who&When Pro to attribute agent failures, scoring it with the official pinned scorer against the paper's GPT-5.4, Claude Sonnet 4.6, GLM-5, and Qwen3.5-122B baselines.

  Jev Practices & Patterns#community#python#benchmarks

  CommunityTokenTrim/jev-agent-failure-benchmarkPython
• jev-search-rerank-eval

  11GitHub stars

  A graded relevance evaluation of Jev as a reranker: 9,831 labelled pairs from 164 queries over a 33,047-item skills catalog, comparing Jev score reranks with BM25, bge-m3, and rank fusion.

  Jev Practices & Patterns#community#python#search

  Communityzhuyansen/jev-search-rerank-evalPython
• Does Jev know when it doesn't know?

  6GitHub stars

  An independent calibration study of Jev over three public benchmarks and 900 rule-generated support tickets, publishing every raw Gateway response and the quantization limits of returned probabilities.

  Jev Practices & Patterns#community#python#calibration

  Communityscienthoon/jev-ood-calibrationPython
• Jev vs LLM: a phishing decision benchmark with a calibration audit

  8GitHub stars

  A public benchmark comparing Jev with Claude Haiku 4.5 on whether an email agent should click a link, over 2,000 PhishNChips emails, with calibration, latency, cost, and five signal questions.

  Jev Practices & Patterns#community#python#security

  Communityanisselbd/jev-phishing-benchPython
• pytest-jev

  7GitHub stars

  A pytest plugin for semantic assertions: jev.expect sends many holds and lacks claims about one text to Jev in a single request and fails visibly when a probability is uncertain.

  Jev Repos & SDKs#community#python#evaluation

  Communityallebee/pytest-jevPython

47 more entries carry this tag. Browse all 143 in the directory.