Jev directory
Jev Benchmarks & Evaluations
291 curated Jev entries. Benchmarks, calibration studies, independent evaluations, and observability write-ups that measure how Jev actually behaves.
By format
291- Docs53
- Repos & sites131
- Posts1
- Articles102
- Playgrounds4
Updated
All 291 Jev Benchmarks & Evaluations
Search the full category, or filter by format and tag.
SlopCop blog post by Jonathan Ellis, published November 12, 2026, reporting a bakeoff of Jev against Gemini Decisions and OpenAI for classifying GitHub event relevance, the switch to GLiDE from Fastino Labs, and lessons on eval design.
Jev Benchmarks & EvaluationsArticle#evaluation#benchmark#research
Community
A five-notebook Jupyter hands-on that runs Jev and Kev, an Apache-2.0 decision model served locally with llama.cpp, over the SMS Spam Collection, comparing accuracy and threshold effects across Kev-4B, 9B, and 27B.
Jev Benchmarks & Evaluations#evaluation#python#local
Communitydaikumatan/jev_kev_spam_handson
CodeCut article by Khuyen Tran comparing TypeSafe Jev, RapidFuzz, and Sentence Transformers on the Abt-Buy product matching benchmark, with per-tool merge counts and a walkthrough of asking Jev which product is the same.
Jev Benchmarks & EvaluationsArticle#evaluation#benchmark#classification
Community
arXiv paper examining numerical nondeterminism in vLLM Split-K LoRA kernels and introducing a deterministic Triton Split-K implementation, measured on GSM8K, MBPP, and Jev-style structured toxicity decisions over Civil Comments.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
arXiv paper presenting AbductionBench, a unified framework and taxonomy for abductive reasoning tasks, instantiated on 27 datasets across ten domains and used to evaluate eight LLMs from four model families on single-choice explanation selection.
Jev Benchmarks & EvaluationsArticle#arxiv#benchmark#evaluation
Community
Hugging Face dataset holding the scored public predictions, latency timings, and environment for the Aster-JEV-27B vision run: 9,867 of 12,210 requests correct, public index 76.7057, mean 507.2 ms per request on one A800 80GB.
Jev Benchmarks & Evaluations#evaluation#benchmark#open-models
Community
Open, deterministic audit benchmark for System 1 decision models, covering calibration under label-preserving perturbations, cross-question interference, and post-hoc attribution faithfulness across five open-weight models and generative baselines.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
arXiv paper evaluating 67 systems, including 47 decision models and 20 LLMs, on 43 Arabic tasks across 9 task families under a shared zero-shot protocol. Jev 1.14.0 scores 69.24 against 68.77 for Jev 1.13.0 and 71.81 for Gemma-4-31B.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
Hugging Face dataset of medical clinical-decision multiple-choice items in Jev request shape, with a clinical_question state and a decision question carrying type, instructions, and criteria. Between 100K and 1M rows across Chinese, English, Japanese, and four more languages.
Jev Benchmarks & Evaluations#evaluation#benchmark#classification
Community
Multilingual benchmark of nine suites in 122 languages that evaluates typed decision models such as Jev, Clef, and pplx-decider, and tests whether confidence thresholds fitted on English transfer to other languages.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
arXiv paper evaluating Jev as a safety guardrail for a Qwen2.5-1.5B-based pipeline across 970 benchmark prompts, comparing an unfiltered LLM, Jev input filtering, and tiered input filtering with selective output review.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#guardrails
Community
Hugging Face dataset of Jev decision-index results from the mjev-vl-4b vision-language checkpoint, pairing decision and probability outputs with execution metadata, image budgets, latency, and SHA-256 hashes for weights and inference code.
Jev Benchmarks & Evaluations#evaluation#calibration#benchmark
Community
arXiv study adapting an online shopping experiment to three decision models, including Jev, to test whether digital nudges steer non-generative System One models. Reports 600 runs per model and compares default and social-influence choices against a control.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#research
Community
Hugging Face dataset of teacher judgments for ROR affiliation point matching, named for Jev distillation. Each row pairs an affiliation text with ROR identifiers and calibrated probabilities, across 200k rows in train and validation splits.
Jev Benchmarks & Evaluations#calibration#evaluation#classification
Community
Same Facts, Different Remedy: Linguistic Profiling and Arbitrariness in LLM and System One Decisions
arXiv paper auditing three remedy-decision systems on US consumer financial complaints: GPT-4o-mini, GPT-5-mini, and Jev, a probabilistic System One model returning a distribution over remedies. It separates group-scale disparities from individual decision instability.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
arXiv study of two-tier web attack triage that holds seven attack families out of training, recalibration and thresholds, then compares zero-shot decision models including Jev with supervised detectors, a local language model, the OWASP Core Rule Set and an anomaly baseline.
Jev Benchmarks & EvaluationsArticle#arxiv#security#evaluation
Community
arXiv paper auditing AI-text humanizers for fact retention and detector evasion, using a rule-based checker and a zero-shot Jev question to separate human originals from humanized rewrites.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
A synthetic question-answer corpus about the Jev family of decision models, published as a diagnostic for fine-tuning pipelines. Version 1.0 ships 10,389 training pairs and 265 held-out evaluation questions drawn from 58 source units, so a training run can be scored before and after.
Jev Benchmarks & Evaluations#evaluation#benchmark#python
Community
Portuguese-language technical study of Jev, TypeSafe AI's System One decision model, benchmarked against a structured-output LLM baseline. Ships a Python 3.11 harness, a 45-task synthetic eval set, and a runner that records decisions, confidence, latency, tokens, and estimated cost.
Jev Benchmarks & Evaluations#github#evaluation#calibration
CommunityGabriel-Abi-Acl/jev-study
arXiv paper comparing two releases of Ru-Jev, a 307M-parameter multilingual cross-encoder trained on thirteen Russian-language tasks, to test whether an augmentation that presents labels in several surface forms helps when callers supply their own option wording.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
Hightouch blog write-up testing whether fast decision models Jev and GPT-6 Luna can grade identity resolution as well as frontier LLMs. Reports out-of-the-box accuracy, calibration differences, and a hybrid Jev plus Opus harness that reaches frontier accuracy at over 4x lower cost.
Jev Benchmarks & EvaluationsArticle#evaluation#calibration#benchmark
Community
arXiv paper presenting RouteAlign, a method that uses task requirements to structure model performance measurement and LLM service selection. Reports quality gains over open Jev-style decision backends at lower generation cost.
Jev Benchmarks & EvaluationsArticle#arxiv#routing#evaluation
Community
Independent audit of a frozen System One model on 142,658 human-labeled comments, benchmarked against 14 frontier reasoning models. It reports AUROC 0.83, ~USD 0.000025 per call, ~100 ms, and separates human disagreement from decision risk.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
arXiv paper testing representation invariance and abstention on measured optical spectra of silicon metasurfaces and gold nanoparticles, evaluating d3-nano, Jev 1.13, three GPT-6 Luna configurations, and Qwen3.5-2B.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
Controlled audit of presentation sensitivity in typed decision models, comparing Jev, Clef Flash, and Perplexity Decider across native choice and score interfaces on 9,000 multiply annotated humor, helpfulness, and summarization items.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
arXiv paper introducing S1CogBench, a benchmark that runs 24 cognitive experiments against System One decision models, including Jev and five open models, to test whether typed decisions respond to evidence, feedback, memory load, and future opportunities.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
arXiv paper presenting a bias audit protocol for System One decision models, covering BBQ, CrowS-Pairs, and SilverSpoon adapted to the typed interface, twelve models plus a no-thinking ablation, and 1,638,000 decisions with swapped options.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
arXiv study that evaluates ride-hailing rebalancing policies under the EU Platform Work Directive's disclosure requirement, using self-interested LLM driver agents built on Claude Haiku 5.5 and Jev in a three-zone model and in Manhattan.
Jev Benchmarks & EvaluationsArticle#arxiv#research#agents
Community
arXiv paper testing whether Jev can rank candidate demonstrations for in-context learning by judging their utility for a given query, and distilling that judgment into a lightweight MLP to cut the cost of large-scale retrieval.
Jev Benchmarks & EvaluationsArticle#arxiv#research#evaluation
Community
arXiv paper presenting the first systematic bias and fairness audit of Jev, a typed-decision model for recruitment. It measures how demographic attributes and candidate cues shift screening scores, rankings, and shortlisting, against a noise floor, plus validity, latency, and cost.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#research
Community
arXiv paper presenting Slot Attack, a black-box attack that appends text to evidence to steer decision models toward an attacker-chosen wrong option. Evaluates seven decision models on 16 Decision Bench tasks across seven domains.
Jev Benchmarks & EvaluationsArticle#arxiv#security#evaluation
Community
Pytest-based framework that judges mocked LLM outputs with Cloudflare clef-flash, a 9B decision model served locally by Ollama. Metrics turn typed probabilities into values between 0 and 1 and PASS/FAIL against a threshold.
Jev Benchmarks & Evaluations#evaluation#local#python
arXiv paper testing whether compact structured memory improves a probabilistic decision model's answers on 30 fully synthetic clinical cases across twelve domains, evaluated with 120 typed questions and reporting accuracy, log loss, Brier score, token and latency costs.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
Hugging Face dataset of 150 crypto-domain user queries, each labeled with a router intent and a guardrail decision. Rows carry id, text, router, guardrail, and source fields, covering 14 router labels and allow, advice, and off_topic guardrails.
Jev Benchmarks & Evaluations#routing#classification#guardrails
Community
arXiv paper proposing OLIVIA, a protocol for look-ahead-free tests of AI decision models, applied to probability-returning models reading S&P 500 news with masked company identities, validation-only selection, and a hash-frozen analysis plan.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
arXiv evaluation of 14 System-1 decision models, including Jev, against four generative judges on eleven judge benchmarks, covering accuracy, calibration, cost, and robustness to instructions injected into judged responses.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
arXiv paper evaluating 22 hosted decision models and two prompted general models as runtime classifiers for a multi-rule coding-agent policy engine. It reports recall, false-positive, latency, and cost results on a 2,643-call synthetic dataset with a sealed holdout.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
A TypeSafe AI Hugging Face dataset of agent trace observability records for evaluation workflows, with cases (111 rows), questions (1.12k rows), and run_results (999 rows) in parquet, including per-provider and consensus decision blocks.
Jev Benchmarks & Evaluations#evaluation#agents#benchmark
Community
Hugging Face dataset from TypeSafe AI of customer-service dialogs with per-policy decisions from OpenAI and Anthropic models and a mean-probability consensus field. Parquet, English, 1K–10K rows.
Jev Benchmarks & Evaluations#evaluation#benchmark#routing
Community
Hugging Face dataset from TypeSafe AI holding invoice documents as tabular and text data for evaluation workflows. Parquet files pair raw invoice text with structured fields such as line items, quantity, unit price, and amount.
Jev Benchmarks & Evaluations#evaluation#benchmark#python
Community
Hugging Face dataset from TypeSafe AI pairing security incident telemetry with model decision outputs. Tabular and text parquet rows, English, 1K–10K records, loadable with Datasets, pandas, or Polars.
Jev Benchmarks & Evaluations#evaluation#benchmark#security
Community
arXiv study applying Jev and Clef-Flash to manufacturing defect detection with as few as two reference images per class, including agentic refinement of text descriptions and probability calibration. Ships replication code, an R decisions package, and a no-code web app.
Jev Benchmarks & EvaluationsArticle#arxiv#calibration#classification
Community
Independent audit repository testing whether Jev's probabilities are calibrated on EIA weekly petroleum reports, comparing Jev via LangChain TypeSafeClassifier against an LLM-as-classifier, rules, and climatology baselines.
Jev Benchmarks & Evaluations#evaluation#calibration#benchmark
Communityn9omi/jev-calibration-audit
Hugging Face dataset holding the run artifacts of the rsi-jev-27b-vision evaluation: per-shard results, vision scores, latency summaries, environment and provenance files, and worker logs, each recorded with byte size and SHA-256 hash.
Jev Benchmarks & Evaluations#evaluation#benchmark#open-models
Community
Case study evaluating Jev 1.13.0 on time-series decision tasks, including classification, event type prediction, oil temperature direction, and stock movement, converting continuous forecasts into discrete decisions and scoring accuracy and AUC.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
arXiv paper introducing PrimaCons, a four-axis benchmark measuring whether Jev gives consistent decisions when the same semantic state is expressed as different typed query primitives, plus the Cross-Primitive Residual Calibrator (CPRC) proposed to reduce that inconsistency.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
Hugging Face dataset from TypeSafe AI for evaluating decision models on workplace documents. Three subsets cover 150 cases, 7.5k questions and 67.5k run results, distributed as parquet in English under an Apache-2.0 license.
Jev Benchmarks & Evaluations#evaluation#benchmark#python
Community
arXiv paper presenting FlipJev, a fast black-box query-based red-teaming attack that finds short universal adversarial suffixes for Jev. It reports experiments on 12 datasets, 10,240 paired prompt-injection tests, and cross-task suffix transfer of 90-100%.
Jev Benchmarks & EvaluationsArticle#arxiv#security#evaluation
Community
Index of all 291 entries
- Comparing Jev, Gemini, and OpenAI Decision Models
- Jev and Kev SMS Spam Hands-On
- Jev vs RapidFuzz vs Sentence Transformers for Duplicate Records
- Reproducible LoRA Inference Through Deterministic Split-K Reduction
- AbductionBench
- Aster-JEV-27B Results
- Can the Confidence and Decisions of System 1 Models Be Trusted?
- Evaluating Jev on Arabic: Comparing Decision Models and LLMs Across Arabic Tasks and Dialects
- Medical Jev 4B Data
- XDecideBench: Evaluating Typed Decision Models and Confidence Policies Across Languages
- Jev as an LLM Safety Guardrail: An Empirical Study of Input Filtering, Selective Output Review, and Over-Refusal
- mjev-vl-4b Jev Index Decision Results
- Nudging Decision Models: Do Digital Nudges Steer Fast, Generation-Free AI Agents?
- ROR Set Point Matching OpenAlex Jev Distill Dataset
- Same Facts, Different Remedy: Linguistic Profiling and Arbitrariness in LLM and System One Decisions
- Escalate or Close: Fast-Tier Models on Held-Out Web Attack Families
- Humanized but Still True? Auditing Facts and Detection in AI-Text Humanizers
- Jev History Corpus: Post-Cutoff Knowledge Canary
- Jev Study
- Label-Wording Augmentation in a Typed-Decision Encoder
- LLM Judges for Identity Resolution at 4x Lower Cost
- RouteAlign: Aligning Task Requirements with Model Profiles for LLM Service Selection
- Two Uncertainties Are Not One: Disagreement versus Decision-Risk in Cheap Structured Content Moderation
- Do Multimodal Models Decide Consistently on Equivalent Optical Spectra?
- Presentation Sensitivity in Typed Decision Models for Subjective Text Evaluation
- S1CogBench
- Swap, Replicate, Read the Mass
- When Disclosure Is Compulsory: Evaluating Ride-Hailing Rebalancing Policies with Self-Interested LLM Driver Agents
- Can Jev Improve Demonstration Retrieval for In-Context Learning?
- Fast, Cheap, and Fair? Evaluating Decision Models
- JevBreak: Your Model, Their Decision
- LLM Output Evaluation with Clef
- Structured Clinical Memory as Context for Probabilistic Decision Models
- Think Fast Jev Queries
- Thinking Fast, Trading Late: System One Decision Models vs the Market
- To Jev or Not to Jev: When Can System-1 Decision Models Replace LLM Judges?
- Decision Models as Policy Gates for Coding Agents
- EvalSafe Agent Trace Observability Dataset
- EvalSafe Customer Service
- EvalSafe Invoice Processing Dataset
- EvalSafe Security Incidents
- Beyond CLIP: Calibrated Few-Shot Defect Detection With System One Decision Models
- Jev Calibration Audit — n9omi
- RSI-JEV-27b-Results
- Benchmarking System One Models for Time-Series Decision Making: A Case Study of Jev
- Cross-Primitive Consistency in Jev
- EvalSafe OneT
- FlipJev: Fast Black-Box Red Teaming Finds Universal Adversarial Suffixes for Typed Decision Models
- LLM As Jev Dataset
- Predicting What Answer Engines Cite: Open Decision Models, Rerankers and a Distilled 4B Student
- SatOps-Decision
- Truncated Reasoning Under a Budget: Evaluating System-1 Models as Decision-Makers
- Decision and System One models in rheumatology: accuracy and calibration
- JEV-Hardmix
- JevGAG Adaptive Temporal Granularity Figures
- Radiology Guideline-derived Decision Graph for Efficient Clinical Decision Support
- When the Top Answer Decides: Gender Bias in Decision Models
- Open-Jev Dataset
- RubriX: A Fast, Fine-Tunable Non-Autoregressive Rubric Judge
- Confidence Is Not a Promise: A Reference Audit of Typed Decision Models
- MedJev: Evaluating Zero-Shot System-One Decision Making in Medicine
- Can JEV Support Clinical Trial Criteria Assessment?
- TypeSafe Experiment
- Understanding Jev-Style System One Models: Does Structure Reduce Selection Bias?
- EditJev: System One Routing for Image Editing Models
- S²-MAS: Society-Scale Multi-Agent Simulation Framework
- System One Models Are Not Better Rerankers
- Decision Index Results: Kieve-JeV Fractal 300M
- How Well Do Decision Models Transfer to Vietnamese? An Empirical Study of Text Classification
- Kieve-JeV MoE 300M Decision Index Results
- Jev Lab — SuperInstance
- Jev Job Matching
- Jev Localization Bench
- Jev Triage — mohammedahmadi1990
- Jev-RAG-Claim-Verification-130K
- Hello ZIO TypeSafe AI
- Your Language Model Is Already a Decision Model
- Audio Jev RLCD v1
- Oss Jev Train Src
- Oss Jev Distill
- Soro Universal Tool Harness
- AgentDecisionBench
- JevGPT Runs
- Xyntetik Typed Decisions Record
- Typed-Decisions Benchmark Results
- JEVWeb
- SocialJev Evals
- Typed Decisions (Deutsch)
- Open-Jev Typed Decision Datasets
- JEV Encoder Decision Index 0.3 Results
- JEV-Contrast
- Jev RAG Chinese Benchmark (S1)
- Jev Decision Distillation Data (Claude-Labelled)
- Open-Jev Typed Decision Datasets (11-47)
- BASIL
- MFRC
- BABE
- JobFit JevBench
- JevBench (Metamorphic Coherence)
- Bekko System One v0 Decision Index 0.2.1 Results
- Calibrated Typed Decisions
- JEVal — huggingface.co
- Visual Jev Decisions v1
- Jev-class CVE Bench
- JevAny-Multimodal-agentic-Public v0.4
- Synthetic Typed Decisions
- 6G-JEV (RANIntent v1)
- JevAlt Training Data
- Bekko System One Dataset v0
- JevGym
- Grounded AI
- Cultivar
- Jev Playground — deepdave98
- Typed Evals
- Trolley benchmark
- Same Statement, Different Script: Taiwan responses in decision models
- ProbabilityBench (code)
- Jev and open decision-model benchmarks
- Scientific Decision Evaluation
- Decision Blackjack
- Jev on SMAC
- JEVals — duberblock
- Systemone Gridworld
- Evaluating Jev as a Prompt-Injection Guardrail
- Jev Speed Race
- JEV Fraud Risk POC
- Systemone Probe
- System One on Gemma 4 E4B
- jev_test Guardrail Benchmark
- Convex Evaluation Visualizer
- ASSAY-001 — independent verification of Jev
- Jev vs GPT-4.1 on a synthetic survey
- System-One Control Bench — socb.dev
- Are AI Decision Models Biased? A Case Study of Jev (arXiv 2610.19619)
- Jev vs LLM comparison
- RSNA Knee SWA Weights with JEV Labels
- JevEval: Jev-as-a-Judge
- LLM Chess
- Jev-as-a-Judge with Comet Opik
- Laya Jev Lab
- JevPokerBench
- Médula
- Jev + Gymnasium
- Jev Papers
- AstroHan's Jev Harness
- Jev vs. the right baselines
- JevRL
- Jev vs Laya benchmark
- Jev Persian Benchmark
- Classifier Benchmark
- BeelineBench
- Jev for trading: claim vs. measured
- Jev GEO Audit
- Jev Chorale Lab
- Jev Architecture Research
- Jev Tool Search
- Jevcheck
- Jev Eval — 4esv
- JEV Decision Benchmarks
- Jev Testbench
- jev-test (Gemma 4 E2B judged by Jev)
- Jev Playground — jyeswak
- Awesome Jev Lab
- Fedjev Bench
- JudgeJev
- Jev Frontier 100
- Jev Measured
- Jev Toolspace
- Jev as a Guardrail
- JevBench: jev vs gev-1.0 — AnyEval
- JevBench — Benchmark Heaven
- When a Judgment Layer's Self-Reported Fields Lie
- Convex Decision Evals
- An Independent Evaluation of TypeSafe's Jev
- Jev-ToM: Evaluating Theory of Mind in Typed Decision Models (arXiv 2610.18610)
- Cheap, Consistent, Calibrated, but Biased? Typed Decision Models as Judges (arXiv 2610.18122)
- ProbabilityBench: Model Evaluation and Calibration with Exact Ground-Truth Uncertainty (arXiv 2610.17016)
- Does a Hosted Decision Model Respect Irrelevant Alternatives? A Pre-Registered Test of Option-Set Dependence in Jev (arXiv 2610.16655)
- Does System One Transfer? Calibrated Typed-Decision Models, Label Budgets, and Translation for Indonesian Regional Languages (arXiv 2610.14112)
- TypedBench: A Benchmark for Calibration, Framing Sensitivity, and Cost in System One Decision Models (arXiv 2610.11392)
- JEV versus LLMs: Accuracy, Cost and Calibration on Seven Political Science Replications (arXiv 2610.06625)
- GraphDecisionBench: Benchmarking System One Models on Graph Tasks (arXiv 2610.06354)
- General Decision Models: Benchmarking and Insights Beyond Jev (arXiv 2610.03935)
- Benchmarking candidate coverage and rejection policy transfer in typed decision models (arXiv 2610.03387)
- To Jev or Not? Evaluating the Accuracy and Efficiency of Structured Decision Models for Hate-Speech Moderation (arXiv 2610.03324)
- Beyond Answer Confidence: A Controlled Audit of Self-Knowledge in a Black-Box Decision Model (arXiv 2610.01006)
- Evaluating and Benchmarking the System One Model Jev (arXiv 2609.37647)
- Calibrated to Whom? Persona and Language Effects on Cultural Values in JEV (arXiv 2609.36399)
- Jev thinks "I don't know'', but doesn't say it: Introducing Sys1Cal-v1 Dataset for Probability Calibration (arXiv 2609.35342)
- Beyond Calibration: Do a Typed-Decision Model's Probabilities Obey the Probability Axioms? (arXiv 2609.33209)
- Typed Decision Models: An Early Evidence Audit and Evaluation Checklist (arXiv 2609.32160)
- Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks (arXiv 2610.11978)
- Can Decision Models Understand Stance? Evaluating Jev Against General-Purpose LLMs (arXiv 2610.11901)
- Nudge the Judge: Uncovering Program Decision Hijacking in Real-World Jev Applications (arXiv 2610.18612)
- Same Attack, Different Judge: A First Controlled Adversarial Comparison of Jev, LLM Judges, and Purpose-Built Safety Classifiers (arXiv 2610.17820)
- JEV as a Judge for Agent Trace Security: An Empirical Comparison with Generative LLM Judges (arXiv 2609.34862)
- JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models (arXiv 2609.31142)
- Just Ask Jev: Reinforcement Learning for Calibrated Decisions as a Zero-Shot Detector of AI Alignment Failures (arXiv 2609.29429)
- Open-Jev Judgments on CallScreenBench: Calibrated One-Pass Scam Screening with a Small Language Model (arXiv 2609.23959)
- The Company a Question Keeps: Auditing Open Jev-Style Models for Retrieval (arXiv 2610.14631)
- How Reliable Are System-One Decisions in Medicine? A Comparative Study of Jev, Jev-like Models, and LLMs (arXiv 2610.18529)
- Same-Number Citation Swaps: Stress-Testing Jev as a Financial Evidence Judge (arXiv 2610.08675)
- A First Glance at Jev for Network Traffic Classification: Accuracy, Processing Time, and Cost (arXiv 2610.00376)
- Decision Readouts for Text-Mediated Video Anomaly Detection: An Exploratory Evaluation of Jev and Qwen (arXiv 2609.34180)
- Jev in Medicine: A Benchmark Evaluation (arXiv 2609.34024)
- Jev Matches 7B Language Models for Speech-Neuroprosthesis Rescoring (arXiv 2609.33538)
- Same Scores, Different Decisions: Evaluating JEV and Language Models for Legal Document Understanding (arXiv 2609.27678)
- Can Jev Judge Radiology Reports? Evaluating a System One Model for Clinical Factuality (arXiv 2609.27607)
- Jev for Scientific Decisions: Evaluating Semantic Choices and Their Consequences (arXiv 2609.24965)
- JEVQA - Video Quality from Metadata, Bitstream, and Pixel Features with a General-Purpose Decision Model (arXiv 2609.24395)
- Intent Interpretation at RIC Timescales: Jev Decision Models versus Large Language Models in 6G Open RAN (arXiv 2609.23136)
- More Features Are Not More Evidence: Limits of Training-Free Human Activity Recognition with Jev (arXiv 2609.36154)
- Suggested Is Not Superseded: A Pre-Registered Evaluation of a System-One Model as a Memory-Retirement Gate (arXiv 2610.15710)
- Fast Models, Slow Evidence: A Paired and Self-Audited Evaluation of System-1 Decision Models for LLM Agent Harnesses (arXiv 2610.02267)
- JEV-as-a-Judge: Accept When Confident, Escalate When Unsure (arXiv 2609.26550)
- When Faster Decisions Do Not Make Better Agents: An Empirical Study of Jev in Agent Workflows (arXiv 2610.13092)
- Laya as a Typed Probabilistic Assessor: An Independent Reproduction and a Preregistered Study of Calibration and Selective Escalation (arXiv 2609.33843)
- Jev-Bench
- JEV Decision Index Results
- Jev Decisions v1
- Open-Jev v1.1 Data
- Jev Evaluation Traces
- JevAlt Benchmark
- JevBench Evaluation Cases
- Jev Alt SystemOne Eval
- JevEmbed Data
- search_jev Unified v2
- next-jev Tetris Demo
- Jev-AR Bench
- Jev Safety Bench
- Korean JEV Benchmark
- Laya vs Jev — huggingface.co
- EdgeIntent v1
- What the Jev
- Zero-Shot IE Bench
- System One on Snapdragon
- Best Decision Model APIs in 2026
- Typed Decisions Benchmark
- Tasksource Jev Typed-Decisions
- Procedural Typed Decisions
- PT-BR Typed Decisions Benchmark
- Jev Distill Corpus v3
- Typed Decision Leaderboard
- S1MB Leaderboard
- Jev Eval — Shogo-nfrealmusic
- Jev Medhallu Benchmark
- Jev for Physical AI
- Zh Decision Bench
- DeepSearcher Jev stopping experiment
- Jev Acento
- Jev Vs. Open Alternatives
- Jev Synthetic Survey
- S1MB
- Social Risk Qwen Jev
- Jev Test — souvikr
- Jev Judge — 00200200
- DecisionBench
- System-One Control Bench — Exemocaro
- Jev Regress Bench
- Jev Fanout Bench
- Classifier Bench
- Jev Verify
- Jev-as-a-Judge for Agent Evals
- Jev for Voice Agent Turn Detection
- Inside Halv's 57.1% Cost Reduction: SWE-rebench Data
- WorkflowEvals
- Jev Decision Index
- Jev Evals
- Jev Crash Course — nadeemcite
- Jev Lab — JoaoClemer
- Jev Sim
- Jev-is-odd
- Apple Silicon Systemone Benchmark
- System1 Fraud Interceptor Demo
- Reflexgate
- JEVChess — vamsi243
- Jev Access Day
- TypeSafe AI Lib
- Jev Crash Course — nadyth
- Jev Kit — FlorianRiquelme
- LangWatch
- OpenWork
- TypeSafe Jev Calibrate For Code Review
- TypeSafe's Jev Can't See. I Made It Guess What I Drew Anyway — mikulskibartosz.name
- Hev Reranker
- SREGym
- Tasksource
- Convex Evals
- Jev Harness Lab
- Jev on r/AmItheAsshole
- Jev calibration study
Start here with the Jev AI model
Featured entries in benchmarks & evaluations.
An open Claude Cowork-style workspace whose eval testkit can call Jev as a typed verification judge on agent-produced artifacts.
Jev Benchmarks & Evaluations#community#typescript#agents
Community
different-ai/openworkTypeScript
different-ai/openworkTypeScriptJev AI model articles
Original guides that go deeper on Jev benchmarks & evaluations.
A benchmark is an argument, not a verdict. How contamination, selection, and vendor framing shape the numbers behind decision models.
Jev Benchmarks & Evaluations#evals#benchmarks#integrity
Read article
A model score is not a workflow score. How harnesses, traces, and end-to-end labels change what an agent evaluation can tell you.
Jev Benchmarks & Evaluations#evals#agents#traces
Read article
Using a fast decision model to judge traces or outputs is not the same as using a frontier LLM judge. What the evidence says about agreement and cost.
Jev Benchmarks & Evaluations#evals#judges#agents
Read article
From the Jev community
Watch walkthroughs, read the threads, and see what builders are posting.
YouTube: Sam Witteveen on System 1 thinking, then Choice, Score, and Noul demos, and chained actions.
YouTube: Nate Herk compares speed and cost against ordinary models, then builds an X feed classifier and a paper-trading prototype.
YouTube: Ray Amjad puts Jev inside an agentic coding loop and looks at what it costs to run.
Hacker News: OpenAI's Luna Decisions API framed as a fast follow to TypeSafe's Jev wave.
matt_d6 pointsOpenAI Answers TypeSafe's Jev with a Decision API Built on LunaRead the thread on Hacker NewsHacker News: Nikola Balić on what made the Jev launch easy to try and easy to share.
nkko2 pointsWhat TypeSafe Got Right with the Jev LaunchRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsHacker News: SafeDep Gryph plus Jev checks for compromised coding-agent behavior.
2 pointsDetecting Compromised AI Coding Agents with Jev and GryphRead the thread on Hacker News