Jev directory
CommunityJev Benchmarks & Evaluations
RSI-JEV-27b-Results
Hugging Face dataset holding the run artifacts of the rsi-jev-27b-vision evaluation: per-shard results, vision scores, latency summaries, environment and provenance files, and worker logs, each recorded with byte size and SHA-256 hash.
- Reading time
- 2 min
- Published by
- Community
- Added
- Nov 2026
Highlights
- Dataset of evaluation run artifacts for rsi-jev-27b-vision, published under upstream-evaluation-data-terms.
- Top-level files include results.jsonl, comparison.json, latency_summary.json, provenance.json, and vision-scores.json.
- Eight shards, numbered 0 through 7, each carry their own results.jsonl, environment.json, and status.json.
- Worker logs worker-0 through worker-7 sit alongside suite_verification.json and completion_check.json.
- Every listed file is recorded with a byte size and SHA-256 hash; the dataset viewer reports a column-name mismatch.
Questions about this entry
Short answers drawn from the linked project or page.
It holds the output artifacts of an evaluation run for rsi-jev-27b-vision: results.jsonl, comparison.json, completion_check.json, environment.json, latency_summary.json, prediction_differences.jsonl, provenance.json, status.json, suite_verification.json, and vision-scores.json.
Everything sits under a runs/rsi-jev-27b-vision directory, with the top-level JSON files, a shards folder containing shards 0 through 7, and eight worker log files named worker-0.log through worker-7.log.
Yes. Each file entry in the listing records both a byte size and a SHA-256 hash, so downloaded artifacts can be compared against the recorded digests.
The dataset card lists the license as upstream-evaluation-data-terms, which governs reuse of the evaluation data.
The viewer raises an error because column names do not match across the file listing, so rows cannot be decoded for preview. The underlying files remain listed with their sizes and hashes.
No. The listing covers evaluation outputs, environment and provenance records, shard status files, and worker logs for rsi-jev-27b-vision; no model weights or inference code appear in the file list.
It is a record of one evaluation run rather than a runnable model or benchmark harness. Use it to inspect or reproduce reported results for rsi-jev-27b-vision, not to run Jev decisions yourself.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: Standard One SH (StandardThinking)
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsMore Jev resources like this
Benchmark comparison of an untrained Qwen3.5-9B against hosted Jev across decision, tool-calling and calibration benchmarks. Reports accuracy, latency and ECE on matched tasks, plus a local adapter that serves Qwen through Jev's typed systemone API.
Jev Benchmarks & Evaluations#evaluation#calibration#benchmark
arXiv paper scoring candidate pages by the citation share they will receive, comparing pretrained rerankers, Jev, eight open decision models and distilled students on GEO-Bench queries answered by ten engines. Releases GEO Citation Ranker 4B, the benchmark, all scorer predictions and code.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
JEV-Hardmix is a synthetic English text dataset on Hugging Face holding 10K to 100K rows of decision-model scenarios: archive clauses with thresholds and retention windows, plus questions whose criteria map options to P0 through P3 priorities. Files are parquet with soft labels.
Jev Benchmarks & Evaluations#evaluation#benchmark#open-models
Community
Hugging Face dataset of synthetic typed-decision records for Jev-style control and probability-estimation tasks. It holds 12 subsets with question, options, target, state_json, and metadata_json fields across train, calibration, validation, test, and ood splits.
Jev Benchmarks & Evaluations#open-models#calibration#evaluation
Community
Hugging Face dataset of run results for the kieve-jev-fractal-300m decision engine, recording per-benchmark tallies of requests, answered, abstained and errored cases along with scores, latency medians and p95s, and retrieval and cost-aware metrics.
Jev Benchmarks & Evaluations#benchmark#evaluation#calibration
Community
An evaluation results dataset publishing per-benchmark outcomes for scairesearch/kieve-jev-moe-300m, including request, answered, abstained, and error counts, scored metrics, successful-request latency medians and p95s, and the runner configuration used for the run.
Jev Benchmarks & Evaluations#benchmark#evaluation#calibration
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.