Jev directory
CommunityJev Benchmarks & Evaluations
mjev-vl-4b Jev Index Decision Results
Hugging Face dataset of Jev decision-index results from the mjev-vl-4b vision-language checkpoint, pairing decision and probability outputs with execution metadata, image budgets, latency, and SHA-256 hashes for weights and inference code.
- Reading time
- 2 min
- Published by
- Community
- Added
- Nov 2026
Highlights
- Publishes Jev index decision results produced by the mjev-vl-4b vision-language checkpoint on Mind Lab's dataset namespace.
- Rows carry model_source fields including checkpoint, base, base_revision, training_step, readout_kind, temperature and precision.
- Effective image budget is recorded as max_image_pixels, max_image_tokens and rows_with_image_tokens.
- SHA-256 hashes cover model weight shards, inference scripts such as scripts/general_mjev_decision_index_engine.py, and raw and gzipped results.
- Released under an MIT license; the Hub row viewer fails on this dataset with a CastError.
Questions about this entry
Short answers drawn from the linked project or page.
It is a Hugging Face dataset published by Mind Lab under the mindlab-research namespace. It holds results from running a Jev-style decision index over the mjev-vl-4b vision-language checkpoint, together with the execution metadata used to produce them.
The dataset is tied to the mjev-vl-4b model. Its model_source structure records a checkpoint, a base model and base revision, a training step, a readout_kind, a mode, temperature, max_sequence_length, max_image_pixels, image policy, candidate policy and precision.
Rows include engine, execution environment, hardware and kit_commit strings, an original_environment_files list, a latency field, and an effective_image_budget struct with max_image_pixels, max_image_tokens and rows_with_image_tokens.
The schema carries an all_decisions_and_probabilities_unchanged boolean alongside raw_results_sha256, gzip_results_sha256 and source_full_results_sha256 fields, which are used to check the published decisions and probabilities against the original run.
The dataset stores weights_sha256 keyed by the five safetensors shards and inference_code_sha256 keyed by files including scripts/general_mjev_decision_index_engine.py, src/spatial_jev/general_mjev.py and src/spatial_jev/decision_targets.py.
The Hub preview fails while casting the stored JSON to the declared feature schema, where struct fields such as model_source and effective_image_budget do not line up with the column layout. Loading the files directly avoids the viewer path.
It is a results and metadata record, not a runnable model, SDK or cookbook. Use it to inspect or re-check one model's decision outputs; it does not provide a downloadable Jev runtime, and the repo card lists an MIT license only.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: Standard One SH (StandardThinking)
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsMore Jev resources like this
Benchmark comparison of an untrained Qwen3.5-9B against hosted Jev across decision, tool-calling and calibration benchmarks. Reports accuracy, latency and ECE on matched tasks, plus a local adapter that serves Qwen through Jev's typed systemone API.
Jev Benchmarks & Evaluations#evaluation#calibration#benchmark
Open, deterministic audit benchmark for System 1 decision models, covering calibration under label-preserving perturbations, cross-question interference, and post-hoc attribution faithfulness across five open-weight models and generative baselines.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
arXiv paper comparing two releases of Ru-Jev, a 307M-parameter multilingual cross-encoder trained on thirteen Russian-language tasks, to test whether an augmentation that presents labels in several surface forms helps when callers supply their own option wording.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
arXiv paper testing representation invariance and abstention on measured optical spectra of silicon metasurfaces and gold nanoparticles, evaluating d3-nano, Jev 1.13, three GPT-6 Luna configurations, and Qwen3.5-2B.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
Hugging Face dataset of synthetic typed-decision records for Jev-style control and probability-estimation tasks. It holds 12 subsets with question, options, target, state_json, and metadata_json fields across train, calibration, validation, test, and ood splits.
Jev Benchmarks & Evaluations#open-models#calibration#evaluation
Community
Hugging Face dataset of run results for the kieve-jev-fractal-300m decision engine, recording per-benchmark tallies of requests, answered, abstained and errored cases along with scores, latency medians and p95s, and retrieval and cost-aware metrics.
Jev Benchmarks & Evaluations#benchmark#evaluation#calibration
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.