Article
CommunityJev Benchmarks & Evaluations
RSNA Knee SWA Weights with JEV Labels
Danial Zakaria's RSNA Knee dataset and writeup release CoAtNet/MIL SWA checkpoints at 0.949 and 0.945 public AUC whose weak labels blend public sources with JEV text-model labels, credited with +0.002 (0.939 to 0.942).
- Format
- Article
- Reading time
- 2 min
- Published by
- Community
- Author
- Danial Zakaria
- Last verified
- Oct 2026
Highlights
- Danial Zakaria (nartaa) publishes CoAtNet/MIL checkpoints trained for 16 epochs at seed 42, with two SWA weights at 0.949 and 0.945 public AUC.
- Weak labels blend public sources with JEV text-model labels; the writeup's table credits "Blend + JEV labels" with +0.002 (0.939 to 0.942).
- JEV HYBRID scores take ACL, MCL, and Fracture from pilkwang's report_labels_v2 and nine other findings from stevenleehans' v4_blend.
- riadmohamed42's JEV labels and verification are credited; a checkpoint-manifest ships SHA-256 values for the two checkpoints.
- The dataset page carries no MRI images or patient data, and Gold-58 expert studies were reused for selection, not independent cross-validation.
Watch out
No licence is stated on the page, and the release contains checkpoints and labels rather than MRI images or patient data. The 0.939-to-0.942 improvement is a single table credit in the author's writeup, and reusing the Gold-58 expert studies for selection means the AUC figures are not an independent cross-validation.
Questions about this entry
Short answers drawn from the linked project or page.
A Kaggle dataset and writeup from the RSNA Knee competition by Danial Zakaria (nartaa): CoAtNet/MIL checkpoints plus weak labels that blend public sources with JEV text-model labels.
The dataset page ships two SWA checkpoints, one at 0.949 public AUC and one at 0.945, from models trained with seed 42 for 16 epochs, alongside a checkpoint-manifest with SHA-256 values.
The writeup's table credits "Blend + JEV labels" with +0.002 over the starting point, 0.939 to 0.942; the accuracy model itself scored 0.949 public AUC.
Public labels plus JEV's own text-model scores: ACL, MCL, and Fracture from pilkwang's report_labels_v2, the nine other findings from stevenleehans' v4_blend, with riadmohamed42's JEV labels and verification credited.
No. The release ships the two SWA checkpoints and the checkpoint-manifest with SHA-256 values; the page states there are no MRI images or patient data in the dataset.
No. The Gold-58 expert studies were reused for selection rather than held out, so the page's AUC figures reflect that selection setup and are not an independent cross-validation of the blend.
The page states no licence for the checkpoints, the manifest, or the labels; the dataset ships models rather than data, so check the Kaggle dataset page and the author's writeup for terms before reuse.
More Jev resources like this
A Kaggle notebook titled "Jev vs LLM comparison", associated with the "Natural Language Processing with Disaster Tweets" competition dataset; the JavaScript-rendered page shows only the title and competition on its public listing.
Jev Benchmarks & EvaluationsArticle#community#benchmark#kaggle
Community
A reproducible experimentation framework and report series for Jev: it runs shared tasks at scale across benchmarks, routing and numeric prediction, and publishes bilingual findings on where the model holds up and where it does not.
Jev Benchmarks & Evaluations#community#python#evals
Community
keta1930/what-the-jevPython
keta1930/what-the-jevPythonA benchmark harness and web UI that cross-compares dozens of zero-shot information-extraction and classification systems — extractor encoders, rerankers, and typed-decision engines like Jev — on accuracy, latency, and cost.
Jev Benchmarks & Evaluations#community#python#benchmark
Community
umstek/zero-shot-ie-benchPython
umstek/zero-shot-ie-benchPythonNotebook benchmark that compares Jev with open models and specialised tools on five document-pipeline chores: language detection (Choice(12)), page orientation (Choice(4)), RVL-CDIP classification (Choice(16)), bundle splitting and parse-tier triage, with liteparse doing all PDF rendering and OCR work.
Jev Benchmarks & Evaluations#community#evaluation#document-processing
Community
run-llama/jev_vs_ossJupyter Notebook
run-llama/jev_vs_ossJupyter NotebookCode and data for evaluating Jev, a System One model, on scientific decisions and how its choices affect downstream results. The S2 collection has 20 groups, 40 scientific Choices and one engineering Choice, scored by deterministic downstream code.
Jev Benchmarks & Evaluations#community#github#benchmark
A Python experiment that adapts Gemma 4 E4B for System One-style decisions: one call takes a situation and a fixed answer list and returns a probability per answer, with per-dataset adapters and held-out results across four datasets.
Jev Benchmarks & Evaluations#community#github#python
Community
omnaiduu/jevPython
omnaiduu/jevPythonFrom the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.