User repository on GitHub
- Forks
- 0
- Last pushed
- Updated Sep 2026
Repo stats from the GitHub API, cached Oct 2026.
CommunityJev Demos & Experiments
Tiny Jev
A local Qwen3-Reranker-0.6B judge that returns choice, score and noul answers from yes/no scores rather than generated text, LoRA-tuned on JGLUE NLI and STS and served from a local FastAPI app.
- Stars
- 0
- Language
- HTML
- Published by
- Community
- Author
- karimatayuta
- Last verified
- Oct 2026
Works forJev for Classification
Used inJev for Software & DevTools
Highlights
- Answers come from Qwen3 yes/no scores rather than generated text, so you get choice, score and noul values you can branch on instead of prose to parse.
- The LoRA pilot is reproducible: a pinned Qwen3-Reranker-0.6B revision, a dependency lock, fixed data and bundled tests.
- The API answers status="abstained" with value=null until business data, calibration and thresholds are ready.
- Training, export and serving are separate steps; train on CPU, MPS or CUDA, then export and evaluate the adapter.
Watch out
The README is explicit that this is a research build, not a production judge: business-grade acceptance, probability calibration and automatic adoption are all listed as not yet reached, and it warns directly against using proposed_value or uncorrected distributions as verified correct-answer probabilities or as grounds for auto-approval. Even the research results are mixed, with JNLI accuracy improving while entailment recall and some probability metrics got worse. A clone contains no model weights at all, since models/ and artifacts/ are gitignored, so you have to download the base model yourself and retrain the adapter to reproduce anything. The recorded run is FP32 on Apple Silicon because the BF16 and FP16 MPS path failed on that machine, and checkpoint selection uses internal development loss only rather than the public test split. Licensing is split too: the base model is Apache-2.0 while JGLUE data is CC BY-SA 4.0. Copy reviewed against the repository README on 2026-10-04, not from running it.
Questions about this entry
Short answers drawn from the linked project or page.
Not yet, and the README tells you not to. Fine-tuning on Japanese NLI and STS is a different thing from validating business quality such as inquiry classification or urgency, and while the business data, probability calibration and adoption thresholds are unfinished the API keeps status="abstained" and value=null. Treating proposed_value or an uncorrected distribution as a verified probability is called out as a misuse.
Mixed, and the project reports it that way. On the fixed pilot of 506 JNLI and 251 JSTS sentence pairs over one epoch, accuracy went up while entailment recall and some probability metrics got worse, so it is explicitly not a uniform improvement. The README points at training/reports/RESULTS.md for how to read the numbers and lists per-class metrics and training volume as the open work.
No. The base model under models/ and the LoRA delta under artifacts/ are both gitignored, and the trained adapter is not in git either. You fetch the pinned base revision with download_model.py, verify the bundled data with prepare_data, run the unit tests, then retrain and export your own adapter before the API will serve a tuned answer.
Because the BF16 and FP16 MPS path failed on the recorded machine, an Apple M5 with 24 GiB running macOS, so training runs in FP32 there. CPU and CUDA are supported with --device cpu and --device cuda, and on CUDA the MPS preference variable is unnecessary. The dependency lock is specific to that macOS environment, so a CUDA box needs matching wheels verified separately.
Qwen/Qwen3-Reranker-0.6B at the fixed revision e61197ed45024b0ed8a2d74b80b4d909f1255473. A different revision is a different experiment and should not be mixed with an existing adapter.
Python 3.12, uv, and a one-time connection to Hugging Face. The recorded machine was an Apple M5 with 24 GiB on macOS, and CUDA machines should use a matching PyTorch wheel.
It stays at status="abstained" and value=null while business data, probability calibration and adoption thresholds are unfinished, and proposed values should not be used as approved decisions or verified answer probabilities.
More Jev resources like this
40GitHub stars
Python decision client exposing Choice, Noul, and Score primitives with isolated order-invariant scoring, a deliberative flywheel that crystallizes hard cases into fast paths, and local or hybrid deployment.
Jev Demos & Experiments#community#jevbest#catalog-import
Community
zhangcy122/OpenJevPython
zhangcy122/OpenJevPython34GitHub stars
Local server for typed-decision models that answers many Jev questions in one forward pass with checkpoint routing and micro-batching.
Jev Demos & Experiments#community#jevbest#catalog-import
Community
0xBakeer/arbiterHTML
0xBakeer/arbiterHTML22GitHub stars
Browser extension that scores social hooks keystroke by keystroke with Jev curiosity, emotion, pattern and clickbait checks plus short fix tips.
Jev Demos & Experiments#community#jevbest#catalog-import
Community
ehui1226/hookmeter-jevHTML
ehui1226/hookmeter-jevHTML13GitHub stars
Free starter that teaches turning one document into one small typed decision, built around a fictional travel agency where recorded observations meet inspectable application rules.
Jev Demos & Experiments#community#jevbest#catalog-import
16GitHub stars
Fifty minimal runnable demos pairing Jev's choice, score, and yes-no primitives against a structured-output language-model baseline, each with synthetic scenarios and a small 2D visual.
Jev Demos & Experiments#community#jevbest#catalog-import
0GitHub stars
A benchmark for browser-using agents hosted on Almond: one prompt, five timed events scored by the page, plus Almond-fastloop, a ~200-line Chrome DevTools loop where Jev picks one bounded action per tick.
Jev Demos & Experiments#community#jevbest#catalog-import
Community
eriestra/browser-use-olympicsHTML
eriestra/browser-use-olympicsHTMLFrom the Jev guides
Original write-ups on what this entry is for.
A game tick is a state plus a legal action list. How Clash Royale, Minecraft, and the earlier Mario, Pokémon, StarCraft, and drone loops ask Jev one step at a time.
Jev Demos & Experiments#gaming#agents#control
Read article
From the Jev community
Posts from builders shipping with Jev right now.