User repository on GitHub
Openjev experiments
- Forks
- 0
- Last pushed
- Updated Sep 2026
Repo stats from the GitHub API, cached Oct 2026.Homepage
CommunityJev Practices & Patterns
Openjev Experiments
A single self-contained Python script around AlexWortega/openjev, a Qwen3.5-4B cross-encoder that answers contradiction, entailment or neutral, exposing pair classification, multiple-choice rerank, a library API and a CLI.
- Stars
- 1
- Language
- Python
- License
- MIT
- Published by
- Community
- Author
- zefir1990
- Last verified
- Oct 2026
About this project
Natural language inference with AlexWortega/openjev, a Qwen3.5-4B cross-encoder that reads a premise and a hypothesis and answers with contradiction, entailment, or neutral. classify.py is a single self-contained script exposing pair classification, multiple-choice rerank, a library API, and a CLI. Python 3.10 or newer (3.13 tested) — transformers 5.x and torch 2.14 both require it. About 9.1 GB of disk for the checkpoint, plus roughly 2 GB resident while running.
Quoted from the project's README. Read the full README.
Works forJev for Classification
Used inJev for Software & DevTools
- Languages
- Python
Highlights
- One script covers pair classification, multiple-choice rerank, a library API and a CLI, and the checkpoint is pulled into HF_HOME on first run.
- The checkpoint dictates its own preprocessing: fixed label order, right padding and last-non-pad-token pooling, and deviating from them degrades accuracy.
- max_length defaults to 256 from the checkpoint's own training config; the upstream snippet's 4096 is both far more expensive and a train/inference mismatch.
- Thread count is chosen from whether the weights exceed physical RAM: two threads when paging from disk, half the cores when the work is compute-bound.
Watch out
The checkpoint is 9.1 GB, which is most of what this costs you, and the measured latency is dominated by re-reading those weights rather than by compute. The README is explicit that its timings are order-of-magnitude: the same single-pair command has been observed at both 130 s and 192 s, and only comparisons made back to back inside one process are meaningful. sdpa is set explicitly because Transformers otherwise downgrades to eager on CPU, and the qwen3_5 linear-attention layers fall back to pure-PyTorch reference implementations with a warning on load that is expected and harmless. Copy reviewed against the repository README on 2026-10-04, not from running it.
Questions about this entry
Short answers drawn from the linked project or page.
The README's numbers on an M2 with 8 GB of RAM, on CPU in bf16, are roughly 12 s to load, about 26 s per pair once the process is warm, and 130 to 192 s for a single pair including load and startup, with a peak resident set of 1.3 to 1.7 GB. Weights stay file-backed and the OS reclaims those pages rather than swapping, so the model fits on a machine with far less RAM than the checkpoint, at the cost of speed: page-fault counts against page reclaims and CPU utilisation around 3% show latency is dominated by re-reading weights. A faster host volume is the most direct speedup available.
Only if the wiring is right, and the README spends most of its design notes on that. Label order is fixed by the checkpoint's id2label as contradiction=0, entailment=1, neutral=2; input is a single string built from its nli_template; the head pools the last non-pad token, so the tokenizer must be right-padded and is not interchangeable with CLS or mean pooling. A distribution that looks degenerate, with one label pinned near 1.0 on every input, means the label order or the pooling has been broken. Correct wiring on standard NLI pairs should land near 0.90.
No, it is quoted from the checkpoint. Its train_result.json reports eval_accuracy 0.8985 on 2,000 held-out examples, and the model repository's results/summary.md reports MNLI-m 0.904 and MNLI-mm 0.907. What the repository measures itself is performance, not accuracy: timings, resident set size and page behaviour, and the README explicitly asks you to treat those as order-of-magnitude figures that vary by nearly half run to run.
Expected. causal_conv1d and flash-linear-attention ship no CPU kernels, so the qwen3_5 linear-attention layers fall back to pure-PyTorch reference implementations and Transformers prints a warning on load. The README says the output is unaffected. You also need Python 3.10 or newer, with 3.13 the tested version, because both transformers 5.x and torch 2.14 require it.
Three things. classify takes --premise and --hypothesis for one pair, or --input a JSONL or CSV file whose rows carry premise and hypothesis fields, with --json for machine output. rerank takes --question and repeated --option flags and prints the zero-based index and the winning option. The same script also exposes a library API, so Python callers need not shell out.
Set HF_HOME to a roomier volume before the first run, then optionally pre-download with hf download AlexWortega/openjev --include qwen3.5-4b-nli. The download is optional because the first run fetches the checkpoint automatically into HF_HOME. The README warns this matters most on machines whose home volume is nearly full.
No. ClassifierFactory.create caches its result, so repeated calls with equal settings reuse one loaded model rather than reloading 9 GB. The CLI accepts batch input through --batch-size, default 8 pairs per forward pass, and the library API classifies a list of TextPair objects in one call. The 0.667 GB vision tower is deleted after load unless --keep-vision-tower is passed.
More Jev resources like this
24GitHub stars
Unofficial laptop study of parallel constrained decoding: prefill once, share the KV cache across fields, and decide all schema fields in one batched forward pass without writing JSON.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
rorshopping/jev-on-a-laptopPython
rorshopping/jev-on-a-laptopPython22GitHub stars
Offline evaluator that checks whether classifier confidences match reality and converts mistake costs into a deployable threshold, emitting an HTML report plus machine-readable threshold files.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
rlaope/jevalPython
rlaope/jevalPython16GitHub stars
Vendor-neutral harness that tests whether Jev reranking improves a small RAG stack, using frozen hybrid BM25 plus BGE-M3 candidates and isolating retrieval, rerank, and generation stages.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
erendikmenn/jev-rag-benchmarkPython
erendikmenn/jev-rag-benchmarkPython37GitHub stars
Minimal Python recipe that turns any causal LLM into a Jev-style classifier by prefilling choice_index and constraining sampling to legal indexes in one forward pass.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
sgoedecke/system-onePython
sgoedecke/system-onePython39GitHub stars
Open replica of the typed decision API on openly available base models, answering choice, score, and true-probability questions in one forward pass with no decoding.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
mithalouni/system-one-openPython
mithalouni/system-one-openPython34GitHub stars
Curated catalog of community Jev applications grouped by use, where each case explains the mechanism and states what the public evidence does and does not prove.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
JackZeng/Jev_appsPython
JackZeng/Jev_appsPythonFrom the Jev guides
Original write-ups on what this entry is for.
Most Jev accuracy problems are state problems. What belongs in a state, what is a pre-made judgment to leave out, and why naming the slice you ask about matters more than the wording.
Jev Practices & Patterns#state#design#primitives
Read article
A Noul returns one number and no separate confidence. How to read the value, pin its boundary with criteria, split compound questions, and set a per-action threshold that matches the cost of being wrong.
Jev Practices & Patterns#noul#primitives#confidence
Read article
A Score is a weighted position, not a ruler. Why the number can land between levels, why levels are judged separately, and how to combine several atomic scores in code instead of one crowded rubric.
Jev Practices & Patterns#score#primitives#scoring
Read article
From the Jev community
Posts from builders shipping with Jev right now.