User repository on GitHub
Jev research manuscript, evidence, and reproducible paper package
- Forks
- 0
- Last pushed
- Updated Sep 2026
Repo stats from the GitHub API, cached Oct 2026.
CommunityJev Practices & Patterns
Paper Package
An evidence and reproduction package for a Jev decision study: the original fixed-split result on DBLP to ACM entity matching and SciFact relation verification, a later fresh repeat that did not resolve it, and the graph-synthesis work that followed.
- Stars
- 1
- Language
- Python
- License
- AGPL-3.0
- Published by
- Community
- Author
- CompleteDotTech
- Last verified
- Oct 2026
About this project
Read the current record. Trace the claim. Replay the evidence. Current manuscript · Current results · Reproduction guide · Original study. This repository preserves an original Jev decision study and subsequent fresh-call, saved-response, and controlled graph-synthesis studies. The current manuscript maps the studies and their evidence types; CURRENTRESULTS.md links the executed reports and negative findings. Both remain author-review research drafts.
Quoted from the project's README. Read the full README.
Works forJev for Classification
Used inJev for Software & DevTools
- Languages
- Python
Highlights
- The original run reported entity-matching macro-F1 rising from 0.9605 to 0.9859 on 413 DBLP to ACM pairs, with a paired interval excluding zero.
- A fresh full repeat on previously inspected data returned a paired 95% interval of [-0.0086, +0.0425] on entity matching, which includes zero.
- The winning formulation bundled question wording, a typed decision contract and demonstrations, and the study never isolated their separate effects.
- A later study spent 3,024 fresh calls on SciFact groups unused in the archived runs and found no matched-volume edge for any added-call policy.
Watch out
Both the current manuscript and the current results file are described by the author as research drafts still under review. The fresh repeat is explicitly not independent validation, and it did not resolve the original entity macro-F1 gain; relation accuracy on the SciFact side moved descriptively from 286/339 to 290/339 with a paired interval of [-0.0175, +0.0428], which also includes zero, and the original relation-verification comparison never resolved an improvement in the first place. The additional-call study deliberately used training source groups unused within the earlier archived runs, and many of the graph studies that follow reuse recorded responses or supplied and simulated inputs, so their controlled target passes do not establish new semantic accuracy or safe unattended graph writes. Note that the manuscript, results and reproduction guides this README links to live in sibling directories above the package, so the links only resolve inside the original layout. Public datasets are acquired on demand and keep their own notices, and reuse boundaries separate the AGPL-3.0-only code from the manuscripts, saved evidence and third-party data. Copy reviewed against the repository README on 2026-10-04, not from running it.
Questions about this entry
Short answers drawn from the linked project or page.
Not on the repeat. The original fixed-split study reported macro-F1 rising from 0.9605 to 0.9859 on 413 DBLP to ACM pairs with a paired 95% interval of [+0.0010, +0.0540] that excluded zero. A fresh full Jev repeat over the same previously inspected data returned [-0.0086, +0.0425], which includes zero, with accuracy moving descriptively from 405/413 to 408/413. The README is explicit that this repeat is not independent validation.
You cannot tell from this study. The selected formulation combined question wording, a typed decision contract and demonstrations, and the README says the study did not isolate their individual effects. If you want to attribute the improvement, you would have to run the arms yourself; the claim-to-evidence map and frozen run are linked for auditing the original numbers.
The current reproduction guide gives clean-checkout Windows and Unix commands with pinned dependencies, exact dataset hashes, offline replay and the original tests, and the verifier preserves the original archive file by file. One thing to know: research workflows only run when explicitly dispatched, so nothing executes on install. The local validation map lists the checks and the per-study commands.
The original project code is AGPL-3.0-only. The README points to a reuse boundaries document that separates code from manuscripts, saved evidence and third-party data, because those are not covered by the code license. Public datasets are acquired on demand and retain their separate notices, and a CITATION.cff is provided for the package with a separate source and data notes file for its inputs.
It never resolved an improvement. The original comparison on 339 SciFact examples found nothing, and the fresh repeat moved accuracy from 286/339 to 290/339 with a paired 95% macro-F1 interval of [-0.0175, +0.0428] that includes zero. The README treats both the current manuscript and the results file as author-review drafts.
No. An additional-call study spent 3,024 fresh calls on SciFact training source groups that earlier archived Jev runs had never touched, and no added-call policy it tested produced an edge advantage at matched volume. That negative result is recorded alongside the reports rather than buried, which is the point of the package.
Less than the count of studies suggests. Many later graph studies reuse recorded responses or run on supplied and simulated inputs, so passing their controlled target checks does not establish new semantic accuracy, nor that graph writes are safe to leave unattended. The verifier still preserves the 161-file original archive file by file.
More Jev resources like this
24GitHub stars
Unofficial laptop study of parallel constrained decoding: prefill once, share the KV cache across fields, and decide all schema fields in one batched forward pass without writing JSON.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
rorshopping/jev-on-a-laptopPython
rorshopping/jev-on-a-laptopPython22GitHub stars
Offline evaluator that checks whether classifier confidences match reality and converts mistake costs into a deployable threshold, emitting an HTML report plus machine-readable threshold files.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
rlaope/jevalPython
rlaope/jevalPython16GitHub stars
Vendor-neutral harness that tests whether Jev reranking improves a small RAG stack, using frozen hybrid BM25 plus BGE-M3 candidates and isolating retrieval, rerank, and generation stages.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
erendikmenn/jev-rag-benchmarkPython
erendikmenn/jev-rag-benchmarkPython37GitHub stars
Minimal Python recipe that turns any causal LLM into a Jev-style classifier by prefilling choice_index and constraining sampling to legal indexes in one forward pass.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
sgoedecke/system-onePython
sgoedecke/system-onePython39GitHub stars
Open replica of the typed decision API on openly available base models, answering choice, score, and true-probability questions in one forward pass with no decoding.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
mithalouni/system-one-openPython
mithalouni/system-one-openPython941GitHub stars
Type-safe generation layer over existing autoregressive models via JSON Schema on SGLang, returning constrained string, numeric, boolean, and enum answers with optional thinking.
Jev Practices & Patterns#community#jevbest#catalog-import
Community
zmtomorrow/TypeARPython
zmtomorrow/TypeARPythonFrom the Jev guides
Original write-ups on what this entry is for.
Most Jev accuracy problems are state problems. What belongs in a state, what is a pre-made judgment to leave out, and why naming the slice you ask about matters more than the wording.
Jev Practices & Patterns#state#design#primitives
Read article
A Noul returns one number and no separate confidence. How to read the value, pin its boundary with criteria, split compound questions, and set a per-action threshold that matches the cost of being wrong.
Jev Practices & Patterns#noul#primitives#confidence
Read article
A Score is a weighted position, not a ruler. Why the number can land between levels, why levels are judged separately, and how to combine several atomic scores in code instead of one crowded rubric.
Jev Practices & Patterns#score#primitives#scoring
Read article
From the Jev community
Posts from builders shipping with Jev right now.