Jev directory
CommunityJev Models & Reimplementations
ruvector-typesafe HWU64 Bank
Trained example bank for @ruvector/typesafe on the HWU64 dataset: 64 home-assistant intents across 21 domains, distributed as bank.json and questions.json with frozen split assignments rather than model weights.
- Reading time
- 2 min
- Published by
- Community
- Added
- Nov 2026
Highlights
- Bank stores labelled examples and frozen splits, not weights; heads and temperature calibration are refit when the engine loads it.
- Measured 79.9% accuracy on a held-out test split of 6,133 utterances across 68 classes.
- p50 latency 5 ms and p95 latency 7 ms with the bge-small-en-v1.5 encoder.
- Trained on 19,552 labelled examples; the bank is encoder-independent but the accuracy number is encoder-specific.
- First decision after loading is slow: probeIterations 4000 over 19,552 examples takes minutes, not milliseconds.
Questions about this entry
Short answers drawn from the linked project or page.
Install the library with npm install @ruvector/typesafe, download bank.json and questions.json from the model repository, then run npx typesafe decide with --questions, --bank, --embedder onnx and the engine options the bank was trained under.
79.9% on a held-out test split of 6,133 utterances covering 68 classes, measured with the bge-small-en-v1.5 encoder. Only that encoder was measured for this bank; the other bundled encoder will load the bank but has no number here.
A bank stores examples and their frozen splits, never hyperparameters, and the head is refit from it on load. Omitting the options refits with library defaults of 400 iterations and head auto, which is a different and materially worse model than the one measured.
No. The bank is described as typed decisions over text, locally, with no API bill and no network in the decision path. The examples and questions files are downloaded once and then used offline.
The dataset licence is CC-BY-4.0 and this bank redistributes the utterance text under that licence. The @ruvector/typesafe code itself is MIT. Credit goes to Liu et al., Benchmarking Natural Language Understanding Services (2019).
It is English only, since both bundled encoders are English sentence encoders, and the label set is closed, so new intents need new examples and a refit. Reported accuracy is on this dataset's own test split, not a claim about your traffic.
Import once, warm it with one throwaway decision, and keep the engine alive. The head is fitted lazily on the first decide, and at probeIterations 4000 over 19,552 examples that fit takes minutes, so do not load a bank per request.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: Standard One SH (StandardThinking)
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsHacker News: Pointer to a DiffusionGemma plus vLLM tutorial for a Jev-shaped classifier.
1 pointHow to build a Jev-style classifier with DiffusionGemma and vLLMRead the thread on Hacker NewsMore Jev resources like this
v-jev 1.4 is a Jev-style decision model on Hugging Face in 0.8B and 4B sizes, built on Qwen3.5 backbones. It returns probabilities for typed questions in one forward pass and adds value extraction and image decisions.
Jev Models & Reimplementations#open-models#classification#calibration
Community
A 7.7M-parameter decision model on Hugging Face that reads text or speech and returns one probability per option for choice, score, and noul questions. Speech runs through femto-asr, giving a 10.1M-parameter spoken pipeline.
Jev Models & Reimplementations#open-models#classification#local
Community
Fine-tuned UltraBERT-380M decision model on Hugging Face that scores a state, question, and option list in one forward pass. Uses sigmoid of yes minus no normalized over options; 72.8% dev and 71.9% test accuracy across 13 datasets, with an evidence head documented on the card.
Jev Models & Reimplementations#open-models#arxiv#calibration
Community
A Hugging Face checkpoint that fine-tunes UltraBERT-380M into a Jev-style decision model: it reads a state, a question and a list of options and picks one in a single forward pass. Reports 72.6% mean accuracy over 13 datasets on the test split.
Jev Models & Reimplementations#open-models#classification#local
Community
Fine-tuned UltraBERT-380M decision model on Hugging Face that reads a state, question, and option list and picks one option in a single forward pass. Scores options with a square-root softplus of yes minus no, normalized over options; 73.2% dev and 71.9% test accuracy over 13 datasets.
Jev Models & Reimplementations#open-models#arxiv#calibration
Community
A 46M-parameter Russian and English decision model published on Hugging Face, tagged for classification, NLI, toxicity, moderation, and calibrated distillation. It ships a custom NQ4ForDecision class and a chat template with an optional thinking block.
Jev Models & Reimplementations#open-models#classification#calibration
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.