Question 1

Does schema-valid output mean the decisions are correct?

Accepted Answer

Yes for validity, no for correctness. All three tested sizes were always schema-valid through the constrained path while naive token-by-token JSON failed, but values can still be wrong and confidence above 0.90 still covered wrong fields.

Question 2

Is the 1.5B model enough or should I move to 7B?

Accepted Answer

On the same engine the small model matched the majority baseline while the larger ones scored clearly higher, at several times the latency. Use the small size for demos and UI work and the larger ones when the primary decision matters.

Question 3

Does the local model match real Jev accuracy head to head?

Accepted Answer

No. On the rebuilt public cases the free local 7B model agreed far less often than published Jev, trailing by about thirteen points. Its advantages are zero cost, privacy, and offline use, not parity.

Question 4

Is this Apple Silicon only, and will a 16 GB M5 Air handle the 7B and 8B runs?

Accepted Answer

Apple Silicon is the tested path. setup.sh builds the venv, installs mlx-lm, and clones the Qwen-2.5-1B-RLCD engine rather than vendoring it. All timings come from one 16 GB M5 MacBook Air, where the 7B and 8B 4-bit runs measured 1.52 s and 2.03 s on the 28-field fraud preset. Windows or NVIDIA means following the separate parallel-decisions GPU guide instead. The static Hugging Face showcase needs no install; the interactive Gradio Space needs a PRO account.

Question 5

How do I run a single decision without the full benchmark?

Accepted Answer

After ./setup.sh builds the virtualenv and clones the engine, run .venv/bin/python tools/demo.py for the fintech fraud preset, or add a preset name and model such as support_triage with mlx-community/Qwen2.5-7B-Instruct-4bit. It prints one value and confidence per field; the static Hugging Face showcase shows recorded outputs with no install at all.

Question 6

What can break the one-pass-for-all-fields claim?

Accepted Answer

Shared first tokens. The README rates one pass for all fields as the common case but notes that fields whose choices share a first token hit a slower fallback, and support-triage cost extra time for collision handling. Raw confidence is a softmax proxy, not trained calibration, and the top speedup figures cite separate warmed GPU workloads, not browser-click timings.

Question 7

What license covers the repo, and which parts are not redistributed?

Accepted Answer

The project's own code and docs are MIT. The decoding engine is cloned at setup from harshatheg/Qwen-2.5-1B-RLCD under Apache-2.0 and is not vendored in the repo, while Jev and the TypeSafe model are not included and the study states it has no affiliation with, or access to, TypeSafe's model.

Jev On A Laptop

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev guides

From the Jev community