Can I trust the probabilities without doing anything first?

No. The calibration section says these models ship over-confident and quotes Laya's model card at mean ECE 0.466 before calibration versus 0.081 after. Collect around fifty labelled examples, run jev calibrate to fit one temperature per question type and option count, then jev eval to see calibrated accuracy, soft accuracy, Brier score and ECE before wiring any gate to them.

Will this beat an LLM classifier on a fifty-label taxonomy?

Probably not. The README states that more than about twenty options per choice degrades quality, that high-cardinality classification in the Banking77 style is weak, and that you should keep option sets small. It also says base checkpoints are near chance on unseen tasks, and to evaluate on your own labelled data first rather than trusting any published number.

Do I need a TypeSafe account and a paid key?

No. The mock provider is deterministic and needs nothing, laya is an open-weights local engine installed with jev setup laya, and the http provider talks to any server exposing POST /v1/systemone. Auto-selection tries the mock, then a base URL, then an API key, then an installed laya, and the README says self-hosted beats the paid API on purpose.

What is the practical difference between the mock provider and laya?

Mock is a hash-seeded fake for wiring and pipeline tests and carries no real signal. Laya is the actual open-weights engine, roughly a 1.7GB download on first use with a configurable checkpoint, and it is the one you evaluate and calibrate. The openjev weights are non-commercially licensed, are never shipped, and are for local experiments only.

How do I install and verify it?

Clone the repo into ~/pave-apps/openpave-jev and run pave install, then jev providers and jev decide --provider mock with a sample state and choice list. A decision model returns a distribution over a fixed answer space in about 35 to 500 ms.

What commands and exit codes does the CLI have?

jev decide, score, noul, ask, eval, calibrate, providers and setup laya. Exit codes are 0 for ok, 1 error, 2 usage and 3 when the answer falls below --min-confidence, so a script can branch on uncertainty.

How does state get passed in?

Through --state with literal text, an @file or a pipe, or --state-json with JSON or a file, or piped stdin. --file accepts a literal JSON string, an @path, a pipe or a bare path, and --no-calibrate bypasses the saved calibration for one call.

CommunityJev Practices & Patterns

Openpave Jev

A PAVE app and CLI for typed decisions (choice, score and noul) with calibrated probabilities, four providers including a local engine, and eval and calibrate commands.

See highlights

Open github.com

Stars: 2
Language: JavaScript
License: MIT
Published by: Community
Author: cnrai
Last verified: Oct 2026

cnrai/openpave-jev

Organization repository on GitHub

View on GitHub

PAVE skill for non-autoregressive decision models (Jev / Laya / TypeSafe): typed Choice, Score and Noul answers with calibrated probabilities

Forks: 1
Last pushed: Updated Oct 2026

Repo stats from the GitHub API, cached Oct 2026.

About this project

Ask non-autoregressive decision models for typed answers with calibrated probabilities: classify, route, score and gate without an LLM round-trip. A decision model ("System One" architecture, Jev-style) doesn't generate text. You hand it a state and typed questions, and it returns a distribution over a fixed answer space — in 35–500 ms, at 1000x lower cost than an LLM call, with probabilities you can gate on.

Quoted from the project's README. Read the full README.

Tagscommunity awesomejev catalog-import

Works forJev for Classification Jev for Ranking Jev for Tool calling

Used inJev for Software & DevTools

Languages: JavaScript

Highlights

All three question types carry confidence, and --min-confidence exits with code 3 so scripts can branch on uncertainty.
Providers run from a hash-seeded mock to a local LayA engine, any /v1/systemone server, and the paid TypeSafe API.
jev eval reports accuracy, soft accuracy, Brier and ECE; jev calibrate fits one temperature per question type and option count.
Auto-selection prefers self-hosted over the paid API: JEV_MOCK, then JEV_BASE_URL, then JEV_API_KEY, then installed laya.

Watch out

The probabilities ship over-confident, with Laya's model card reporting mean ECE 0.466 before calibration against 0.081 after, so the README says to collect fifty-odd labelled examples and calibrate before trusting them, with calibration auto-applying on every call unless you pass --no-calibrate. Base checkpoints are near chance on unseen tasks, with the Laya typed-decisions checkpoint at 0.362 accuracy zero-shot. Choice sets beyond about twenty options degrade quality and high-cardinality classification is weak. The speed and cost framing in the README is the project's own rather than a comparison it measured. Copy reviewed against the repository README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

More Jev resources like this

hermes-and-jev-play-minecraft

Minecraft setup where Hermes plans at milestones, Jev picks one bounded harness action per step, and Mineflayer code owns validity and execution with no screenshots.