How big is the option-order problem this actually fixes?

Reordering the same options can shift probabilities by many points. Pijev evaluates several orderings, aligns by label, and returns the mean with the top mean as the choice.

Are the better-calibration claims proven or just empirical?

For losses convex in predicted probabilities, Jensen's inequality bounds the averaged loss by the average of individual losses, covering Brier score and log loss on the included predictions.

Does it change Noul and Score answers or confidence semantics?

Only Choice answers are averaged; Noul and Score pass through unchanged, and the returned confidence is the selected label's mean probability rather than the native score.

How much setup and extra cost does the wrapper add?

Change only the import, keep official response types and question names, and set auto, all, or a seeded integer budget; async clients work the same way.

How do I adopt it, and what changes in my code?

pip install pijev, export TYPESAFE_API_KEY, and change only the import from typesafe_sdk to pijev for TypeSafeClient, Choice, Noul and Score. Async works the same way by importing AsyncTypeSafeClient and using async with and await client.system_one(...).

How many orderings does it use by default, and can I control that?

n_permutations defaults to auto, a balanced set where every option appears in every position and after every other option equally often: K orderings for K options, or 2K when K is odd. An integer samples that many distinct orderings, all of them when there are fewer, all uses every ordering, and seed fixes the sample.

What are the request limits and shape?

All permutations of a Choice run in one system_one request, so the state is sent once and the payload is roughly S+MQ rather than M(S+Q) for separate calls. A request holds at most 720 expanded questions and is subject to the server's limits; Noul and Score each add one unchanged question and are not averaged.

CommunityJev Repos & SDKs

Pijev — TypeLLM

Drop-in Jev client wrapper that averages Choice predictions across option orderings in one request for stabler probabilities.

See highlights

Open github.com

Stars: 34
Language: Python
License: APACHE-2.0
Published by: Community
Author: TypeLLM
Last verified: Oct 2026

TypeLLM/pijev

Organization repository on GitHub

View on GitHub

Permutation Invariant Jev

Forks: 1
Last pushed: Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.

About this project

Better Jev probabilities with theoretical guarantees—all with a one-line import change. pijev turns Jev's option-order variations into a single averaged prediction, with Brier score and log loss guaranteed no worse than the average across the included orderings. All permutations run in one systemone request. Keep the same calls and official response types; pijev handles the rest.

Quoted from the project's README. Read the full README.

Tagscommunity jevbest catalog-import

Works forJev for Classification

Used inJev for Software & DevTools

Languages: Python

Highlights

One-line import swap keeps the same calls and official response types with original question names.
Balanced default ordering set with options for all permutations or a seeded random sample under request limits.
Averaging guarantee covers Brier and log loss against picking one included ordering, with strict Brier gains on disagreement.
Single-request batching sends state once with payload estimates, usage-based costing, and async support.

Watch out

Third-party extension, not an official SDK; multi-ordering requests still count as multiple decisions for backend limits, latency, and billing. Copy reviewed against the repository README on 2026-10-04, not from running it.

Questions about this entry

Short answers drawn from the linked project or page.

Other listings of this project

Forks and copies of this project in the directory. This entry is the canonical one.

pijev

Terminal coding agent on Pi where Jev shortlists skills, ranks retrieved excerpts, and triages tool failures while the coding model handles reasoning and edits.

Jev Agents & MCP#community#awesomejev#catalog-import

Community

tonyzdev/pijevTypeScript

More Jev resources like this

decido

A Python layer over a small decision-provider contract that adds validated distributions, batching, budgets, ranking and exportable records, with Jev as the first provider.