OpenJev — zhangcy122

Single-prompt option lists can flip when order changes. With order_invariant enabled each candidate is scored in an isolated pass and normalized with a commutative softmax across all orderings.

Standard high-confidence queries use the fast local path while borderline or out-of-distribution queries escalate to a reasoning model, then the boundary and precedent are stored for later fast-path hits.

Point at local vLLM, SGLang, Ollama, or a Laya ModernBERT endpoint for fully offline runs, or use the hybrid gateway to keep frequent queries local and escalate edge cases to cloud APIs behind a circuit breaker.

Install with pip install openjevpro, try python3 -m openjevpro.demo with no GPU, and use mock mode for offline development and CI before pointing at a real inference endpoint.

With order_invariant enabled each candidate is scored in an isolated forward pass without competitor tokens in context, then normalized with a strictly commutative softmax, which the README states gives 100% permutation invariance across all orderings with deltaP under 1e-4. For single-pass zero-drift execution in under 35 ms it points at the dedicated Laya ModernBERT-large 322M encoder, which is natively bidirectional.

The README frames full generative LLM routing as three liabilities: 800 ms to 2,500 ms per routing step burning output tokens and dollars, random JSON formatting glitches and markdown wrap failures, and severe option order bias where permuting candidates flips classification by 15% to 20% in autoregressive models, which fails compliance and auditability. It pitches typed, calibrated, order-invariant primitives instead.

HybridJevGateway routes 70% or more of high-frequency intent queries to local Tier 1 models under 20 ms at zero cloud cost and escalates only edge cases, which the README says cuts monthly bills by 60% or more. A CircuitBreaker with a failure threshold of 3 and a 30 s recovery time catches cloud timeouts and HTTP 429s and degrades to calibrated local decisions instead of raising unhandled 500s.