ekzhang/openjev-sglangArchived
User repository on GitHub
Jev-compatible API endpoint based on open models (prefill-only)
- Forks
- 46
- Last pushed
- Updated Oct 2026
Repo stats from the GitHub API, cached Oct 2026.Homepage
Jev directory
CommunityJev Repos & SDKs
Openjev Sglang
A server that implements the TypeSafe/Jev HTTP API with Qwen3.6-35B-A3B on SGLang 0.5.19, deployed on Modal with radix caching and breakable prefill CUDA graphs, plus a separate FastAPI process.
- Stars
- 333
- Language
- Python
- Reading time
- 3 min
- Published by
- Community
- Author
- ekzhang
- Last verified
- Sep 2026
About this project
A server implementing the TypeSafe/Jev HTTP API with Qwen3.6-35B-A3B on SGLang. Each container one B200 with SGLang 0.5.19's Rust frontend, radix caching, and breakable prefill CUDA graphs. A separate Python API process uses FastAPI, uvloop, the Rust-backed HF tokenizer, and pooled asynchronous HTTP connections to SGLang on localhost. CUDA dependencies stay in SGLang's container; uv sync on your laptop installs only the API, deployment tools, and tests.
Quoted from the project's README. Read the full README.
Works forJev for Classification
Used inJev for Software & DevTools
Highlights
- Each container runs one B200 with SGLang 0.5.19's Rust frontend, radix caching, and breakable prefill CUDA graphs.
- CUDA dependencies stay in SGLang's container; uv sync on a laptop installs only the API, deployment tools, and tests.
- Deploys to Modal as an unauthenticated Server that autoscales with no container cap and scales to zero after five idle minutes.
- A smoke command covers all three answer types, a 64-answer question, and rejection of 65 answers, reporting startup wait, latency, and cache usage.
- Probabilities are conditioned on the supplied options and label ordering, and confidence is 1 - H(p)/log(number_of_options), clamped to [0, 1].
Quickstart
bash
uv sync
uv run modal setup
uv run modal run modal_app.py
uv run modal deploy modal_app.py
uv run openjev smoke https://YOUR-SERVER.us-west.modal.directWatch out
No license file, so reuse terms are unclear. Needs Modal and a B200 for deployment (or a compatible SGLang backend with matching model revision), and the first GPU start downloads weights and captures CUDA graphs.
Questions about this entry
Short answers drawn from the linked project or page.
The README says it was an early experiment to reproduce Jev and that SGLang now ships a native decisions endpoint implemented the same way, pointing readers at the SGLang docs. That note is the first thing in the file, so treat this as a reference implementation rather than the recommended route.
It implements the /v1/systemone HTTP API, with jev-latest kept as a compatibility alias for the configured Qwen model and the public id set to Qwen/Qwen3.6-35B-A3B. Limits are tighter than the hosted service though: 64 questions per call, 2 to 64 answers per question, 32,768 tokens per branch and 16 concurrent evaluations. Chat state is text only, with no image or audio content.
Each container runs one B200. Autoscaling has no container cap and scales to zero after five idle minutes, so a scaled-to-zero server returns 503 while it starts and the smoke command retries. The first build pulls a large SGLang image, and the first GPU start downloads weights and captures CUDA graphs; later starts reuse the volume but still recapture graphs.
Yes. The Modal Server is deployed with unauthenticated=true, so the printed modal.direct URL is reachable without credentials. SGLang's generation and administration routes stay on localhost and are not forwarded, but the evaluation API itself is the public surface. The repository also has no license file, so reuse terms are unclear.
Defaults are 64 questions, 2 to 64 answers per Choice or Score, 2 MiB JSON, 32,768 tokens per branch including output, 262,144 total submitted input tokens, 16 simultaneous evaluations and 64 simultaneous backend calls. Invalid requests get 422, oversized bodies 413, overload 529 with Retry-After, and backend timeouts 504.
It deploys as an unauthenticated Modal Server with routing_region us-west, no explicit container cap, and scales to zero after five idle minutes. A scaled-to-zero Server returns 503 while it starts, so the included smoke command retries startup responses.
The README says yes. This was an early experiment to reproduce Jev after it came out, and SGLang now ships a native decisions endpoint implemented the same way, so the new API should be used instead.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: A Jev-shaped model on Cerebras and Qwen
X: Jev 1.14 is now on OpenRouter
X: Standard One SH (StandardThinking)
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
More Jev resources like this
An Apache-licensed System One decision server that reads Choice, Score, and Noul probabilities from DiffusionGemma 26B-A4B through vLLM on NVIDIA or MLX on Apple silicon, so TypeSafe SDKs work unchanged.
Jev Repos & SDKs#community#python#open-models
Community
razorback16/openjevPython
razorback16/openjevPythonA local server and Python package that runs open models with Hugging Face Transformers or PyTorch and returns Choice, Score, and Noul answers read from next-token logits instead of generated JSON.
Jev Repos & SDKs#community#python#open-models
Community
featherless-ai/simple-jevPython
featherless-ai/simple-jevPythonA Python library that turns Jev-style decision models into ordinary function calls: check returns a bool, classify picks one option, label picks every option that applies, and score rates on a scale, with probabilities behind every answer.
Jev Repos & SDKs#community#github#python
Community
jlowin/vibecheckPython
jlowin/vibecheckPythonAn independent local alternative to Jev: it takes English or Chinese context, questions, and candidate answers, and returns probabilities over the supplied candidates from a frozen Qwen3-4B backend.
Jev Repos & SDKs#community#python#open-models
Community
zhihz/openjevPython
zhihz/openjevPythonA Python decorator that turns a Pydantic model into Jev questions: bool becomes a Noul, a Literal becomes a Choice, and a bounded int becomes a Score, with answers validated back into the model.
Jev Repos & SDKs#community#python#pydantic
Community
aaazzam/jevPython
aaazzam/jevPythonA TypeScript-first toolkit billed as the standard library for System One models: ready-made classify, extract, pickOne, rank and verify methods, a service-agent runtime that runs conversations turn by turn, and a one-message order parser. A Python port is in preview and the project is in alpha.
Jev Repos & SDKs#community#github#typescript
Community
Engid/question-kitTypeScript
Engid/question-kitTypeScriptFrom the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.