Skip to content
JevDirectory.org
CommunityRepos & SDKs263 starsVerified 2026-09-22

openjev-sglang

A server that implements the TypeSafe/Jev HTTP API with Qwen3.6-35B-A3B on SGLang 0.5.19, deployed on Modal with radix caching and breakable prefill CUDA graphs, plus a separate FastAPI process.

Open github.comekzhang/openjev-sglang
Category
Repos & SDKs
Published by
Community
Author
ekzhang
Added
2026-09-22
Tagscommunitypythonopen-modelssdkgateway

Highlights

  • Each container runs one B200 with SGLang 0.5.19's Rust frontend, radix caching, and breakable prefill CUDA graphs.
  • CUDA dependencies stay in SGLang's container; uv sync on a laptop installs only the API, deployment tools, and tests.
  • Deploys to Modal as an unauthenticated Server that autoscales with no container cap and scales to zero after five idle minutes.
  • A smoke command covers all three answer types, a 64-answer question, and rejection of 65 answers, reporting startup wait, latency, and cache usage.
  • Probabilities are conditioned on the supplied options and label ordering, and confidence is 1 - H(p)/log(number_of_options), clamped to [0, 1].

Quickstart

bash
uv sync
uv run modal setup
uv run modal run modal_app.py
uv run modal deploy modal_app.py
uv run openjev smoke https://YOUR-SERVER.us-west.modal.direct

Watch out

No license file, so reuse terms are unclear. Needs Modal and a B200 for deployment (or a compatible SGLang backend with matching model revision), and the first GPU start downloads weights and captures CUDA graphs.

More like this

OpenJev
300GitHub stars
An Apache-licensed System One decision server that reads Choice, Score, and Noul probabilities from DiffusionGemma 26B-A4B through vLLM on NVIDIA or MLX on Apple silicon, so TypeSafe SDKs work unchanged.
Repos & SDKs#community#python#open-models
Communityopenjev
Simple Jev Project
475GitHub stars
A local server and Python package that runs open models with Hugging Face Transformers or PyTorch and returns Choice, Score, and Noul answers read from next-token logits instead of generated JSON.
Repos & SDKs#community#python#open-models
Communitysimple-jev
Laya
14.3kGitHub stars
Open multilingual System 1 decision models with published checkpoints for choice, score and noul questions, plus a router that dispatches each request to the right checkpoint in one forward pass.
Repos & SDKs#community#python#open-models
Communitylaya
Back to all resources

From the community

Posts from builders shipping with Jev right now.

Follow @typesafeai

Fast browser use with Stagehand

Kyle Jeong
Kyle Jeong
Browserbase
@kylejeong
·

we built blazing fast computer/browser use with Jev + @Stagehanddev. this task cost $0.001 and executed at near instant speed (in a remote browser btw) the loop: observe the page, send a11y tree as state + actions as questions, Jev decides the next action, then Stagehand Show more

Watch on X
Diogo Almeida
Diogo Almeida
TypeSafe AI
@CompleteSkeptic

After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI? I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev • 20-200x faster • 40-400x

Watch on X
Reply
Read 30 replies

LLM-as-a-judge, sped up

Yuchen Jin
Yuchen Jin
Databricks
@Yuchenj_UW
·

Jev has spoken. It picked which model is AGI. 20–200x faster. 40–400x cheaper. This could make things like LLM-as-a-judge insanely fast and nearly free. (I tried a bunch of prompts and still didn’t burn through $0.10.)

Image
Diogo Almeida
Diogo Almeida
TypeSafe AI
@CompleteSkeptic

After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI? I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev • 20-200x faster • 40-400x

Watch on X
Reply
Read 94 replies

Instant compaction with Jev

A Claude session from 1M to 86K tokens

Alex Volkov
Alex Volkov
CoreWeave
@altryne
·

This is actually insane. This uses @typesafeai Jev model, as a plugin in Claude to review all the un-nesseasary tool calls, and it takes 1s to run! Like, literally, 1 second to take my Claude session from nearly 1M to ... 86K tokens! 😮 Ask your claude to install it and be  Show more

Image
Image
tamara
tamara
@tamarajtran

found the perfect use case for @typesafeai Jev: instant compaction in 2026, why is compaction still a summarization prompt? Jev can make it instant by scoring every tool call and dropping what’s irrelevant

Watch on X
Reply
Read 164 replies

Vercel's fx safety reviewer, 18x faster

Guillermo Rauch
Guillermo Rauch
Vercel
@rauchg
·

We're seeing extraordinary results from @typesafeai. Default mode in 𝚏𝚡 is auto, with a safety reviewer analyzing every command. That reviewer runs on GPT Luna today. Jev is up to 18x faster (p95) *and* more accurate. It's coming to @vercel AI Gateway and likely new default.

Pranit
Pranit
Vercel
@fazxes

We benchmarked fx auto mode (safety) classifier with @typesafeai's Jev. tl;dr: ~5-18x faster and more accurate than 𝚐𝚙𝚝-𝟻.𝟼-𝚕𝚞𝚗𝚊, our current top choice

Image
Reply
Read 137 replies

Jev lands on OpenRouter