Organization repository on GitHub
Native vLLM serving for Jev decision models
- Forks
- 11
- Last pushed
- Updated Oct 2026
Repo stats from the GitHub API, cached Oct 2026.
CommunityJev Integrations & Adapters
Vllm Jev
Serves Jev-style decision checkpoints through vLLM on Linux, with an MLX preview for Apple Silicon, exposing Choice, Noul and Score answers over HTTP.
- Stars
- 430
- Language
- Python
- License
- APACHE-2.0
- Published by
- Community
- Author
- mode-io
- Last verified
- Oct 2026
About this project
Fast structured decisions on Linux and Apple Silicon. Documentation · Demos · Getting Started · Supported Models · Example · Updates. The same Valen checkpoint answers 24 structured questions about one video, using eight sampled frames and one A800 per runtime. In this live run, Valen's PyTorch reference path took 5.31 s, while vLLM Jev took 1.22 s (4.3× faster); all 24 selected answers agreed. Model loading is excluded. Watch the MP4.
Quoted from the project's README. Read the full README.
Works forJev for Classification
Used inJev for Software & DevTools
- Languages
- Python
Highlights
- One serve command with a Hugging Face id selects and verifies the native protocol, then starts on a local port.
- Linux gets native vLLM scheduling, batching, compilation, KV cache and metrics; Mac uses MLX or PyTorch MPS.
- Decision readouts include scalar candidate branches, marker scores and Laya's trained decision head.
- Choice, Noul and Score are all structured HTTP decisions; Valen also accepts short video on Linux and Mac.
Watch out
The Apple Silicon path is labelled a preview in the README, and the extra decision adapters are experimental: JevK5 is limited to 2-16 options per question, and on concurrent Linux batches Kev-0.8B has been seen changing its selected label while This-That and Decider-2B change the top Score level. The plugin also pins vLLM 0.29.0 and Python 3.12+. Copy reviewed against the repository README on 2026-10-04, not from running it.
Questions about this entry
Short answers drawn from the linked project or page.
It is a preview, and the README is explicit about the split: native vLLM on Linux with an NVIDIA GPU, MLX for supported text models and Valen multimodal on Apple Silicon, and PyTorch MPS only for Laya. Setup is a one-time shell script, then the same serve command with a Hugging Face model id.
The README treats that as a real risk, not a rounding detail. On concurrent Linux batches Kev-0.8B can change the label it selects, and This-That and Decider-2B can change the most likely Score level. Those consistency limits are listed as still under investigation.
The table covers Open-Jev 2B and 9B, three Laya variants, OpenJev-0.6B, Tiny-Jev, Valen for text, images and short video, two vjev-vision checkpoints, then experimental Kev, Decider, Mica, This-That, JevK5 and RSI-Jev entries. Anything outside those tables is not claimed as supported.
No. The README states that serving does not train; setup only downloads and prepares the checkpoint and verifies the native protocol. Model loading is also excluded from the demo timings, so the published speed comparisons are inference only.
On Linux with an NVIDIA GPU, uv pip install . from the repository, then vllm-jev serve with a supported Hugging Face model ID such as ZefanCai/Open-Jev-2B. It downloads and prepares the model, then starts native vLLM at http://127.0.0.1:8795, and later runs reuse the prepared checkpoint. Standard vllm serve flags like --port still apply.
On a live video-understanding run, the same Valen checkpoint answered 24 structured questions about one video using eight sampled frames and one A800 per runtime: the PyTorch reference path took 5.31 seconds while vLLM Jev took 1.22 seconds, 4.3 times faster, and all 24 selected answers agreed. Model loading is excluded from those timings.
Valen, vjev, RSI-Jev v4.0-VL, and Clef accept text and images, and Valen and Clef also accept short videos; platform availability and request formats are model-specific. On Apple Silicon, Valen multimodal decisions run through MLX, and Laya through PyTorch MPS.
More Jev resources like this
1.1kGitHub stars
A section-by-section guide to agent harnesses - loop, tool runtime, permissions, hooks, subagents, skills, memory, background work - using Claude Code, mini-swe-agent, Hermes Agent, and deepseek-harness as worked examples.
Jev Integrations & Adapters#community#jevbest
691GitHub stars
A Python speech-to-speech avatar framework: swappable VAD, STT, LLM, and TTS modules behind one streaming pipeline, with WebSocket, telephony, LINE, and OpenAI-compatible channel adapters on top.
Jev Integrations & Adapters#community#jevbest
Community
uezo/aiavatarkitPython
uezo/aiavatarkitPython61GitHub stars
The model-integration layer for the Milvus vector database, shipping adapters for embedding and reranker models from hosted providers, Jev among them, plus local backends.
Jev Integrations & Adapters#community#jevbest
Community
milvus-io/milvus-modelPython
milvus-io/milvus-modelPython41GitHub stars
Typed Python planner runtime that turns batched Jev action distributions into executable probability trees and graphs for finite decision tasks.
Jev Integrations & Adapters#community#jevbest#catalog-import
Community
Chuf-H/jev-treePython
Chuf-H/jev-treePython25GitHub stars
Agent runtime with a terminal UI and a messaging gateway that keeps memory, self-built skills, session search, scheduling, and subagents across Telegram, Discord, Slack, WhatsApp, Signal, and CLI.
Jev Integrations & Adapters#community#jevbest#catalog-import
Community
wundercorp/lokiPython
wundercorp/lokiPython51GitHub stars
GraphRAG layer that runs a 4-phase ingest-to-answer pipeline over Neo4j, Memgraph, AGE or Kuzu using Laya or Jev decisions instead of per-hop LLM calls.
Jev Integrations & Adapters#community#jevbest#catalog-import
From the Jev community
Posts from builders shipping with Jev right now.