Skip to content

Social Risk Qwen Jev

Chinese-language research project for multimodal social-risk moderation: a local Qwen3-VL-32B with language-side LoRA emits structured evidence JSON, and a fixed Jev policy turns that evidence into risk bands, with frozen test leaderboards and pure-Jev ablations.

Stars
6
Language
Python
License
MIT
Published by
Community
Last verified
Oct 2026

Highlights

  • A frozen test set of 170 items ranks 13 methods by Macro-F1; TF-IDF + Logistic Regression C=4 tops it at 90.36%.
  • At threshold 0.40, LoRA evidence + Jev reaches 79.74% Macro-F1 versus 63.04% for pure Jev text on the same set.
  • Qwen3-VL-32B runs 4-bit with language-attention LoRA rank 8; about 19.92M trainable parameters, vision frozen.
  • The offline demo and result recomputation run without API keys, network access or GPU.

Watch out

MIT licensed. Dataset images, per-item predictions and weights are not published; results come from a small frozen set and the README says they are not a SOTA claim.

Questions about this entry

Short answers drawn from the linked project or page.

Reactions & coverage

Posts, threads, and videos about this entry from around the web.

YouTube: Building a contextual brand-safety scorer on Jev from scratch.

LinkedIn: What multimodal decision models actually change, and what they do not.

LinkedInJevDirectory.org

X: A website slop detector that checks 35 tells in 243 ms

X: Breaking, golden nugget, or slop

Related terms

Glossary definitions related to this entry.

More Jev resources like this

Notebook benchmark that compares Jev with open models and specialised tools on five document-pipeline chores: language detection (Choice(12)), page orientation (Choice(4)), RVL-CDIP classification (Choice(16)), bundle splitting and parse-tier triage, with liteparse doing all PDF rendering and OCR work.
Jev Benchmarks & Evaluations#community#evaluation#document-processing
3GitHub stars
Evaluation scripts, MCP tools, and a skill router built to measure where jev-1.13.0 fits in agent harnesses, with raw results under eval/.
Jev Benchmarks & Evaluations#community#evaluation#agents
0GitHub stars
Independent pre-registered audit of Jev on Spanish: 19,200 calls over 3,200 paired items on XNLI, PAWS-X, MASSIVE and Belebele. A Spanish state costs 3.0-6.4 accuracy points and roughly doubles ECE on the two hardest tasks, while Spanish instructions change nothing; a CLI reruns the same comparison on your own data.
Jev Benchmarks & Evaluations#community#evaluation#calibration
60.9kGitHub stars
LiteLLM's proxy adds TypeSafe guardrail hooks and a Jev complexity classifier for policy judgments and request routing across 100+ model providers.
Jev Integrations & Adapters#community#python#gateway
Agentic search evaluation inside Zilliz DeepSearcher: a standalone harness replaces the ChainOfRAG reflection call with one Jev Noul per round to decide whether to stop within a seven-round budget, then compares stopping behavior, evidence recall and decision cost on 100 sampled 2WikiMultiHopQA queries.
Jev Benchmarks & Evaluations#community#rag#evaluation
4.9kGitHub stars
LangWatch Instant Evals ships a TypeSafe Jev classifier that batches Noul, Score, and Choice judgments on production traces in one POST per text.
Jev Benchmarks & Evaluations#community#typescript#evaluation
Back to all resources

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai