Article
CommunityJev Benchmarks & Evaluations
Reproducible LoRA Inference Through Deterministic Split-K Reduction
arXiv paper examining numerical nondeterminism in vLLM Split-K LoRA kernels and introducing a deterministic Triton Split-K implementation, measured on GSM8K, MBPP, and Jev-style structured toxicity decisions over Civil Comments.
- Format
- Article
- Reading time
- 2 min
- Published by
- Community
- Added
- Nov 2026
Highlights
- Atomic Split-K=8 produced different FP32 shrink outputs in 97.61% of identical-input comparisons.
- The deterministic Triton Split-K=8 showed no observed variation under identical-input replay.
- Repeated atomic inference changed outputs in 17.30% of GSM8K and 14.96% of MBPP comparisons.
- Structured-decision probabilities varied in 81.47% of atomic comparisons, including decision flips.
- Deterministic Split-K=8 kept 16.8-21.1% speedup over unsplit reduction at 1.02-2.62% added latency versus atomic.
Questions about this entry
Short answers drawn from the linked project or page.
It studies numerical nondeterminism introduced by Split-K kernels used to parallelize low-rank LoRA multiplications in vLLM, where partial results are combined with floating-point atomic additions, and proposes a deterministic Triton Split-K implementation.
Under identical-input replay, Atomic Split-K=8 produced different FP32 shrink outputs in 97.61% of comparisons, while the Deterministic Split-K=8 variant showed no observed variation.
The work covers GSM8K mathematical reasoning, MBPP code generation, and Jev-style structured toxicity decisions on Civil Comments.
Structured-decision probabilities varied in 81.47% of atomic comparisons, and the variation included decision flips rather than only small numeric drift.
The deterministic Triton Split-K=8 preserves parallelism, achieving 16.8-21.1% latency-based speedup over unsplit reduction with only 1.02-2.62% additional latency relative to Atomic Split-K=8.
Controlled tracing shows shrink-output differences propagating into the KV cache, and accumulated cache differences can change token decisions and final answers.
No. The abstract describes a deterministic Triton Split-K implementation and its measurements, but it does not state a code repository, license, or maintaining organization.
More Jev resources like this
Evaluation of MedJev, an open System One model that runs locally on a laptop, across ten medical benchmarks for knowledge, diagnosis, and patient-level prediction on MIMIC IV and MIMIC IV ED, with a two-parameter recalibration method and a released evaluation harness.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
Open, deterministic audit benchmark for System 1 decision models, covering calibration under label-preserving perturbations, cross-question interference, and post-hoc attribution faithfulness across five open-weight models and generative baselines.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
Multilingual benchmark of nine suites in 122 languages that evaluates typed decision models such as Jev, Clef, and pplx-decider, and tests whether confidence thresholds fitted on English transfer to other languages.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#benchmark
Community
arXiv paper evaluating Jev as a safety guardrail for a Qwen2.5-1.5B-based pipeline across 970 benchmark prompts, comparing an unfiltered LLM, Jev input filtering, and tiered input filtering with selective output review.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#guardrails
Community
arXiv paper comparing two releases of Ru-Jev, a 307M-parameter multilingual cross-encoder trained on thirteen Russian-language tasks, to test whether an augmentation that presents labels in several surface forms helps when callers supply their own option wording.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
Independent audit of a frozen System One model on 142,658 human-labeled comments, benchmarked against 14 frontier reasoning models. It reports AUROC 0.83, ~USD 0.000025 per call, ~100 ms, and separates human disagreement from decision risk.
Jev Benchmarks & EvaluationsArticle#arxiv#evaluation#calibration
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.