Mini-Jev — abirshhishirroy

State and candidates are concatenated into one sequence, passed through the frozen Qwen2.5-3B backbone, then a boolean mask mean-pools hidden states only over each candidate's tokens. A 2-layer MLP maps each pooled vector to a scalar logit, and a grouped softmax over candidates yields the final probabilities.

The README states the 3B parameter model fits on consumer GPUs, so no OpenAI or Anthropic API calls are required. Cost per 1,000 inferences is listed as $0.00 local, against roughly $0.30 for GPT-4o-mini.

No. The full dataset is excluded to keep the repository lightweight. The exact line format is shown in data/samples/train_sample.jsonl, with each line holding a task, state, candidate list, and correct_index.

The project is MIT licensed and targets Python 3.12 with PyTorch 2.4 and HuggingFace Transformers. The backbone is Qwen2.5-3B-Instruct, kept frozen while the decision head and LoRA adapters are trained.

The custom dataset spans 7 tasks, including safety gating, hallucination validation, context filtering, query routing, and tool routing. Reported per-task accuracy includes 97.32% on safety gating, 100.00% on query routing, and 76.67% on hallucination validation.

General tool routing, where it scores 82.33% against GPT-4o-mini's 89.75%. The author attributes this to the larger model's broad semantic understanding, while Mini-Jev leads on structured validation, strict rule-following, and safety tasks.

It serves as the decision engine for a CLI coding agent that writes Python unit tests, routing between read_file, grep_search, and run_bash while gating dangerous shell commands. It also filters retrieved chunks and validates faithfulness in an end-to-end RAG pipeline.