Skip to content

Decide, Don't Generate: Competitive Dimensional ABSA with Jev's Typed Decisions (arXiv 2609.35293)

A system that decomposes all three tasks of SemEval-2026 Task III Track A into typed Jev decisions aligned with 488 CPU-fitted coefficients. It reaches 1.0645 RMSE on valence-arousal regression and 52.09/44.06 continuous F1 on extraction, above fine-tuned Llama-3.3-70B and GPT-OSS-120B baselines.

Stars
7
Language
Python
Format
Article
Published by
Community
Last verified
Oct 2026

TypeSafe Jev (System One) on DimABSA, SemEval-2026 Task 3 Track A, all three subtasks: V/A regression (lowest 10-corpus micro RMSE among full-coverage teams), triplet extraction, quadruplet extraction. Pure-Jev inference with BM25 examples and train-fitted calibration.

Forks
0
Last pushed
Updated Sep 2026

Repo stats from the GitHub API, cached Oct 2026.

Highlights

  • Decomposes all three tasks of SemEval-2026 Task III Track A into typed Jev decisions.
  • Aligns decisions with the annotation scheme through 488 coefficients fitted on CPU, with no backbone tuning.
  • Reaches 1.0645 RMSE on valence-arousal regression over ten corpora in six languages, the lowest of participating systems.
  • Triplet and quadruplet extraction reach 52.09 and 44.06 continuous F1.
  • Supervised calibration roughly halves the raw regression error in the ablations.

Watch out

arXiv preprint posted September 28, 2026 (cs.CL) by Yiqun Zhang, Peidong Wang, Zihan Wang, and Shi Feng; 14 pages. The headline numbers against the Llama-3.3-70B and GPT-OSS-120B baselines come from the authors' own evaluation of the shared task, and roughly half the regression gain comes from supervised calibration fitted on task data.

Questions about this entry

Short answers drawn from the linked project or page.

More Jev resources like this

A training-free framework with two implementations that turn emotion classification into Jev judgments: one decomposes tasks into atomic judgments, the other aggregates complementary judgment paths. Across eight datasets standard Jev averages 62.93% macro-F1 versus 67.28% for the strongest LLM baseline.
Jev Practices & PatternsArticle#community#research#arxiv
Community
JevCascade allocates inference-time compute for prefill-only decision models using readout stability: when only the candidate menu changes, post-intervention accuracy is already determined by the cached first-pass distribution. Menu-only interventions are predicted to within 4.2 points across six model families.
Jev Practices & PatternsArticle#community#research#arxiv
Community
An analysis of ordinal-scale bias in JEV 1.13 and three open KEV models: across 36 ordinal datasets, final decisions use only 67-76% of the effective gold support versus 87-102% on four nominal tasks. BA-LoRA post-training raises utilization from about 47% to 86%.
Jev Practices & PatternsArticle#community#research#arxiv
PACT turns contrastive training pairs into four loss terms for single-token typed-decision models. On a frozen 324-item holdout it matches the published recipe's accuracy (84.6% versus 85.2%) while cutting answer flips under relabelling from 13.8% to 9.8%.
Jev Practices & PatternsArticle#community#research#arxiv
A system that codes Texas crash narratives as gated, typed Jev decisions returning probabilities and no text: a screen covered 499,500 narratives and 195,857 were coded with a 27-question schema. The typed model attains F1 of 0.908 against human labels, and recalibration cuts calibration error 3.3-fold.
Jev Practices & PatternsArticle#community#research#arxiv
Community
A theoretical study of recalibrating many related yes/no questions that each have few labels, motivated by decision models such as Jev. It derives minimax compound-risk rates and shows pooled isotonic fitting with one per-question correction achieves them, while tuning questions separately loses a factor.
Jev Practices & PatternsArticle#community#research#arxiv
Community

From the Jev guides

Original write-ups on what this entry is for.

More than eighty preprints now evaluate, attack, and reproduce the typed-decision class. The broadest board, the coherence gap, the rejection bottleneck, and the attacks that still work.
Jev Practices & Patterns#research#evaluation#calibration
Read article
TypeSafe publishes evals, but the numbers that matter are the ones measured on your task. A tour of the independent benchmarks, field reports, and the recurring finding that question design dominates.
Jev Practices & Patterns#benchmarks#evaluation#calibration
Read article
A Choice can only pick from its options. How rejection options, candidate coverage, and escalation bands keep a narrow taxonomy honest under inputs it never anticipated.
Jev Practices & Patterns#rejection#confidence#patterns
Read article
Back to all resources

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai