Skip to content
JevDirectory.orgTraffic count unavailable. View site traffic
Evaluation & TrainingTemperature fitting

Temperature scaling

Dividing a model's logits by one fitted number per setting to bring its probabilities closer to observed frequencies.

All 164 terms
Category
Evaluation & Training
Also known as
Temperature fitting
Related terms
5
Directory entries
67
Docs
—
Added
2026-10-10

Definition

Laya's card reports mean ECE falling from 0.466 to 0.081 on the English checkpoint and 0.314 to 0.106 on the multilingual build after refitting one temperature per question type and option count, and warns that the shipped builds arrive over-confident.

The fit belongs to the data it was fitted on. Mapika's readout uses a temperature as a function of option count, T(n) = max(0.05, 10.124 − 1.633 ln n), and GEV-26B ships separate calibration files for scoring and for gating, with a noul temperature of 1.214 in the gold file used for automatic actions.

Tagscalibrationevaluation

Related terms

Definitions that connect to this one.

Platt scalingA two-parameter sigmoid fit on labeled data that maps a model's raw probability onto a better-calibrated one.Evaluation & TrainingIsotonic regressionA non-decreasing fit from raw confidence to observed accuracy. It can follow a miscalibration that is not a sigmoid.Evaluation & TrainingExpected calibration errorThe average gap between predicted confidence and observed accuracy, grouped into bins. Lower is better, and the number depends on the bins.Evaluation & TrainingBrier scoreA score for a full probability vector against one observed label. Lower is better, and a confident wrong answer costs more than a hedge.Evaluation & TrainingCalibrationThe property that outcomes given a probability of 0.2 occur about 20% of the time, measured across many predictions.Answers & Confidence

From the Jev directory

Laya on Hugging Face
The Hugging Face hub for Laya's open System 1 checkpoints: the 421M English model at the repo root, a 322M multilingual build in a subfolder, and the typed-decisions fine-tune, all Apache-2.0.
Jev Models & Reimplementations#community#open-models#local
Community
Decider Chat Gemma-4-31B
An untrained readout: stock Gemma-4-31B weights plus a decider config that reads option letters at the answer slot, scored #2 of 70 on Decision Index v0.2.1 at 57.33 against Jev 1.13.0's 57.91.
Jev Models & Reimplementations#community#open-models#local
Community
AutoTrust GEV-26B-Decide
AutoTrust's Gemma-based decision model with adaptive thinking: System 1 answers below 0.8 confidence hand off to the full Gemma 4 26B and mix the two by half, behind POST /v1/decide.
Jev Models & Reimplementations#community#open-models#local
Community
Laya Typed-Decisions
The fine-tuned Laya checkpoint for four typed-decision workflows: 0.766 accuracy on 2,000 held-out decisions, above Jev 1.13.0's published 0.727 and the teacher self-agreement ceiling it trained past.
Jev Models & Reimplementations#community#open-models#local
Community
Typed Decisions Benchmark
The 400-case, 2,000-decision benchmark open models train and report against: four workflows, state plus typed questions, and gold distributions averaged from three teacher samples.
Jev Benchmarks & EvaluationsDocs#community#benchmarks#evaluation
Community
Tasksource Jev Typed-Decisions
2.5 million typed decisions recast from 670 sources: choice, score, and noul rows with per-row licenses, soft annotator targets, and Jev-audited mix weights.
Jev Benchmarks & EvaluationsDocs#community#evaluation#calibration
Community

61 more matching entries in the full directory.

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai