Skip to content
JevDirectory.orgTraffic count unavailable. View site traffic
Evaluation & TrainingLog lossCross-entropy

Cross-entropy loss

The training objective that penalizes a confident wrong probability monotonically, and the proper scoring rule behind log loss in evaluation.

Open docs.typesafe.aiAll 164 terms
Category
Evaluation & Training
Also known as
Log loss, Cross-entropy
Related terms
5
Directory entries
7
Docs
docs.typesafe.ai
Added
2026-10-12

Definition

Cross-entropy loss measures the gap between the predicted distribution and the true label, penalizing a confident wrong prediction far more than a hesitant one. It is the standard objective for a decision head, and the same quantity appears at evaluation time as log loss.

It is a proper scoring rule: it is minimized, in expectation, by reporting your true probability, which is why it trains calibration rather than only accuracy. Its unbounded penalty on confident errors is also why it can be sensitive to label noise.

Tagstrainingevaluation

Related terms

Definitions that connect to this one.

KL divergenceA directional measure of how far one probability distribution is from another, used to compare, distill, and monitor decision heads.Evaluation & TrainingProper scoring ruleA reward or error measure that is optimized only by reporting true probabilities — the training signal behind RLCD.Evaluation & TrainingDistillationTraining a smaller model to reproduce a larger model's output distribution, including its probabilities rather than labels alone.Evaluation & TrainingBrier scoreA score for a full probability vector against one observed label. Lower is better, and a confident wrong answer costs more than a hedge.Evaluation & TrainingFine-tuningContinuing training on your own labelled decisions, which is where an open checkpoint stops being a proof of concept.Evaluation & Training

From the Jev directory

Jev Distill Corpus v3
The 741,000-row distillation corpus behind AutoTrust's Jev students: choice, score, and noul rows with full target distributions across yuri and openjev source streams, Apache-2.0.
Jev Benchmarks & EvaluationsDocs#community#evaluation#distillation
Community
AI Primer: RLCD and Calibration
Why TypeSafe trains decision models with RLCD instead of RLHF: calibrated probabilities where 0.2 outcomes happen about 20% of the time, and the case for machine-to-machine automation.
Jev Guides & ArticlesDocs#official#docs#evaluation
Official
Jebadiah
5GitHub stars
Open System One decision models (27B, 9B and 4B on Qwen chat bases, in bf16, GGUF and MLX) that answer Choice, Noul and Score questions with a probability for every option from one forward pass. The repo carries the trainer, data builders, evaluation harnesses and a record of every run.
Jev Models & Reimplementations#community#decision-models#open-weights
Communitygetainode/jebadiahPython
Luce
8GitHub stars
Open recipe for calibrated decision models: an LLM teacher writes training data from a one-sentence task description, a LoRA plus decision head on Qwen3-4B-Base answers Choice, Score and Noul questions with calibrated probabilities, evaluated next to Jev on identical items.
Jev Models & Reimplementations#community#calibration#training
Communityscienthoon/lucePython
tev1
227GitHub stars
Together AI's open recipe and weights for a Jev-inspired decision model fine-tuned on Qwen3.5-4B, with the full data pipeline, training config, and saved benchmark results.
Jev Repos & SDKs#community#python#open-models
Communitytogethercomputer/tev1Python
The Bitterest Lesson
A TypeSafe blog essay arguing that in ML the order that matters is doing the right task, then data, then compute, then algorithms, using the InstructGPT result as its example.
Jev Guides & ArticlesArticle#official#article#models
Official

1 more matching entry in the full directory.

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai