Skip to content

Scoring Without Measuring: Bands, Rubrics, and Composite Triage

A Score is a weighted position, not a ruler. Why the number can land between levels, why levels are judged separately, and how to combine several atomic scores in code instead of one crowded rubric.

Published
Category
Jev Practices & Patterns
Resources cited
5
Reading time
3 min
Tags#score#primitives#scoring#triage

A Score rates a state against an ordered list of levels and returns a probability-weighted position that can land between them. That one sentence contains the trap: a Score looks like a measurement and is not one. Used well it is the best triage primitive Jev has; used as a ruler it quietly lies.

What the number actually is

The score is each level's index times its probability. On a three-level rubric, 1.43 means the probability sat mostly on level 1 with some on level 2 — an expectation over your levels, not a physical quantity. There is no tenth of a unit between level 1 and level 2 to observe; the value is the model's weighted position on your scale.

The docs are blunt about the consequence: do not interpolate between levels to reconstruct an exact number. Arithmetic belongs in code, and the jaggedness page lists counting and numeric precision among Jev's weak spot on purpose.

Levels are judged separately

Each level description is scored on its own, so "worse than the previous level" adds no signal — the model never compares the descriptions to each other. Write each level as a standalone state of the world: cosmetic, no functional impact; degraded feature with a workaround; blocking, no workaround. Two to ten levels, spaced so a reader would not hesitate to pick one.

Use bands, not points

Because the value is a position rather than a measurement, branch on bands: below 1.0, 1.0 to 2.0, above 2.0. The confidence docs describe the same move for probabilities. A band is honest about the resolution of the primitive; a hard comparison to 1.43 implies a precision that is not there.

Combine dimensions yourself

When a decision has several dimensions — severity, urgency, blast radius — do not cram them into one rubric. Ask a separate atomic Score for each and combine them in code, the approach the composite scoring pattern names. The weights are then yours to state and audit, instead of buried in an instruction no one can inspect.

severity = response.scores["severity"].score
urgency = response.scores["urgency"].score
priority = 0.7 * severity + 0.3 * urgency

Keep the legend with the score

A Score answer carries a legend that maps each level number back to its description. Log it. A stored 1.43 is meaningless once the rubric changes, and the rubric will change. The primitives overview treats the legend as part of the answer for exactly this reason.

Where Score is the wrong tool

  • As a classifier. If the levels are not ordered, use a Choice.
  • As a yes/no gate. If there are only two outcomes and no order, use a Noul.
  • As a measurement. If you need "how many" or "how long," count in code one question per item.
  • As a ranking of many candidates. Score each candidate, but combine with a relevance model as the rerank cookbook does rather than asking one Score to order a list.

Score is the right primitive for triage, rubric work, and feeding a code-side policy. Treat the number as a position on your scale, and it will do its job without pretending to be more than it is.

From the Jev directory

The resources behind this article.

Rate content against 2 to 10 ordered levels. Returns a probability-weighted score that can land between levels, per-level probabilities, and confidence, with the arithmetic left to code.
Jev Practices & PatternsDocs#official#primitives#score
Official
How TypeSafe reports certainty, how it differs from probability, and how to use it architecturally to gate and route decisions.
Jev Practices & PatternsDocs#official#confidence#routing
Official
Break a judgment into atomic Score questions, normalize each by its top level, and combine them with weights you control in code, as in the resume example's four scored dimensions.
Jev Practices & PatternsDocs#official#patterns#score
Official
One question per query-candidate pair raises top-1 accuracy from 5% to 18% and top-10 accuracy from 38% to 62% on 40 CLERC legal queries.
Jev CookbooksDocs#official#cookbook#reranking
Official
A maintained list of jev-1.13's known failure modes, literal reading, unreliable counting, dates as text, indirection, context rot, and contradictory criteria, each with a guardrail.
Jev Practices & PatternsDocs#official#docs#evaluation
Official
Back to all articles

More Jev articles

Most Jev accuracy problems are state problems. What belongs in a state, what is a pre-made judgment to leave out, and why naming the slice you ask about matters more than the wording.
Jev Practices & Patterns#state#design#primitives
Read article
A Noul returns one number and no separate confidence. How to read the value, pin its boundary with criteria, split compound questions, and set a per-action threshold that matches the cost of being wrong.
Jev Practices & Patterns#noul#primitives#confidence
Read article

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai