Question 1

What is judge_rails?

Accepted Answer

A Ruby gem that turns semantic judgments into ordinary ActiveRecord attributes. A judgment model answers typed questions about text and returns calibrated probabilities; this gem turns that into a column on your model — indexable, sortable, paginable and kept up to date.

Question 2

How do I define a judged attribute?

Accepted Answer

Declare the source text with judge_source, then add judge_attribute lines: Judge.noul for a yes/no question, Judge.choice for named options and Judge.score for an ordered scale. Generated scopes such as urgency_above(0.8) and intent_is('billing') cover your queries.

Question 3

Does judge_rails require Rails?

Accepted Answer

No. The README says it is also a plain Ruby client, with no Rails and nothing outside the standard library. The ActiveRecord integration is the headline use, but the client works standalone.

Question 4

How do I install and set it up?

Accepted Answer

Add gem 'judge_rails', then run bin/rails generate judge:install, bin/rails generate judge:attribute Ticket urgency:noul intent:choice frustration:score, and bin/rails db:migrate. Set JEV_API_KEY in the environment, or judge.api_key in Rails credentials; TYPESAFE_API_KEY works too.

Question 5

Can I evaluate it without an API key?

Accepted Answer

Yes. TypeSafe Jev is reached through an early-access waitlist, but judge_rails_demo boots with 250 support tickets already judged, seeded from a committed file. Run bin/rails db:prepare db:seed and bin/rails server to see it without any API call.

Question 6

What is the json version caveat?

Accepted Answer

On activesupport 8.1, pin json below 3: it calls ::JSON.parse(json, options) with a positional hash, which json 3.x rejects. The README notes it surfaces on a jsonb column with a non-nil default — exactly the sidecar — so it looks like a bug in this gem but is not.

Question 7

Did adding detailed criteria help accuracy?

Accepted Answer

On the demo, yes: the object form (what, not_for, examples per option) raised agreement with reference labels by about four points on both questions, for about 290 more input tokens per ticket, which the README calls a thousandth of a cent. Those labels come from two model annotators, not people.

Judge Rails

Highlights

Quickstart

Questions about this entry

From the Jev guides

From the Jev AI model community