Skip to content
JevDirectory.org
CommunityTools & Integrations811 starsVerified 2026-09-22

AI CLI

Vercel Labs' terminal CLI for the AI SDK: its `ai evaluate` command answers named Boolean, Choice, and Score questions, with Jev as the default evaluation model and one unchanged input shared across all questions.

Category
Tools & Integrations
Published by
Community
Author
vercel-labs
Added
2026-09-22
Tagscommunitytypescriptclievaluationvercel

Highlights

  • `ai evaluate` takes repeatable --boolean, --choice, and --score flags or a named question map in JSON; Jev is the default evaluation model.
  • Stdin maps to the SDK's state and typed flags build the named questions; with --json, stdout carries only metadata.
  • Boolean returns P(true) from 0 to 1, Choice returns one supplied option plus its distribution, and Score uses Jev's probability-weighted mean.
  • Requires Node.js 22+ and an AI Gateway API key or a provider key; evaluate requests time out after 30 seconds by default.
  • The same binary generates text, images, video, and audio, supports piping between commands, and exits 2 on partial generation failure.

Quickstart

bash
npm install -g ai-cli
cat ticket.txt | ai evaluate \
  --boolean "refund=Refund requested?" \
  --choice "team=Which team?" \
  --choices "team=billing,support" \
  --score "tone=How positive?" \
  --levels "tone=angry,neutral,happy"

Watch out

The README states Apache-2.0 though the digest lists no license file; it needs Node.js 22+ and a Vercel AI Gateway key (or a provider-specific key), and question limits are enforced by the SDK and provider.

More like this

17GitHub stars
A CLI that grades markdown and text files against plain-Markdown rulesets with Jev, running every rule against every line in parallel and caching unchanged lines to avoid repeat API calls.
Tools & Integrations#community#typescript#cli
Communityslop-grader
392GitHub stars
A staged code-review workflow that uses Jev for bounded judgments over Git diffs or whole codebases, with a local dashboard. Policy and thresholds stay in code while Jev screens five risk areas.
Tools & Integrations#community#typescript#code-review
Communityjev-review
273GitHub stars
An experimental Sutro CLI that evaluates CSV, Parquet, and JSONL data with Jev, asks you to label ambiguous and random audit rows, and uses GEPA to propose improved function definitions.
Tools & Integrations#community#python#cli
Communityjev-align
Back to all resources

From the community

Posts from builders shipping with Jev right now.

Follow @typesafeai

Browser Use Ultrafast, powered by Jev

A really smart switch statement

hype-free explanation of jev: jev does not replace gpt / claude jev is just a *really* smart switch statement like if 2016 ml classifiers got 2026 levels of intelligence it's a new* type of tool that will make a lot of workloads insanely fast, cheap, and accurate * = and by Show more

Diogo Almeida
Diogo Almeida
TypeSafe AI
@CompleteSkeptic

After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI? I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev • 20-200x faster • 40-400x

Reply

When a designer gets Jev

Full Jev video tutorial

The case against Jev-scored compaction

This is a terrible compaction strategy that fundamentally doesn't understand how compaction and context management work. Seems like a lot of people are confused so let's break this down. 1. Compaction isn't a filter The role of compaction is to clean up history to keep the Show more

tamara
tamara
@tamarajtran

found the perfect use case for @typesafeai Jev: instant compaction in 2026, why is compaction still a summarization prompt? Jev can make it instant by scoring every tool call and dropping what’s irrelevant

Reply

Classifying rows in DuckDB