Organization repository on GitHub
Build calibrated AI Functions from human feedback using Jev and GEPA.
- Forks
- 26
- Last pushed
- Updated Oct 2026
Repo stats from the GitHub API, cached Oct 2026.Homepage
Jev directory
CommunityJev Tools & CLIs
Jev Align — sutro-sh
An experimental Sutro CLI that evaluates CSV, Parquet, and JSONL data with Jev, asks you to label ambiguous and random audit rows, and uses GEPA to propose improved function definitions.
- Stars
- 309
- Language
- Python
- License
- APACHE-2.0
- Reading time
- 3 min
- Published by
- Community
- Author
- sutro-sh
- Last verified
- Sep 2026
About this project
jev-align is an experimental CLI from Sutro for building AI Functions with TypeSafe's Jev. It finds uncertain examples, asks you to label them, and uses GEPA to improve the function. Use it in your application, keep learning from production examples, and share portable functions through ai-functions.dev. Start the CLI with either jeva or jev-align. Use pip install jev-align if you do not use uv. The guided setup discovers local CSV, Parquet, and JSONL files and includes three ready-to-run examples.
Quoted from the project's README. Read the full README.
Used inJev for Software & DevTools
Highlights
- Each round measures uncertainty, selects ambiguous rows plus a random audit sample, runs GEPA, and shows a definition diff to accept, reject, or rewind.
- Supports binary, multiclass, multilabel, and score tasks with 5, 10, 15, or 20 training annotations per round and a default GEPA budget of 300 metric calls.
- Runs Jev through TypeSafe AI, Vercel AI Gateway, or Cloudflare Workers AI; the Vercel and Cloudflare routes do not need a TYPESAFE_API_KEY.
- Published functions go to ai-functions.dev with their definition, labels, and rationales; the registry is public, so publish only data you can share.
Quickstart
bash
uv tool install jev-align
export TYPESAFE_API_KEY=YOUR_API_KEY
export OPENAI_API_KEY=YOUR_OPENAI_KEY
jevaWatch out
Apache-2.0. Needs Python 3.11+ and a TYPESAFE_API_KEY or Vercel/Cloudflare credentials, plus an OpenAI, Anthropic, or Gemini key for GEPA; Sutro is not affiliated with TypeSafe AI.
Questions about this entry
Short answers drawn from the linked project or page.
No weights are touched. Each round measures uncertainty, selects ambiguous rows plus a random audit sample, runs GEPA over your accumulated labels, and shows you a definition diff to accept, reject or rewind. The saved artifact is an accepted definition plus its input signature and runtime backend, not a checkpoint.
At least two. Jev needs a TYPESAFE_API_KEY, or Vercel AI Gateway or Cloudflare Workers AI credentials, neither of which needs a TypeSafe key. GEPA runs a separate reflection model, so you also need an OpenAI, Anthropic or Gemini key, or any LiteLLM provider including a local vLLM endpoint passed with --reflection-model.
You choose 5, 10, 15 or 20 training annotations per round, with an optional 20% held-out set, a default GEPA budget of 300 metric calls, and the first 1,000 rows used unless the dataset is smaller. You review every label, and a higher training score never accepts a proposal automatically.
A published function ships the accepted definition, input signature, runtime backend, learning configuration, labeled inputs, labels, splits and optional rationales. It excludes unlabeled source rows, local paths and credentials, but the registry is public and does not verify that every annotation was human-made, so do not publish labels you cannot share. Unpublishing is reversible yet cannot recall copies already downloaded.
Four: binary True or False, multiclass with exactly one fixed label, multilabel with zero or more fixed labels, and score with one level from an ordered rubric. Repeated --class options define classes and repeated --score-level options define the score levels.
TypeSafe AI, Vercel AI Gateway or Cloudflare Workers AI, chosen in guided setup and saved with the function. The Vercel and Cloudflare routes do not need a TYPESAFE_API_KEY. GEPA's reflection model is configured separately and can be any LiteLLM provider, including a local vLLM endpoint.
Load a saved function with AIFunction.load(".jev-align/runs/<run-id>", capture=True) and run it to capture useful production examples. Later resume through jeva functions to label them, and GEPA uses that feedback to propose the next version.
Other listings of this project
Forks and copies of this project in the directory. This entry is the canonical one.
Inline verifier that checks LLM responses and agent plans with one Jev call across alignment heads and returns block, flag, or pass for gates and hooks.
Jev Practices & Patterns#community#awesomejev#catalog-import
Community
caiovicentino/jev-alignJavaScript
caiovicentino/jev-alignJavaScriptMore Jev resources like this
A Rust CLI and PyO3 library that streams JSONL and Parquet rows through Jev Noul and Score rubrics, writing clean and rejected records to disk without rewriting content.
Jev Tools & CLIs#community#rust#cli
Community
AkashPriyadarshii/jev-curateRust
AkashPriyadarshii/jev-curateRustA grep-style CLI where the pattern is a plain-English description: each line becomes one Jev Noul question, judged concurrently with matches printed in input order, working on tail -f as well as files.
Jev Tools & CLIs#community#python#cli
Community
keltokhy/jgrepPython
keltokhy/jgrepPythonA semantic code search CLI that asks one yes/no Jev question per function and ranks the returned probabilities, screening a whole codebase for a pattern in seconds for cents.
Jev Tools & CLIs#community#python#cli
Community
sufianetaouil/everyPython
sufianetaouil/everyPythonVercel Labs' terminal CLI for the AI SDK: its `ai evaluate` command answers named Boolean, Choice, and Score questions, with Jev as the default evaluation model and one unchanged input shared across all questions.
Jev Tools & CLIs#community#typescript#cli
Community
vercel-labs/ai-cliTypeScript
vercel-labs/ai-cliTypeScriptA CLI that scores code quality by asking Jev yes/no questions about each file, runs your existing test command, and turns uncovered paths into small actionable queries for agents.
Jev Tools & CLIs#community#rust#cli
Community
supercorp-ai/supercovRust
supercorp-ai/supercovRustAn ast-grep semantic linter where each rule is one natural-language question judged by Jev: 98 rules flag name/body drift, stale comments, weak tests, quiet failures, and commits whose diffs contradict their messages.
Jev Tools & CLIs#community#typescript#cli
Community
mizchi/jev-lintTypeScript
mizchi/jev-lintTypeScriptFrom the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.