Is Jev part of the desktop app, or only of its test suite?

Only of the test suite. The README describes the desktop app, the OpenWork MCP server and the org control plane without mentioning Jev. The Jev usage is an evaluator inside the evals testkit, so it runs in your tests rather than in the product.

Can I run it locally without an account or a subscription?

Yes for the local desktop app. Everything outside ee/ is MIT-licensed and no OpenWork account is needed locally. OpenWork Den under ee/ is source-available and requires a subscription in production, though it stays free for organizations of up to five users.

What does the Jev verification judge actually verify?

It scores agent-produced artifacts against typed pass-fail questions inside the eval specs, with a verification dictionary and matching tests supplying the wording. The criteria are therefore defined by your eval code rather than fixed by the tool.

Will this send my project files to a hosted model?

Cloud is optional and files stay local in the default path. Jev calls go to TypeSafe, so artifact text handed to the verifier leaves your machine, and exactly what gets sent is not documented in the README.

How do I get the OpenWork dev build running?

Use Node 24 and pnpm 11: clone the repo, run corepack enable, then pnpm install and pnpm dev, which launches the Electron desktop app with hot reload. npm and yarn are not supported, and pnpm is pinned through packageManager in package.json.

What does the OpenWork MCP expose to an agent?

Four tools: search_capabilities finds what you can use, execute_capability runs it, and list_skills and get_skill list your skills or read one SKILL.md directly. Adding the remote MCP server opens a browser so you can sign in and choose an OpenWork organization.

Where do the Jev verification evals actually live?

Executable coverage lives in evals/specs/**/*.test.ts, with app-driving journeys in .e2e.test.ts files. Run a PR-lane spec with pnpm evals:pr specs/ .test.ts, or an app-driving journey locally with pnpm evals:e2e --local.

CommunityJev Benchmarks & Evaluations

OpenWork

An open Claude Cowork-style workspace whose eval testkit can call Jev as a typed verification judge on agent-produced artifacts.

See highlights

Open github.com

Stars: 24k
Language: TypeScript
Published by: Community
Author: different-ai
Last verified: Oct 2026

different-ai/openwork

Organization repository on GitHub

View on GitHub

The open-source alternative to Claude Cowork (powered by opencode)

Forks: 2.4k
Last pushed: Updated Oct 2026

Repo stats from the GitHub API, cached Oct 2026.Homepage

About this project

OpenWork is the free, open-source alternative to Claude Cowork and Codex: a desktop app for macOS, Windows, and Linux where AI agents do real work on your own files. It is built on OpenCode, works with any model — 50+ providers, your own API keys, or local models via Ollama — and lets teams share skills and MCP servers. Download OpenWork · GitHub releases · Docs. Read this in: 简体中文 · 繁體中文 · 日本語.

Quoted from the project's README. Read the full README.

Tagscommunity typescript agents evaluation

Works forJev for Evaluation Jev for Verification

Used inJev for Software & DevTools

Languages: TypeScript
Capabilities: Open sourceRequires API key

Highlights

evals/packages/testkit/src/verification-jev.ts implements a Jev-backed verification evaluator.
evals/verification-dictionary.md pairs with tests in verification-jev.test.ts for typed pass-fail judgments.
The verification walkthrough frames Jev as a typed pass-fail judge over agent-produced work.

Watch out

License NOASSERTION in GitHub metadata. Eval path is optional within a broader opencode-powered product.

Questions about this entry

Short answers drawn from the linked project or page.

Related terms

Glossary definitions related to this entry.

Precision and recall

More Jev resources like this

LangWatch

4.9k

LangWatch Instant Evals ships a TypeSafe Jev classifier that batches Noul, Score, and Choice judgments on production traces in one POST per text.

Jev Benchmarks & Evaluations#community#typescript#evaluation

Community

langwatch/langwatchTypeScript

jev-regress-bench

Reproducible benchmark asking which approved agent answers changed meaning after a config edit: one Jev Choice (same, fact_differs, action_differs, specificity_differs) caught all 19 real changes with 13 false alarms across 109 before/after pairs, versus 33 for a three-tier stack and 19 for an LLM judge.

Jev Benchmarks & Evaluations#community#benchmark#regression-testing

Community

redhatpanda/jev-regress-benchTypeScript

jev-harness-lab

Evaluation scripts, MCP tools, and a skill router built to measure where jev-1.13.0 fits in agent harnesses, with raw results under eval/.

Jev Benchmarks & Evaluations#community#evaluation#agents

Community

Aitejiu/jev-harness-labPython

hev reranker

A BEIR reranking recipe that asks up to 30 parallel Noul relevance questions in one Jev state, with RESULTS.md against cross-encoders and hosted rerankers.

Jev Benchmarks & Evaluations#community#evaluation#python

Community

hev/rerankerPython

S1MB

System One Mosaic Benchmark: a Python evaluator and Next.js viewer comparing Jev and open decision models across 137 English Choice, Noul and Score benchmarks from 106 dataset subsets, with recorded evaluation conditions and a public Hugging Face leaderboard.

Jev Benchmarks & Evaluations#community#benchmark#leaderboard

Community

hotchpotch/S1MBPython

jev-judge

Python CLI that runs declarative YAML, JSON or JSONL test suites natively on Jev Noul, Choice and Score decisions for RAG faithfulness, hallucination detection and agent safety, with watch mode, parallel batching, a pytest plugin and GitHub Actions PR reporting.

Jev Benchmarks & Evaluations#community#cli#ci-cd

Community

00200200/jev-judgePython

Back to all resources

From the Jev community

Posts from builders shipping with Jev right now.

Follow @typesafeai