Question 1

What is jev-testbench and why does it exist?

Accepted Answer

It is a measurement bench for Jev as a typed probabilistic decision primitive. Its stated purpose is to answer not 'is it good?' but 'what exactly did it do, and what can I actually conclude from that?', with every real API call appended to an append-only JSONL log.

Question 2

What is the main result?

Accepted Answer

A negative one, stated without softening: NO_STRONG_JEV_SPECIFIC_NOVEL_FINDING_YET. Nothing in the repository is both new relative to TypeSafe's published material and supported by the repository's own data.

Question 3

What evidence is frozen, and how is it protected?

Accepted Answer

Two canonical logs: 42 core records and 12 P3 records, each with a published SHA-256 hash. The core log records the resolved model as jev-1.13.0 on 42 of 42 records and usage of 21,767 input and 3,594 output tokens as returned by the API, never estimated.

Question 4

Why are there two separate logs instead of one?

Accepted Answer

Because the 42-record core evaluation is frozen historical evidence: appending to it would invalidate a published hash. The P3 exploration kept its own log for the same reason, as the evidence provenance docs describe.

Question 5

What does it say about cognitive debt?

Accepted Answer

At the meta level the project documents what happens when AI-assisted project complexity grows faster than the maintainer's understanding, kept deliberately separate from the evidence half and written up in docs/AGENTIC_ENGINEERING_AND_COGNITIVE_DEBT.md.

Question 6

Does the testbench show that Jev is good or bad?

Accepted Answer

Neither. There is no ground truth anywhere in the bench, so there is no accuracy score to rank, and every number describes one payload, one account and one session. TypeSafe's own performance statements are recorded as official claims about their workload, never as verified results.

Question 7

How was the project built and how do I read it?

Accepted Answer

It was built through Agentic Engineering, with a bilingual technical blog and an offline test suite. Start with the README and the evidence provenance docs, then read 'what you cannot conclude' before reusing any number; the release is a v0.1.0 evidence freeze.

Jev Testbench

Highlights

Questions about this entry

From the Jev guides

From the Jev AI model community