CommunityJev Guides & Articles
Jev for QA
Katalon's guide to Jev in software testing: where bounded judgments help QA, why deterministic assertions stay first, the documented weak spots that hit test checks, and the caveat that a constrained answer can still be wrong.
- Format
- Article
- Published by
- Community
- Author
- Joe Sturgess
- Last verified
- Oct 2026
Highlights
- Maps Jev to QA: failure triage, next-action choice, semantic UI checks, bug routing, and agent guardrails.
- Cites Arbon's 131 ms median loop and Callstack's 14-second, $0.0023 mobile QA run.
- Warns the schema stops wrong types, not wrong answers, and to pin the model version for regression tests.
- Recommends defaulting to unknown, never pass, when Jev is unavailable.
Watch out
Vendor-authored guidance for a testing audience; the useful parts are the independent citations and the failure-mode list, not the product framing.
Questions about this entry
Short answers drawn from the linked project or page.
Small bounded decisions inside a test loop: triaging failed tests, choosing an agent's next action, checking UI text for meaning, routing bugs, and blocking risky agent actions. It returns a choice, a score, or a probability, not text.
No. The guide is explicit that deterministic assertions come first wherever the expected result is precise, and Jev only handles the judgments a rule cannot make. It can still pick the wrong option.
Jason Arbon's Playwright loop, with 126 Jev calls in 24.3 seconds and a 131 ms median round trip, and Callstack's agent-device mobile QA, where one run took 14 seconds and cost $0.0023 in inference.
That the schema stops wrong types but not wrong answers, that the jev-latest alias moves under you so versions should be pinned for regression tests, and that page text can attack the model with injected instructions.
The guide recommends deciding in advance what the pipeline does when Jev is unavailable, and defaulting to unknown rather than pass, so an outage cannot silently turn into a green run.
Pick one decision, put rules in front of it, pin the model version, and compare Jev's answers against a set of past failures your team already labeled. If it does not beat the rules, you have lost a week, not a release.
Katalon, a testing-tools vendor, as an independent buyer's guide for its audience; it cites TypeSafe's docs and several community experiments rather than only its own product.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: End-to-end tests run by agents
X: Jev Engineering: a 10-step roadmap
More Jev resources like this
34GitHub stars
An independent ecosystem tracker that rescans GitHub, X, and five directories every three hours, publishing 220+ documented Jev cases with confidence grades and a JSON API.
Jev Guides & Articles#community#directory#data
Community
everyinfra/jev-radar
everyinfra/jev-radarThe Hacker News thread for frodi-karlsson/onesie, covering stdin questions, shell-safety gates, and calibration over labelled commands.
Jev Guides & ArticlesPost#community#integration#evaluation
Community
2.3kGitHub stars
A curated community list of public projects and practices built on Jev, grouped by direct Jev application domain with explicit inclusion rules and a category for practice signals like threads and interviews.
Jev Guides & Articles#community#awesome-list#documentation
Community
yibie/awesome-jevPython
yibie/awesome-jevPython632GitHub stars
A curated list of System One and typed decision models: hosted APIs, open weights, runtimes, SDKs, apps, benchmarks, and papers, in English and Chinese.
Jev Guides & Articles#community#awesome-list#open-models
958GitHub stars
An independent, community-maintained catalog of open-source projects built with Jev, citing public evidence behind each entry it lists.
Jev Guides & Articles#community#typescript#awesome-list
Community
heyjunpenn/awesome-jevAstro
heyjunpenn/awesome-jevAstro501GitHub stars
A curated link list for the System One and Jev ecosystem: TypeSafe's own launch material, open-weight decision models, benchmarks and commentary, with each entry annotated by what its authors measured.
Jev Guides & Articles#community#jevbest
Community
OmniJev/awesome-jevJavaScript
OmniJev/awesome-jevJavaScriptFrom the Jev guides
Original write-ups on what this entry is for.
Every decision path has a failure posture, whether you chose one or not. How to put retries in the client, bound the whole call, and make the fallback an explicit, per-action policy.
Jev Practices & Patterns#reliability#retries#api
Read article
Replacing a rule with a model in one step is how thresholds get tuned on the wrong data. The three stages, what each one learns, and the one-flag rollback that makes a canary safe.
Jev Practices & Patterns#evaluation#rollout#confidence
Read article
A test loop is a long series of small judgments. What the Playwright, mobile QA, and CI experiments report: two calls per probe, 131 ms medians, and near coin-flip failures to design around.
Jev Practices & Patterns#testing#qa#confidence
Read article
From the Jev community
Posts from builders shipping with Jev right now.