Question 1

What does Jev actually decide in a Sedum test?

Accepted Answer

In goal mode each turn asks one Jev Choice for the next operation plus a speculative target among the page's offered elements, capped at 24 requests, 18 actions and 120 seconds. An independent verify claim must clear two Noul questions (holds at >= 0.75, contradicted flagged at >= 0.5) before the test passes, since the planner's done is never a verdict.

Question 2

How is Sedum kept cheap enough for every PR?

Accepted Answer

It is open source, runs locally on Playwright with your own key, and caches locator results in Git metadata. The README's worked example prices a 20-person team's PR suite at $38-$91 a month instead of per-step platform pricing of $4,875.

Question 3

What do I need before I can run Sedum for the first time?

Accepted Answer

Node.js 20.19 or newer, a TypeSafe key or compatible endpoint, and credentials for the browser. In an empty git repo run npm install -D sedum-cli, npx sedum init, npx sedum browsers install chromium, copy .env.example to .env and set TYPESAFE_API_KEY, then npx sedum run tests/example.test.ts --headed. If anything is missing, npx sedum doctor names it and says how to fix it.

Question 4

Can Sedum run against a provider other than TypeSafe?

Accepted Answer

Yes. For any compatible endpoint set TYPESAFE_BASE_URL to its API root and TYPESAFE_API_KEY to that provider's key, plus TYPESAFE_DEFAULT_MODEL if the model name differs. Cloudflare Workers AI is a separate option: select provider: {name: clef, model: clef} or clef-flash and set CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_AUTH_TOKEN. Both paths return probabilities, never free text.

Question 5

What does Sedum actually send to the model?

Accepted Answer

Prompts, scoring and caching live in the repository, so the request path is inspectable rather than a black box. A step sends what it needs to resolve the sentence on the page, and values wrapped in secret() are typed by Playwright but never reach the model or the reports. Writing {{name}} in a sentence and passing the value separately keeps the prompt identical across runs.

Question 6

Do tests have to be TypeScript, and how does a failed test get triaged?

Accepted Answer

Either format works and both run side by side: TypeScript files use test() with ai steps plus ordinary Playwright, and YAML files hold a list of sentences with data, $VAR secrets, shared modules pulled in with use:, and before/after steps. On failure, npx sedum run --reporter markdown writes a report with the sentence, file and line, the scores, the page text judged, the elements considered and a rerun command; page text is fenced and labelled untrusted so a page cannot inject instructions.

Question 7

How does Sedum treat a claim that only marginally holds?

Accepted Answer

Every claim is scored against a threshold you set and then checked for contradicting evidence, so a marginal pass is flagged rather than silently green, and changing a step to a false claim fails the run with exit code 1. CI is documented for GitHub Actions, GitLab and Jenkins with JUnit and job summaries, and the CLI adds filters, parallel runs and sharding, retries, timeouts and reporters on top.

Sedum

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev guides

From the Jev community