Q: What does a run cost, and what does CI actually need?

The questions for a step go out in one request of about a second, so cost tracks unresolved steps rather than steps executed — and committed lockfiles mean CI spends nothing. Run jevcumber features/ --frozen in CI and no TYPESAFE_API_KEY is needed at all. Concurrency is capped at four workers outside --frozen because Jev has its own rate limits, while --frozen defaults to your CPU count; --headed always implies a single worker, and console output prints per scenario block so lines never interleave.

Q: How much of my step wording has to fit what jevcumber can resolve?

More than with plain Cucumber, because Jev selects rather than generates. Put data in quotes — it selects values and never invents them — name controls as they appear, as "the Log in button", the "email field", or quoted I click "Log in", and navigate with a literal URL or path, or a "/login" path plus --base-url for portable features. "Given I am on the login page" gives it nothing to select from. Note that "search for", "submit" and "look up" type and press Enter, while "fill in ... with" only types.

Q: How do I set it up and run the shipped example?

Requires Node.js 20+. Run npm install -g jevcumber, then jevcumber install-browser to download the Chromium build it drives. The repo ships an example with its lockfile already recorded, so you can watch a replay with jevcumber examples/ --frozen --headed without any API key.

Q: What reporters and failure evidence does it produce?

The console reporter is the default; json and junit are also available, and --reporter can be repeated to run several at once, writing to results.json or results.xml under the report directory, jevcumber-report by default. --report-dir changes it and --no-report disables it, while --trace records a Playwright trace per scenario and keeps it for scenarios that did not pass.

Q: Can I run setup or teardown code around each scenario?

Yes, through hooks in jevcumber.config.js or .mjs. beforeScenario runs once per scenario before its first step, and afterScenario runs once after its last step with that scenario's results. A beforeScenario that throws fails the scenario outright and skips its steps; an afterScenario that throws is reported as an appended failed step.

Question 1

Is this actually deterministic, or does a model judge every run?

Accepted Answer

Only after the first run. That run records what each step resolved to in <name>.feature.lock.json next to your feature file; commit it, and from then on --frozen replays it with no API calls, no API key and no model variance, failing on a missing or stale entry. A step you add or reword is resolved on the next run, and only that step. Outside --frozen, recorded steps replay but described expectations that could not be pinned stay live-judged every time.

Question 2

What happens when the model is not sure which control I meant?

Accepted Answer

It refuses to guess and shows its working: "Jev was not confident about the element (0.10 < 0.6)", then the candidates — button "Log in" at 45%, link "Sign up" at 35%, none at 20% — and a request to reword the step. The floor is --min-confidence, 0.6 by default and 0.75 for page-sourced values. An ambiguous step counts as a failure: the exit code is 1 if any step failed, was ambiguous, or was undefined.

Question 3

What does a run cost, and what does CI actually need?

Accepted Answer

The questions for a step go out in one request of about a second, so cost tracks unresolved steps rather than steps executed — and committed lockfiles mean CI spends nothing. Run jevcumber features/ --frozen in CI and no TYPESAFE_API_KEY is needed at all. Concurrency is capped at four workers outside --frozen because Jev has its own rate limits, while --frozen defaults to your CPU count; --headed always implies a single worker, and console output prints per scenario block so lines never interleave.

Question 4

How much of my step wording has to fit what jevcumber can resolve?

Accepted Answer

More than with plain Cucumber, because Jev selects rather than generates. Put data in quotes — it selects values and never invents them — name controls as they appear, as "the Log in button", the "email field", or quoted I click "Log in", and navigate with a literal URL or path, or a "/login" path plus --base-url for portable features. "Given I am on the login page" gives it nothing to select from. Note that "search for", "submit" and "look up" type and press Enter, while "fill in ... with" only types.

Question 5

How do I set it up and run the shipped example?

Accepted Answer

Requires Node.js 20+. Run npm install -g jevcumber, then jevcumber install-browser to download the Chromium build it drives. The repo ships an example with its lockfile already recorded, so you can watch a replay with jevcumber examples/ --frozen --headed without any API key.

Question 6

What reporters and failure evidence does it produce?

Accepted Answer

The console reporter is the default; json and junit are also available, and --reporter can be repeated to run several at once, writing to results.json or results.xml under the report directory, jevcumber-report by default. --report-dir changes it and --no-report disables it, while --trace records a Playwright trace per scenario and keeps it for scenarios that did not pass.

Question 7

Can I run setup or teardown code around each scenario?

Accepted Answer

Yes, through hooks in jevcumber.config.js or .mjs. beforeScenario runs once per scenario before its first step, and afterScenario runs once after its last step with that scenario's results. A beforeScenario that throws fails the scenario outright and skips its steps; an afterScenario that throws is reported as an appended failed step.

Jevcumber

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev guides

From the Jev community