Question 1

What is this Jev and NZQA marking prototype?

Accepted Answer

It is a concept prototype of AI-assisted marking for any standard from its written criteria alone, with no labelled data and no model per standard. It targets the long tail of low-volume standards that would never justify training a dedicated model, and it is explicitly not endorsed by NZQA.

Question 2

How does one marking call work?

Accepted Answer

For each script, one TypeSafe Jev call returns guardrails as a noul question (is it a genuine attempt, and does it need safeguarding follow-up), a score distribution for each performance criterion, and an overall grade across Not Achieved, Achieved and optional Merit or Excellence.

Question 3

What are the three standards included in the prototype?

Accepted Answer

US 32405 Literacy Writing, the real Year 10 co-requisite standard, with criteria paraphrased from performance criteria 1.1-1.4; Level 3 Ag and Hort as an illustrative low-volume standard; and a build-your-own flow where a new standard can be created in 60 seconds by editing the task and criteria.

Question 4

How do I run the prototype?

Accepted Answer

On WSL or Ubuntu with Node.js 20 or later, clone the repository, copy .env.example to .env and set TYPESAFE_API_KEY, then run npm start with no npm install needed. Open http://localhost:3000, pick a standard and click Mark all scripts; npm run mock runs a no-API demo labelled as mock mode.

Question 5

Is this endorsed by NZQA or usable for real marking?

Accepted Answer

No. The repository says NOT ENDORSED BY NZQA - CONCEPT ONLY, all learner responses are synthetic, and the reference grades are the author's judgement rather than official results. It is a demonstration of the approach, not a production marking system.

Question 6

What does the UI reveal about what Jev is doing?

Accepted Answer

Per script you can see how each grade and criterion was read from Jev's answer: the index, the label sent, whether Jev's returned legend matches, and the probability. It also shows the exact request body, the raw response, a copy-as-curl action, and /log.html lists every call.

Question 7

Could it replace human marking?

Accepted Answer

Not as documented. The README positions it as triage: auto-accept confident scripts and focus people on the uncertain cases, which is the stated goal. It remains a prototype with synthetic responses and author-judged reference grades, and it is not endorsed by NZQA.

Jev Test — cgbarlow

Highlights

Quickstart

Questions about this entry

Other listings of this project

From the Jev guides

From the Jev AI model community