Question 1

Are these meters fact-checking, or opinions with a number attached?

Accepted Answer

They are Jev's probabilities, not a fact-check. Both speakers in a debate get the same five questions — factual claim, evasive, contradicts self, emotional appeal, dodged question — and the highlight clips are chosen by one fixed rule, so the index compares speakers on an identical rubric rather than verifying anything.

Question 2

Can I trust the 99% preset accuracy figure?

Accepted Answer

It is the project's own eval, reported in eval/RESULTS.md: 200 held-out sentences written separately from the presets, including near-misses like a polite thank-you before a real answer or a promise containing a number, with AUC 1.00 per question. Their first preset version scored 94.5%, so treat the number as a measurement of this rubric, not of your content.

Question 3

What does scoring a full video actually cost?

Accepted Answer

The demo debate took 1,191 Jev calls and 5,955 yes/no answers over 1,182,843 input tokens for $0.0497, at roughly 0.4 s median per call. That is one request per sentence, which is why transcripts are cached and runs are resumable — a crash or a tweak does not repay transcription or scoring. A 33-sentence mock run costs about $0.002.

Question 4

Will it run on my machine, and where does the key live?

Accepted Answer

The installer targets macOS and Linux with Python 3.9+ and ffmpeg; transcription is mlx-whisper on Apple Silicon and faster-whisper elsewhere, and jevmeter doctor checks Python, ffmpeg, speech-to-text, and the key. Windows needs the manual install. The key is read from TYPESAFE_API_KEY, then a git-ignored .env, then ~/.config/jevmeter/config.json.

Question 5

How do I run it, and can I write my own questions?

Accepted Answer

Clone the repo, run ./install.sh, paste your TypeSafe key, then run jevmeter: the wizard asks for a video, a preset, and whether you want highlights or a full-length meter, and writes <name>.jevmeter.mp4 next to the source. A preset is a small JSON file, so you can write your own questions, and passing a link instead of a file needs yt-dlp installed.

Question 6

How do speakers and transcripts work when more than one person talks?

Accepted Answer

You can pass a transcript whose lines start with NAME: to mark turns; names come from that file and timing from the audio. Speakers left out of --speakers are not scored, but their latest turn is still handed to Jev as context, and --label sets the name shown on screen. Scoring is one Jev request per sentence, with the speaker's history and the last question as context.

Question 7

What comes out of it, and what is the licence?

Accepted Answer

Two edit modes: highlights auto-picks clips, adds a hyperlapse and a scoreboard end card for a minute or two of postable video, and full runs the meter across the whole range or a slice of it. Pillow renders the overlays and ffmpeg does the export, keeping the original audio and adding synthesized whooshes and blips, loudness-normalised for social. The project is MIT licensed.

Jevmeter

About this project

Highlights

Questions about this entry

From the Jev guides

From the Jev AI model community