Question 1

Nine of nine scams caught with zero false alarms - is that a real result or a tuned one?

Accepted Answer

Tuned. The README states that the weights and two of the eleven questions were adjusted after looking at the same 16 hand-written samples in src/shared/samples.ts that the number is measured on, and calls it a smoke test rather than an accuracy measurement. That is exactly why the repo ships npm run eval: point it at your own labeled messages with the -- flag and re-measure before trusting the thresholds or the floor that protects serious red flags.

Question 2

Can I retune the thresholds without paying for more Jev calls?

Accepted Answer

Yes, and that is a design property rather than a convenience. The policy layer in src/shared/policy.ts is a pure function combining Jev's judgments with the code-derived signals into a weighted risk score, so moving the threshold sliders just re-scores the answers already returned. There is also a floor: any single serious red flag at 90% or above cannot be marked safe no matter what the total works out to.

Question 3

What happens if someone finds the URL to my deployment?

Accepted Answer

The README is explicit that /api/check has no auth and no rate limit, so anyone with the link can spend your TypeSafe credits. It is hosted on Vercel with TYPESAFE_API_KEY set in the project's environment variables, and pushes to main deploy automatically. The stated plan is to take the endpoint down or put protection in front of it after the demo, which makes it a demo-shaped deployment rather than something to leave open.

Question 4

What does Jev actually see for each message?

Accepted Answer

The README splits the work deliberately: signals.ts reduces the message to observable facts such as link domains, shorteners, risky TLDs, IP links and sender type, then check.ts asks eleven narrow questions in one parallel call. It does not spell out exactly what state is included in the Jev request, so whether the raw text is forwarded is something the README does not answer. It is also silent on token cost per message and expected latency.

Question 5

How many questions does scam-shield ask per message, and in how many calls?

Accepted Answer

Eleven narrow questions in one parallel call: eight Nouls such as asks for credentials, unusual payment, or link does not match the brand, one Score for pressure, and one Choice for the scam pattern. The answers come back from a single TypeSafe request.

Question 6

Which facts are extracted in code instead of by Jev?

Accepted Answer

Code observes link domains, URL shorteners, risky TLDs, IP links, and whether the sender is a phone number, short code, or email address. Policy then combines those code signals with Jev's judgments into a weighted risk score with a floor so one red flag above 90% cannot be marked safe.

Question 7

How do I run scam-shield's offline tests versus a live evaluation?

Accepted Answer

npm test runs unit tests for link parsing and policy with no API calls. npm run eval runs live over the labeled samples, and you can point it at your own file with npm run eval -- your-messages.json. npm run dev serves the app at localhost:5173.

Scam Shield

About this project

Highlights

Questions about this entry

More Jev resources like this

From the Jev guides

From the Jev community