Jev directory
CommunityJev Models & Reimplementations
Phishing Spam Jev-Like Classifier
A DistilBERT prototype from akumar0205 that sorts historical email into legitimate and spam, reporting 0.987 accuracy and 0.983 spam F1 on a held-out SpamAssassin split; it is not a phishing or malware detector.
- Reading time
- 2 min
- Published by
- Community
- Author
- akumar0205
- Last verified
- Oct 2026
- Languages
- Python
- Capabilities
- Open sourceSelf-hostedRuns locally
Highlights
- DistilBERT (distilbert-base-uncased) fine-tuned for binary email triage: label 0 legitimate, label 1 spam.
- Held-out split of 890 messages: accuracy 0.987, spam F1 0.983, spam PR-AUC 0.998, 6 spam false negatives.
- Training data: 4,675 SpamAssassin messages after duplicate removal (2,781 legitimate, 1,894 spam).
- Reads normalized subject, header, body and URL strings up to 512 tokens; low-confidence input may route to review.
- Card states the confidence threshold is a prototype policy choice, not a guarantee of calibration.
Quickstart
python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
repo = "akumar0205/phishing_spam_jev_like_classifier"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo).eval()
text = "Subject: Example subject\nBody: Example email text"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
scores = model(**inputs).logits.softmax(dim=-1)[0]
print({"legitimate": scores[0].item(), "spam": scores[1].item()})Watch out
Card reviewed 2026-10-17; the 0.987 accuracy and 0.983 spam F1 are self-reported on a held-out split of the same historical SpamAssassin corpus, the card calls the confidence scores estimates rather than calibrated probabilities, and it warns against autonomous security use or treating the model as a phishing detector.
Questions about this entry
Short answers drawn from the linked project or page.
A fine-tuned DistilBERT sequence classifier for binary email triage: legitimate versus spam, trained and evaluated on a historical SpamAssassin corpus. It is a prototype, not a phishing or malware detector, and it should not be treated as a complete email security system.
On a held-out group-stratified split of 890 messages: accuracy 0.987, legitimate F1 0.989, spam F1 0.983, macro F1 0.986 and spam PR-AUC 0.998, with 6 spam false negatives. The results describe this split only.
The Apache SpamAssassin public corpus (easy_ham, hard_ham, spam, spam_2) mapped to ham or spam: 4,675 messages after exact duplicate removal, comprising 2,781 legitimate and 1,894 spam items, tokenized from normalized text at up to 512 tokens.
The repository name frames it as a JEV-like classifier, meaning a small model returning a decision-oriented output with label scores and a routing action rather than generated text; the card itself does not mention Jev or TypeSafe anywhere.
No. The card says confidence scores are model estimates, not calibrated probabilities or proof of safety, and that the threshold in the training artifacts is a prototype policy choice, so route uncertain inputs to human review instead of trusting a raw score.
It does not authenticate senders, inspect attachments, resolve or visit links, or verify SPF, DKIM or DMARC, and it can miss new campaigns, adversarial examples, non-English messages and content unlike its training data, per the card's out-of-scope section.
Apache-2.0 for the model artifact, with base model attribution to distilbert-base-uncased. Message-text copyright remains with the original senders per the corpus documentation, dataset terms are described separately in ATTRIBUTION.md, and no raw emails ship here.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
X: Standard One SH (StandardThinking)
LinkedIn: What multimodal decision models actually change, and what they do not.
LinkedInJevDirectory.org
X: 1,565 emails, six decision models
Hacker News: Decision Index 0.2.1 scores open reproductions of Jev on a frozen suite.
jackbravo4 pointsDecision Index 0.2.1 – 70 open reproductions of TypeSafe Jev's Decision ModelRead the thread on Hacker NewsMore Jev resources like this
Fastino's 1B English member of the GLiNER2.5 classification family: pass any label set at call time for intent, routing, sentiment or multi-label tags and get labels back in a single forward pass with no generated tokens. It averages 59.6% exact-match on Fast Decisions.
Jev Models & Reimplementations#community#open-models#local
Community
Fastino's 340M English classification model and the headline member of the GLiNER2.5 family: pass any label set at call time and get intent, routing, sentiment or multi-label tags back in one forward pass, at 60.2% exact-match average on the Fast Decisions suite.
Jev Models & Reimplementations#community#open-models#local
Community
A 0.6B non-autoregressive decision model on Qwen3-0.6B that returns calibrated Choice, Score and Noul distributions in one forward pass, scoring 95.8% on its 22,110-item held-out test split.
Jev Models & Reimplementations#community#open-models#local
Community
The 322M mmBERT-base build of Laya for 100+ languages: state plus typed questions return typed answers with probabilities in one forward pass, with up to 8,192 tokens for long documents.
Jev Models & Reimplementations#community#open-models#local
Community
The Hugging Face hub for Laya's open System 1 checkpoints: the 421M English model at the repo root, a 322M multilingual build in a subfolder, and the typed-decisions fine-tune, all Apache-2.0.
Jev Models & Reimplementations#community#open-models#local
Community
Superagent's security triage model: a continual fine-tune of AutoJev-27B that returns calibrated probabilities for prompt-injection and content checks, with a release policy at 0.70 unsafe.
Jev Models & Reimplementations#community#open-models#local
Community
From the Jev guides
Original write-ups on what this entry is for.
From the Jev AI model community
Posts from builders shipping with Jev right now.