cisco-ai-defense/skill-scanner
Organization repository on GitHub
Security Scanner for Agent Skills
- Forks
- 331
- Last pushed
- Updated Oct 2026
Repo stats from the GitHub API, cached Oct 2026.Homepage
CommunityJev Tools & CLIs
Skill Scanner
Cisco's security scanner for agent skills added a System One analyzer: it sends the full skill text to POST /v1/systemone as one advisory Noul question and records the probability, while being deliberately unable to emit a finding or change a severity.
- Stars
- 2.6k
- Language
- Python
- Published by
- Community
- Last verified
- Oct 2026
About this project
A best-effort security scanner for AI Agent Skills that detects prompt injection, data exfiltration, and malicious code patterns. It combines pattern-based detection (YAML + YARA-X), AST and dataflow analysis, an optional LLM-as-a-judge, and a bounded CEL decision layer over typed detector facts. Supports OpenAI Codex Skills and Cursor Agent Skills formats following the Agent Skills specification. With --lenient, also scans non-standard formats such as Claude Code .claude/commands/.md and flat markdown skill repos.
Quoted from the project's README. Read the full README.
- Languages
- Python
- Capabilities
- Open sourceRequires API key
Highlights
- Asks one Noul question per skill: is this skill malicious or unsafe to install?
- Measured on 439 records the scores inverted: AUC 0.25-0.31, benign scored as riskier.
- The tier cannot emit a finding, change a severity or alter a verdict; it is off by default.
- Refuses plaintext endpoints off loopback and refuses redirects to protect the skill's source.
- Oversize content over MAX_STATE_BYTES = 24,000 is skipped, not truncated.
Watch out
Apache-2.0 per its README, though GitHub metadata reports NOASSERTION. The System One analyzer is a deliberate negative result: the available model scored benign skills as riskier than malicious ones, so the tier stays advisory and off by default until a model demonstrates separation on a corpus.
Questions about this entry
Short answers drawn from the linked project or page.
Because the protocol is model-agnostic and the measured failure is a property of the model, not the interface. On 439 records the probabilities were inverted (AUC 0.25-0.31), so gating on them would make the scanner worse; the integration ships with demonstrated separation on a corpus as the precondition for acting on the score.
A recorded advisory probability. The tier sends the concatenated skill text to POST /v1/systemone as one Noul question and stores the answer as last_result for inspection and review triage, adding no finding and changing no severity. Every recommended scanning setup still runs the LLM judge for actual findings.
Measured on MaliciousSkillBench's frozen test split of 839 malicious and 545 harmless skills, rules alone catch 7.7% of malicious skills at HIGH severity at a 4.0% false-positive rate. With the LLM judge on Gemma 4 26B at the balanced preset, 66.7% reach review at MEDIUM+ with a 15.4% FPR and 33.7% are blocked at HIGH with a 6.4% FPR; the quiet preset drops to 50.3% recall at 7.2% FPR.
CPython 3.11 through 3.14 plus uv, pip, pipx or Homebrew: `uv pip install cisco-ai-skill-scanner` or `uv tool install` for a standalone binary. Wheels bundle the CEL helper for glibc Linux on x86-64 and arm64, macOS 14+ and Windows x86-64; anything else builds from source and needs Go 1.27.1 or newer. The recommended presets and settings need 2.2.0 or newer.
OpenAI Codex Skills and Cursor Agent Skills formats following the Agent Skills specification. With --lenient it also scans non-standard layouts such as Claude Code .claude/commands/*.md files and flat markdown skill repositories.
It emits SARIF for GitHub Code Scanning, ships a reusable GitHub Actions workflow, and returns exit codes for build failures via --fail-on-severity. There is also a standard pre-commit framework integration so skills are scanned before every commit.
It refuses plaintext endpoints off loopback and refuses redirects, so the skill source is not exposed to a downgraded or redirected connection. Content larger than MAX_STATE_BYTES, 24,000, is skipped rather than truncated, and the analyzer stays off by default.
Reactions & coverage
Posts, threads, and videos about this entry from around the web.
YouTube: Kubernetes admission control with a decision model, fail-closed.
Hacker News: SafeDep Gryph plus Jev checks for compromised coding-agent behavior.
2 pointsDetecting Compromised AI Coding Agents with Jev and GryphRead the thread on Hacker NewsHacker News: Show HN: Agent Chaperone screens tool calls and results with Jev before the agent sees them.
sepehrsafari5 pointsShow HN: Agent Chaperone – Screen AI agent tool calls and results with JevRead the thread on Hacker NewsX: Secret detection with Jev, tested
More Jev resources like this
8GitHub stars
Supervisor for unattended coding agents: it tracks cross-turn state and asks one batched Jev Choice whether a trajectory should continue, pause or escalate, while deterministic code owns dangerous-command blocking, budget caps and Ed25519-signed receipts.
Jev Tools & CLIs#community#guardrails#safety
Community
VeridicalTech/EdwardPython
VeridicalTech/EdwardPython1GitHub stars
Go linter whose rules are plain-English questions: AST matchers select nodes and Jev answers one Noul per fragment above the rule's threshold. It runs standalone, under go vet, and as a golangci-lint plugin, with cached answers so runs only ask about changed code.
Jev Tools & CLIs#community#linter#go
Community
arturobermejo/semcheckGo
arturobermejo/semcheckGo0GitHub stars
A Claude Code PreToolUse hook that blocks known key formats locally and sends only masked unknown high-entropy strings to Jev for a Noul judgment, blocking at 0.80 and asking the human from 0.30 or whenever Jev is unavailable.
Jev Tools & CLIs#community#security#claude-code
0GitHub stars
OpenAI-compatible /v1/chat/completions proxy that answers an agent host's approval-guardian call with APPROVE, DENY or ESCALATE. Jev answers one Choice question with a calibrated distribution; missing keys, timeouts and malformed answers all fail closed to ESCALATE.
Jev Tools & CLIs#community#agent-safety#guardrails
Community
oppih/approval-judge-bridgePython
oppih/approval-judge-bridgePython110.5kGitHub stars
Multi-agent trading research with optional Jev screening of StockTwits and Reddit posts for company relevance before sentiment analysis runs.
Jev Tools & CLIs#community#python#trading
Community
TauricResearch/TradingAgentsPython
TauricResearch/TradingAgentsPython32GitHub stars
A Java gateway that asks Jev four questions about a deposit or withdrawal, then applies a direction-aware routing matrix in code.
Jev Tools & CLIs#community#java#security
Community
klauswg/jev-guardJava
klauswg/jev-guardJavaFrom the Jev guides
Original write-ups on what this entry is for.
The request shape barely changes across surfaces; the surrounding contract does. When to call the API directly and when a gateway, a Worker binding, or a router entry earns its extra hop.
Jev Tools & CLIs#gateway#vercel#cloudflare
Read article
Taking a decision checkpoint out of Python: what the GGUF and browser WebGPU exports preserve, what they change, and the parity evidence the model cards publish.
Jev Tools & CLIs#open-models#local#onnx
Read article
Jev labels a record that already exists. Code moves, flags, or holds it. How the document, mail, and transfer tools keep the action out of the model.
Jev Tools & CLIs#classification#email#security
Read article
From the Jev community
Posts from builders shipping with Jev right now.