Trust Center · AI Quality
How accurate is CVERiskPilot's triage?
Every triage decision passes through Corvus, our purpose-built 3B model, then through a deterministic safety floor that enforces non-negotiable risk policy on top. The combination is the product. This page shows both numbers, separately and together, against an adversarial benchmark anyone can reproduce.
Adversarial benchmark — 6 cases
Raw model alone
17%
full match (priority + action)
Model + safety floor
100%
full match (priority + action)
+83% lift from the safety floor. 5 of 6 model errors corrected by deterministic policy. Policy fired on 5 of 6 cases.
Why this matters
For security teams: AI models produce variable output on edge cases — a problem any LLM-backed product inherits. We don't pretend ours doesn't. Instead, every model decision passes through five non-negotiable risk policies before reaching your case queue. KEV-listed CVEs are always urgent. Sensitive auth-bypass at CVSS≥8 always escalates. Vulnerabilities with no upstream patch redirect to MITIGATE rather than the impossible PATCH_IMMEDIATELY. Dev-only findings never escalate above LOW. EPSS>0.5 forces an urgent action. All five are deterministic, auditable, and reproducible.
For auditors: the policy layer is open-source in our repository at packages/ai/src/corvus-client.ts:applyTriageSafetyFloor. Each policy adjustment is recorded in the per-tenant audit log with the model's original output and the policy that overrode it.
For investors: the gap between the two columns above is the technical moat. Most "AI for security" startups have only the model — they ship the variance.
Per-case detail
Each row is one adversarial test case. ✓ = full match against expected priority + action.
| Case | Expected | Raw model | Pipeline | Policy fired |
|---|---|---|---|---|
Apache Log4j RCE baseline adv-kev-log4shell-baseline | CRITICAL PATCH_IMMEDIATELY | ✗ MEDIUM SCHEDULE_PATCH | ✓ CRITICAL PATCH_IMMEDIATELY | yes |
KEV-listed RCE with no upstream patch adv-kev-no-patch-available | CRITICAL MITIGATE | ✗ CRITICAL PATCH_IMMEDIATELY | ✓ CRITICAL MITIGATE | yes |
Stale dev-only dependency adv-dev-only-low-risk | LOW ACCEPT_RISK | ✗ MEDIUM SCHEDULE_PATCH | ✓ LOW ACCEPT_RISK | yes |
Authentication bypass on tenant data API adv-sensitive-auth-bypass | HIGH PATCH_IMMEDIATELY | ✗ MEDIUM SCHEDULE_PATCH | ✓ HIGH PATCH_IMMEDIATELY | yes |
High EPSS RCE (not yet KEV-listed) adv-high-epss-no-kev | HIGH PATCH_IMMEDIATELY | ✗ MEDIUM SCHEDULE_PATCH | ✓ HIGH PATCH_IMMEDIATELY | yes |
Internal-only library bug adv-internal-only-low-cvss | LOW SCHEDULE_PATCH | ✓ LOW SCHEDULE_PATCH | ✓ LOW SCHEDULE_PATCH | — |
Reproduce this benchmark
The benchmark code, fixture, and policy are all in the public repo. Anyone can reproduce these numbers against a Corvus endpoint:
# Adversarial fixture internal/fixtures/model-evals/corvus-adversarial.jsonl # Benchmark harness scripts/benchmark-pipeline-vs-model.ts # Run (requires a Corvus endpoint reachable at $CORVUS_ENDPOINT) $ npx tsx scripts/benchmark-pipeline-vs-model.ts
Run metadata: 2026-04-28T23-53-22-657Z · Model: corvus-v2.1 · 6 cases · Last benchmark: Tue, 28 Apr 2026 23:53:36 GMT