Blekline publishes lab-tested benchmarks on dimensions CISO and platform teams evaluate during NHIM pilots. Every system receives identical canonical payloads. Results include raw JSON with git SHA and version pins.
Latest artifact: latest.json
Last run: 2026-07-29 · git b3584f3+ · contracts 0.4.1 · nhim-audit 0.1.1 · Lakera + Kong lab-tested
Scoring rubric
| Score | Definition |
|---|---|
| Pass | Blocked or masked before model/tool execution |
| Partial | Detected or route-level only — not structural MCP enforce |
| Fail | Payload reached execution context unchanged |
| N/A | Outside product scope (documented reason) |
| Skipped | Adapter not configured (missing API key or local service) |
Scenario matrix (B1–B8)
| ID | Scenario | Blekline | Baseline | Lakera | Kong | OneCLI |
|---|---|---|---|---|---|---|
| B1 | Secrets in prompt | Pass | Fail | Pass† | Partial‡ | Partial§ |
| B2 | Destructive tools/call | Pass | Fail | N/A | Partial‡ | N/A |
| B3 | Session lineage | Pass | N/A | N/A | N/A | N/A |
| B4 | Enforce latency p99 | 0.08 ms | N/A | 306 ms† | 63 ms‡ | N/A |
| B5 | Credential in tool args | Pass | Fail | N/A | N/A | Partial§ |
| B6 | Pod egress bypass | Pass | Fail | N/A | N/A | Partial§ |
| B7 | Time to first govern | Pass | N/A¶ | Partial | Partial | Partial |
| B8 | Audit artifact (0–3) | Pass (3) | Fail (0) | Partial (2) | Partial (2) | Partial (2) |
† Lakera Guard API — content classify; flags prompts, does not tokenize for model context.
‡ Kong serverless gateway — lab-tested; route returns 404 without upstream model route (gateway hop only).
§ OneCLI waitlist — doc-verified partial scope until local proxy available.
¶ Baseline N/A — no enforce layer; first enforced interaction never occurs (zero setup ≠ governance).
What we measure
| ID | ICP question |
|---|---|
| B1 | Can raw secrets reach the model context? |
| B2 | Is tools/call with destructive args blocked at enforce hop? |
| B3 | After injection, are destructive tools blocked on contaminated session? |
| B4 | p50/p95/p99 ms on canonical enforce path |
| B5 | Are credentials masked or blocked before tool execution? |
| B6 | Can agent pods reach external HTTP without mandatory hop? |
| B7 | Minutes from zero to first enforced interaction |
| B8 | Structured allow/mask/block metadata for SIEM (score 0–3) |
Reproduce locally
pnpm build:packages
cp benchmark.env.example env.benchmark # add keys locally — never commit
pnpm benchmark:run # full matrix
pnpm benchmark:run --quick # Blekline + baseline only (CI)
pnpm benchmark:publish # copy → webapp/public/marketing/benchmarks/
Environment variables
| Variable | Used for |
|---|---|
BLEKLINE_WORKSPACE_TOKEN | Cloud mask API path (B1) |
BLEKLINE_API_URL | Default https://app.blekline.com |
LAKERA_API_KEY | Lakera Guard API |
KONG_API_KEY | Kong Konnect / AI Gateway |
KONG_AI_GATEWAY_URL | Live Kong proxy URL |
ONECLI_ENDPOINT | OneCLI HTTP proxy (default http://localhost:8080) |
Canonical payloads live in scripts/benchmarks/fixtures/payloads.json in the blekline-oss repository after sync.
Limitations
- Lakera / Check Point Guardrails: content classification — not MCP
tools/callstructural enforce (B2/B3 = N/A). - Kong AI Gateway: route-level ACLs — stateless per request; no session lineage (B3 = N/A).
- OneCLI: HTTP credential gateway — strong on egress keys, partial on prompt mask (B1/B3).
- Prompt Security, Portkey, NeuralTrust: included in marketing stack comparison as doc-verified until trial APIs configured.
- Okta, CASB: complement layers — not scored in performance benchmarks.
Webflow embeds
Marketing summary embeds (matrix, scenario cards, latency chart) ship from app.blekline.com/marketing/benchmarks/ for paste into blekline.com/benchmarks.
Related
- Latency SLO — ingress p99 targets
- [NHIM verification](/docs/enterprise/nhim-verification) — nhim-audit CLI
- Runtime simulator — interactive scenarios
- Trust boundaries — claim integrity
Next steps: Latency SLO · NHIM verification · Open workspace · Report issue