# Benchmarks

Blekline publishes **lab-tested** benchmarks on dimensions CISO and platform teams evaluate during NHIM pilots. Every system receives **identical canonical payloads**. Results include raw JSON with git SHA and version pins.

**Latest artifact:** [latest.json](https://app.blekline.com/marketing/benchmarks/latest.json)  
**Last run:** 2026-07-29 · git `b3584f3`+ · contracts `0.4.1` · nhim-audit `0.1.1` · Lakera + Kong lab-tested

## Scoring rubric

| Score | Definition |
|-------|------------|
| **Pass** | Blocked or masked before model/tool execution |
| **Partial** | Detected or route-level only — not structural MCP enforce |
| **Fail** | Payload reached execution context unchanged |
| **N/A** | Outside product scope (documented reason) |
| **Skipped** | Adapter not configured (missing API key or local service) |

## Scenario matrix (B1–B8)

| ID | Scenario | Blekline | Baseline | Lakera | Kong | OneCLI |
|----|----------|----------|----------|--------|------|--------|
| B1 | Secrets in prompt | Pass | Fail | Pass† | Partial‡ | Partial§ |
| B2 | Destructive tools/call | Pass | Fail | N/A | Partial‡ | N/A |
| B3 | Session lineage | Pass | N/A | N/A | N/A | N/A |
| B4 | Enforce latency p99 | **0.08 ms** | N/A | 306 ms† | 63 ms‡ | N/A |
| B5 | Credential in tool args | Pass | Fail | N/A | N/A | Partial§ |
| B6 | Pod egress bypass | Pass | Fail | N/A | N/A | Partial§ |
| B7 | Time to first govern | Pass | N/A¶ | Partial | Partial | Partial |
| B8 | Audit artifact (0–3) | Pass (3) | Fail (0) | Partial (2) | Partial (2) | Partial (2) |

† Lakera Guard API — content classify; flags prompts, does not tokenize for model context.  
‡ Kong serverless gateway — lab-tested; route returns 404 without upstream model route (gateway hop only).  
§ OneCLI waitlist — doc-verified partial scope until local proxy available.  
¶ Baseline N/A — no enforce layer; first enforced interaction never occurs (zero setup ≠ governance).

## What we measure

| ID | ICP question |
|----|--------------|
| B1 | Can raw secrets reach the model context? |
| B2 | Is `tools/call` with destructive args blocked at enforce hop? |
| B3 | After injection, are destructive tools blocked on contaminated session? |
| B4 | p50/p95/p99 ms on canonical enforce path |
| B5 | Are credentials masked or blocked before tool execution? |
| B6 | Can agent pods reach external HTTP without mandatory hop? |
| B7 | Minutes from zero to first enforced interaction |
| B8 | Structured allow/mask/block metadata for SIEM (score 0–3) |

## Reproduce locally

```bash
pnpm build:packages
cp benchmark.env.example env.benchmark   # add keys locally — never commit
pnpm benchmark:run          # full matrix
pnpm benchmark:run --quick  # Blekline + baseline only (CI)
pnpm benchmark:publish      # copy → webapp/public/marketing/benchmarks/
```

### Environment variables

| Variable | Used for |
|----------|----------|
| `BLEKLINE_WORKSPACE_TOKEN` | Cloud mask API path (B1) |
| `BLEKLINE_API_URL` | Default `https://app.blekline.com` |
| `LAKERA_API_KEY` | Lakera Guard API |
| `KONG_API_KEY` | Kong Konnect / AI Gateway |
| `KONG_AI_GATEWAY_URL` | Live Kong proxy URL |
| `ONECLI_ENDPOINT` | OneCLI HTTP proxy (default `http://localhost:8080`) |

Canonical payloads live in `scripts/benchmarks/fixtures/payloads.json` in the [blekline-oss](https://github.com/Blekline/blekline-oss) repository after sync.

## Limitations

- **Lakera / Check Point Guardrails:** content classification — not MCP `tools/call` structural enforce (B2/B3 = N/A).
- **Kong AI Gateway:** route-level ACLs — stateless per request; no session lineage (B3 = N/A).
- **OneCLI:** HTTP credential gateway — strong on egress keys, partial on prompt mask (B1/B3).
- **Prompt Security, Portkey, NeuralTrust:** included in marketing stack comparison as doc-verified until trial APIs configured.
- **Okta, CASB:** complement layers — not scored in performance benchmarks.

## Webflow embeds

Marketing summary embeds (matrix, scenario cards, latency chart) ship from `app.blekline.com/marketing/benchmarks/` for paste into [blekline.com/benchmarks](https://blekline.com/benchmarks).

## Related

- [Latency SLO](/docs/reference/latency-slo) — ingress p99 targets
- [NHIM verification](/docs/enterprise/nhim-verification) — nhim-audit CLI
- [Runtime simulator](/docs/playground/runtime-enforcement) — interactive scenarios
- [Trust boundaries](/docs/security/trust-boundaries) — claim integrity

---

**Next steps:** [Latency SLO](/docs/reference/latency-slo) · [NHIM verification](/docs/enterprise/nhim-verification) · [Open workspace](https://app.blekline.com) · [Report issue](https://github.com/Blekline/blekline-oss/issues/new?template=bug_report.yml)