Quick lookup: given a guardrail, what controls does it cover?
Guardrail
NIST AI RMF
OWASP LLM
NIST 800-53
keyword_blocklist
MEASURE 2.6
LLM06
SI-3, SI-10
length_limit
MEASURE 2.7
LLM04
SI-10
regex_pattern
MEASURE 2.7
LLM01, LLM06
SI-10
pii_detection
MEASURE 2.10, GOVERN 1.4
LLM06
SC-28
rate_limiter
MEASURE 2.7
LLM04, LLM10
AC-7
system_prompt_leak
MEASURE 2.7
LLM01, LLM06
SI-4
toxicity
MEASURE 2.6
—
SI-3
safety_check
MEASURE 2.3, 2.6
LLM01
SI-4
adversarial_detection
MEASURE 2.7
LLM01
SI-3, SI-4
topic_restriction
MEASURE 2.6
—
AC-4
topic_enforcement
MAP 2.3
—
AC-4
role_redaction
MEASURE 2.10
LLM02
SC-28
hallucinated_links
MEASURE 2.3
LLM09
—
tone_enforcement
MEASURE 2.11
—
—
bias_detection
MEASURE 2.11
—
—
pii_leakage
MEASURE 2.10, GOVERN 1.4
LLM02, LLM06
SC-28
competitor_mention
GOVERN 5.1
—
—
rbac_guard
GOVERN 4.1
LLM08
AC-2, AC-3, AC-6
data_access_guard
GOVERN 1.4, MAP 3.4
LLM06
AC-3, AC-4
mcp_guard
MANAGE 3.1
LLM05, LLM07
SC-7
action_guard
MANAGE 2.4
LLM08
AC-6
tool_allowlist
MAP 3.4
LLM07
AC-6
tool_use_control
MAP 3.4
LLM07
AC-3
tool_call_rate_limiting
MEASURE 2.7
LLM04
AC-7
tool_call_validation
MEASURE 2.7
LLM07
SI-7, SI-10
tool_output_sanitization
MEASURE 2.10
LLM02
SC-28
sensitive_action_confirmation
MANAGE 2.4
LLM08
AC-6
action_classification
MAP 3.4, MAP 5.1
LLM08
AC-6
scope_boundaries
GOVERN 4.1
LLM08
AC-3, AC-4, SC-7
loop_detection
MEASURE 2.7
LLM04
SI-4
budget_controls
GOVERN 6.1
LLM04, LLM10
—
delegation_control
GOVERN 4.1
LLM08
AC-5, AC-6
memory_guardrails
MEASURE 2.7
—
SI-7
memory_pii_scrubbing
MEASURE 2.10
LLM06
SC-28
memory_injection_detection
MEASURE 2.7
LLM01
SI-3, SI-4
memory_retention_policies
GOVERN 1.4
—
AU-11
memory_access_control
GOVERN 4.1
LLM06
AC-3
chain_of_thought_monitoring
MEASURE 2.8
LLM01, LLM09
SI-4
context_window_guardrails
MEASURE 2.7
LLM04
SI-4
7. Evidencing Compliance
For auditors asking “how do you demonstrate control X?”, point to:
Evidence Type
Location
Guardrail config
config/default.yaml + per-tenant configs in Redis
Event logs
Elasticsearch index votal-shield-logs*
Audit trail
Redis append-only stores (audit:{tenant}, decisions:{tenant}), queryable via /v1/shield/audit and /v1/shield/decisions/{tenant}
Per-guardrail metrics
votal.guardrail.latency_ms, votal.guardrail.passed in ES
Risk scoring
event.risk_score, event.severity in every event
Trace correlation
trace.id, agent.key, votal.session_id across all events
Blocked attack samples
Query ES for event.kind: "alert" AND event.risk_score >= 90
RBAC enforcement
rbac_guard decisions logged with agent.key + votal.role_name
8. Kibana Queries for Compliance Reports
# LLM01 (Prompt Injection) — all blocked attacks
votal.attack_type: ("prompt_injection" OR "jailbreak") AND event.outcome: "failure"
# LLM06 (Sensitive Info Disclosure) — all PII events
votal.guardrail.name: ("pii_detection" OR "pii_leakage") AND votal.guardrail.passed: false
# MEASURE 2.10 (Privacy) — PII blocks by tenant
agent.key: acme-* AND votal.guardrail.name: pii_*
# MANAGE 4.3 (Incidents) — critical alerts last 24h
event.kind: "alert" AND event.severity: "critical" AND @timestamp >= "now-24h"
# AC-2 (Account Management) — new agent activity
agent.key: * AND event.action: "request"
# SI-4 (Monitoring) — all guardrail activity
NOT url.path: ("/health" OR "/ping")
9. Auditor evidence pack — a repeatable checklist
When an auditor or customer security team asks for evidence, produce this pack. It
maps one-to-one to the control tables above and is reproducible on demand.
#
Evidence item
How to produce it
1
Control inventory
This document (guardrail → framework control matrix, §1–6).
2
Live guardrail config
GET /v1/shield/policies/{tenant}/bundle/export — the exact enforced policy bundle, versioned.
3
Policy change history
GET /v1/shield/policies/{tenant}/{policy}/versions — who changed what, when, with rollback points.
4
Enforcement decisions
GET /v1/shield/decisions/{tenant}?action=block&since=… — every block/warn with agent, tool, policy, reason, IP, timestamp.
5
Full audit trail
GET /v1/shield/audit?since=…&until=… — all guardrail decisions for the period, tenant-scoped.
6
Blocked-attack samples
SIEM/Kibana: event.kind:"alert" AND event.risk_score>=90 (queries in §8).
7
Per-control effectiveness
Per-guardrail metrics (block rate, latency) from storage/guardrail_metrics.py (90-day retention).
Once the tamper-evident audit ships: GET /v1/shield/audit/verify (chain intact) + signed …/audit/export bundle verifiable offline.
Tip for staged rollout evidence: run new controls in monitor mode
(action: warn|log) first; the decision audit shows would_block volume so you
can evidence “tuned before enforced” for change-management review.