Bottom line
Inference, low confidence: put an existing rail toolkit or scanner library in front of your model and its tools (NeMo Guardrails, Guardrails AI, or LlamaFirewall for tool calls), attack it with an existing red-team tool (garak, PyRIT, promptfoo), and do not write your own. Wrap both, because no project in the evidence publishes a false-positive or false-negative threshold you can inherit: keep your own labeled corpus of prompts that must pass and prompts that must be blocked, calibrate the model you use as a judge on its own set, and treat any "it blocks jailbreaks" claim as unproven until those two receipts exist. Confidence is low because the evidence is file trees only: no test was run and no CI result was checked.
Adopt, do not rebuild
License checked on 2026-09-23; non-permissive licenses are named in the text.
- NVIDIA-NeMo/Guardrails Apache-2.0: input, output, dialog, retrieval and execution rails configured in Colang, with jailbreak detection and LlamaGuard integration built in; it tests rails against recorded model responses and keeps real-model tests in a separately gated tier. [Verified] (ecosystem/pickup/_evidence/guardrails.md:7
Reference input/output/dialog/retrieval/execution rails toolkit (Colang), shipped by NVIDIA
; ecosystem/pickup/_evidence/guardrails.md:24real-LLM QA tier separated from unit tier
) - guardrails-ai/guardrails Apache-2.0: a guard-and-validator framework with a hub of validators and a machine-readable spec language for declarative checks. [Verified] (ecosystem/pickup/_evidence/guardrails.md:8
Guard + Validator architecture with a validators Hub
) - meta-llama/PurpleLlama (LlamaFirewall, Llama Guard, Prompt Guard) LicenseRef-Llama-3.2-Community AND MIT: moderation classifiers and a separate firewall that gates agent tool calls, the family of rails for tools rather than prompt text. License: mixed and partly source-available. The repository root and the Llama Guard and Prompt Guard directories are under Llama Community License agreements (an acceptable-use policy, a "Built with Llama" display requirement, and a separate Meta license above 700 million monthly active users); LlamaFirewall, CodeShield and CyberSecEval are MIT. [Verified] (ecosystem/pickup/_evidence/guardrails.md:10
LlamaFirewall (agent tool-call gating)
; ecosystem/pickup/_evidence/guardrails.md:34guardrails for tool input/output are a separate category from prompt text rails
; stack/licenses.tsv:53LLAMA 3.2 COMMUNITY LICENSE AGREEMENT
; stack/licenses.tsv:53Llama-Guard/ and Llama-Prompt-Guard-2/ dirs carry Llama 2/3.2/4 Community Licenses
; stack/licenses.tsv:53display, >700M MAU needs a Meta license
; stack/licenses.tsv:53LlamaFirewall/, CodeShield/, CybersecurityBenchmarks/, SensitiveDocClassification/ are MIT
) - NVIDIA/garak, microsoft/PyRIT, promptfoo/promptfoo Apache-2.0 MIT MIT: red-team tools with enumerated attack probes, jailbreak datasets and evasion strategies; writing your own attack library would recreate them. [Verified] (ecosystem/pickup/_evidence/guardrails.md:12
probes (attack generation) cleanly separated from detectors (success judging)
; ecosystem/pickup/_evidence/guardrails.md:13Red-teaming is a first-class eval surface, not a bolt-on.
)
Copy these practices
- Keep attacks and verdicts in separate code: the code that generates attacks and the code that judges success share no helpers and have mirrored test trees, so an attack cannot quietly grade itself. Starter kit: Anti-reward-hacking law adopted. [Verified] (ecosystem/pickup/_evidence/guardrails.md:23
Split attack generation from success detection
) - Pass and block cases as parameters in code: every scanner carries a labeled corpus, and the must-pass and must-block prompts sit in the tests as parameter tables, not in a spreadsheet. Starter kit: Acceptance shape: positive observable + planted negative + no-claim line (provisional). [Verified] (ecosystem/pickup/_evidence/guardrails.md:27
FP/FN cases live in code as parameters, not in a spreadsheet
; ecosystem/pickup/_evidence/guardrails.md:26every scanner ships with a labeled example corpus and a timing harness
) The source of this practice, protectai/llm-guard, is archived: copy its corpus-and-timing shape, but do not adopt the library. [Verified] (stack/licenses.tsv:77ARCHIVED on GitHub (archived=true): no further maintenance
) - Calibrate the judge on its own set: the model that scores refusals gets its own validation set, kept and versioned apart from the attack corpus, so judge drift is measured. Starter kit: none. [Verified] (ecosystem/pickup/_evidence/guardrails.md:30
the judge that scores refusal gets its own FP/FN calibration set; judge drift is measured, not assumed
; ecosystem/pickup/_evidence/guardrails.md:29keep the attack corpus and the judge-calibration set (scorer_evals) as separate, versioned artifacts
) - Record model calls once, replay them in CI: rail logic tests replay recorded responses or mock outputs and never call a live model; real-model tests run only in a flagged tier. Starter kit: none. [Verified] (ecosystem/pickup/_evidence/guardrails.md:32
record once, replay in CI; rails' LLM calls never hit the network in unit tests
) - Split CI by concern: separate workflows for fast pull-request tests, the full suite, latest dependencies, documentation examples that must run, and a dependency audit. Starter kit: none. [Verified] (ecosystem/pickup/_evidence/guardrails.md:33
PR-fast suite, full suite, latest-deps suite, examples-check (docs examples must run), dependency audit
)
Build only if
- You need an enforcement boundary that the agent cannot route around, where every path to a protected action provably crosses a check. franken_alignment's maintainer argues text rails and middleware cannot give that property, but the design that would is not built and disclaims being a security boundary today, so this motive is real for some builders and unmet by anyone in the evidence, including FrankenSuite. [Maintainer claim] (packets/franken_alignment-assessment.md:67
guardrails are advisory text filters or client-side middleware
) and [Verified] (packets/franken_alignment-assessment.md:96DecisionClosure / effect gate / broker / adapters are G1+ work — not implemented
) - The licenses do not justify building. If the Llama Community License terms on Llama Guard and Prompt Guard do not fit your product, the MIT-licensed LlamaFirewall and CodeShield remain usable, and NeMo Guardrails and Guardrails AI are Apache-2.0; LLM Guard's archiving is not a reason to build either, since its MIT license permits a fork and Guardrails AI covers the same scanner role. [Verified] (stack/licenses.tsv:53
LlamaFirewall/, CodeShield/, CybersecurityBenchmarks/, SensitiveDocClassification/ are MIT
; stack/licenses.tsv:67NVIDIA-NeMo/Guardrails Apache-2.0 permissive
; stack/licenses.tsv:33guardrails-ai/guardrails Apache-2.0 permissive
; stack/licenses.tsv:77protectai/llm-guard MIT permissive
)
Where FrankenSuite touches this
- franken_alignment designs an external control plane for tool-using agents (one-shot permits, conserved rights, independently checkable receipts); only an in-memory reference model exists, its security notes say it is not a boundary, and its packet grades it TRL 3, Explore. [Verified] (packets/franken_alignment-assessment.md:106
The Rust reference model is not a sandbox, credential broker
; packets/franken_alignment-assessment.md:22(TRL 3 — see §4.9). It clears
) - franken_engine drives a six-step decision ladder from allow to quarantine for untrusted JavaScript extensions, with signed evidence per decision; TRL 4 to 5, Explore. [Verified] (packets/franken_engine-assessment.md:119
allow / challenge / sandbox / suspend / terminate / quarantine
; packets/franken_engine-assessment.md:197Technology readiness | TRL 4–5 | Lab-validated components
) - franken_node's threat model already has a row for agent tool abuse (capability chains, attenuated delegation, signed action receipts); the packet reads that as an agent-safety fit, not a shipped guardrail; TRL 4, Explore. [Inference] (packets/franken_node-assessment.md:300
the threat model's MCP-mutation-abuse row (audience-bound capability chains, attenuated delegation, signed action receipts)
; packets/franken_node-assessment.md:205Technology readiness | TRL 4
) - franken_snowflake gates agent writes behind explicit arming: nothing mutates a warehouse until the profile opts in, with a dry-run and confirm step; TRL 6, Explore. [Verified] (packets/franken_snowflake-assessment.md:143
nothing mutating until the profile opts in (WRITE_ENABLED)
; packets/franken_snowflake-assessment.md:182Technology readiness (TRL 1–9) | 6
)
What we cannot say
- Whether protectai/llm-guard will be maintained: the pack recorded it as active (last push 2026-07-08), but a read-only
gh api repos/protectai/llm-guardon 2026-09-23 returnedarchived: true, so it is no longer recommended as an incumbent and appears only as a practice source [Verified] (ecosystem/pickup/_evidence/guardrails.md:9protectai/llm-guard | 3,207 | 2026-07-08
; stack/licenses.tsv:77ARCHIVED on GitHub (archived=true): no further maintenance
). - Whether any of these rails or detectors work as described: the evidence is repository metadata and file trees, no test was executed and no CI result was checked (ecosystem/pickup/_evidence/guardrails.md:42
I did not execute tests or verify CI pass/fail state
). - What false-positive and false-negative thresholds are defensible: no evidence repository publishes one, so each project must set its own (ecosystem/pickup/pickup-guardrails.md:360
no evidence repo publishes a norm
). - HarmBench is cited only through its downstream use: the repository has not changed since 2024 and is treated as a benchmark spec, not maintained code (ecosystem/pickup/_evidence/guardrails.md:38
treat it as the canonical benchmark spec, not a maintained codebase
). - PyRIT's use in 100+ red-team operations is Microsoft's own statement (ecosystem/pickup/_evidence/guardrails.md:11
used in 100+ red-team operations per its paper
). - whylabs/langkit, rebuff and strongreject are left out: stalled, missing, or too thin (ecosystem/pickup/_evidence/guardrails.md:40
a guardrail clean-room should not model a stalled project
; ecosystem/pickup/_evidence/guardrails.md:41via API; do not cite it. (A third-party awesome-list
; ecosystem/pickup/_evidence/guardrails.md:39verified but too thin to carry a trend line
). - Stars are a point-in-time trend signal, not a safety score (ecosystem/pickup/_evidence/guardrails.md:43
use them as trend signal, not as quality scores
).
Revisit when
- A rail project publishes measured false-positive and false-negative rates on a pinned, labeled corpus with a calibrated judge: that becomes the threshold to beat.
- An open tool-call enforcement layer shows, with a checked coverage argument, that every path to a protected action crosses it; that would test the one motive this verdict leaves for building.
- HarmBench's published judge numbers are reproduced without the original classifier weights, which would make it a runnable oracle rather than a spec.
Sources
- Pickup companion: ecosystem/pickup/pickup-guardrails.md, the charter seed, oracle candidates, and first claim registry for this area.
- Evidence pack: ecosystem/pickup/_evidence/guardrails.md, the repositories checked against the GitHub API, with dates.
- Verdict source: stack/guardrails.md, written under stack/METHOD.md.
Starting a build here?
Begin from the pickup companion for Guardrails and red-teaming: it carries the charter seed, the oracle candidates and how to verify them, and the first claims to register, so a new project starts from evidence instead of from scratch.