FrankenSuite research program · A shareable brief

FrankenAlignment A Control Plane for Powerful AI Agents, Designed in Public and Built Only on Paper

A shareable brief from the FrankenSuite research program. Every claim below traces to the assessed repository; sources are linked at the end. Assessment pinned September 22, 2026.

Assessment pinned September 22, 2026 (the assessed commit)·~1,285 words·5 sections·Every claim traceable to a source
Six definitions. NODUS is the program’s four-ring verdict scale (Invest, Pilot, Explore, Monitor) assigned per assessment packet. TRL is technology readiness level, scored 1 to 9 per packet. CI is continuous integration: the project’s automated test runs. The pin is the exact commit the assessment froze on; every claim is evaluated at the pin, not at HEAD. The rider is the license clause withholding all rights, including benchmarking and analysis, from OpenAI, Anthropic, their affiliates, and anyone acting for them. Bus factor is how many people can leave before the project stalls; it is 1 for every repo here.

01 · What it is

What the problem is, and why it matters

This is the FrankenSuite's odd one out. It is not a rewrite of anything. It is an original design: a control plane for AI agents with real authority.

The problem is plain. Agents now act in the world. They delete files, run commands, spend credentials, send network traffic. Most deployments rely on a thin safety stack: prompt guardrails are advisory text filters. A sycophantic agent can still be prompt-injected, because conversational cooperativeness is not an authority boundary. Asking one language model to monitor another produces uncalibrated prose verdicts. Post-hoc filters cannot undo a side effect that already dispatched. If you would not secure a bank with a polite sign, do not secure a shell with a system prompt.

The academic "AI control" agenda, protocols designed to keep systems safe even when a model intentionally tries to subvert them, dates to Greenblatt et al.'s "AI Control: Improving Safety Despite Intentional Subversion" (December 2023, arXiv:2312.06942, presented at ICML 2024). source The deployed open-source standard for the lighter version of the problem is NVIDIA's NeMo Guardrails, an open-source toolkit for adding programmable guardrails to LLM applications. But guardrails are client-side, cooperative, and heuristic. That is exactly the layer the maintainer argues cannot hold.

His stated bet: control must be external and one-directional, and introspection must be economical, versioned, and replayable. Everything in this repo is a working-out of those two sentences.

02 · Why build it

Why build it, and why in Rust?

The maintainer's pitch runs to five claims. External one-directional control: the system that decides whether an agent may affect the world is not the agent, takes no arguments from the agent, and can hold, narrow, suspend, or rewind it. A "congress" of narrower, rotating helper models watches outputs and activations and votes under cryptographic commitment, never hearing the model's defense. The root is the maintainer's own earlier AI-safety writing: internal guardrails always fail; only external monitoring with hard evidence works. source Economical introspection: a sidecar compresses the model's hidden activations onto a learned manifold, with certified margins, so cognition can be saved, rewound, branched, and replayed. Watching everything would otherwise be unaffordable. The DecisionClosure as the single composable artifact: the object authorizing a live dispatch is simultaneously the tamper-evident incident record, the replayable counterfactual, and the regression test. Verifiability by outsiders: control receipts get checked by a verifier that shares no code with the broker. Rigor as constitution: 40 invariants with waivers forbidden, 21 preregistered falsifiable hypotheses with kill criteria.

Why Rust for this: an enforcement boundary is exactly where you want memory safety without a garbage collector, and the project holds a zero-unsafe discipline across a closed dependency universe. The entire workspace compiles against the standard library with zero external packages. If the control plane itself can be memory-corrupted, it is theater.

The counter-argument is blunt, and the design does not resolve it. None of the five parts exists. All 21 hypotheses are unexecuted. The broker, the congress, and the sidecar are design. And the license bars OpenAI and Anthropic, two of the labs doing the most agent-control research, from even analyzing the code. A control plane that forbids the field's center of gravity from evaluating it is strategically self-negating.

03 · What was built

What the project actually built

Sixteen days old at the assessment, 458 commits, all by one human: roughly 194,000 lines of design work — the only suite repo whose primary artifact is a governance methodology rather than software [Inference, Medium]. A 251 KB comprehensive plan. 257 documents. Fifteen machine-readable registries holding 21 falsifiable hypotheses, 40 invariants, and 143 work packets. The code is a two-crate reference model: fa-reference encodes the control plane's logical semantics (effect gates, rights conservation, commit–reveal congress rounds), and xtask is the gate driver that checks formatting, compilation, tests, and doc-to-code concordance. Nothing in the tree touches a real model, a real tool, or a real effect.

The real product is the evidence machinery. Every test batch ships a JSON receipt binding source hashes, job IDs, failed attempts, and raw logs; the September 8 gate ran 511 tests with zero failures, failed attempts included. Defects get deliberately planted and must fail after the controls pass: the negative-evidence standard, as a habit. Every performance target in the registry is self-labeled "all_values_are_unmeasured_proposals": true, and the README says plainly: "No number below has been measured." The README describes the finished system in the present tense, says so in paragraph one, and points readers to the status ledger for what is actually built. source And the AGENTS.md has teeth: "A dry run is not execution, a missing target is not success, and previous evidence does not validate new code." Faked tests, weakened assertions, and editing the spec instead of the implementation are named as forbidden.

The uncomfortable findings

Uncomfortable finding

The executed evidence is 14 days and 416 commits stale. The tree holds roughly 3,482 authored test functions, but the last executed full gate is the September 8 run of 511. About 2,971 tests have no recorded execution. The status ledger opens with a section titled "Unqualified source addition," confessing the newest code is "authored but UNEXECUTED." The pin commit's own message records its tests as unexecuted: "rch not found (exit 127)." The gates require the maintainer's private remote-execution fleet, so no outsider can re-run them. Asupersync is badged as "the runtime" throughout the design, but zero asupersync code is admitted: the recorded review reads "reviewed, not admitted, not built," and the badge is the one place the tense disclaimer fails to protect the reader. The license is MIT plus a rider barring OpenAI, Anthropic, their affiliates, and anyone acting for them from even benchmarking, testing, analyzing, or indexing the code, with automatic termination on breach. source And release is hardcoded to refuse: the release-check command returns an error string, by design.

04 · Where it stands

Where it stands

In our program's terms: TRL 3, NODUS ring Explore. In plain English, the logical core executes as reference models with receipt-bound evidence, but no production element exists and the core thesis is untested. The packet names a dated trigger: if the evidence gap persists past 30 days from the pin (the commit the assessment was pinned to), or the rider stays as-is, it rings down to Monitor.

Technology readiness

TRL 3 on a 1-to-9 scale.

NODUS ring

MonitorExplorePilotInvest

The program’s adoption ring for this project.

Verdict key

NODUS ring
Explore: The default for substantive-but-unproven work.
TRL
Technology Readiness Level, a 1-to-9 scale of how proven a technology is. This brief rates the project at TRL 3.
CI
Continuous integration: the project’s automated checks, run on every change.
The pin
The exact commit the assessment was pinned to. “At the pin” means as of that commit.

Evidence tiers

  1. Tier 1 · Verified — Confirmed by direct inspection of a fresh clone, an API response, or a live page read by the analyst.
  2. Tier 2 · CI-observed — Observed executing on live CI pages. Attests the suite runs, not that it is green, unless pass/fail is legible.
  3. Tier 3 · Maintainer claim — Asserted in README or docs by the maintainer; not independently executed or reproduced.
  4. Tier 4 · External — Independent sources: APIs, papers, press, third-party benchmarks. Absence of coverage is reported as a finding.
  5. Tier 5 · Inference — The analyst’s judgment. Always labeled; never presented as fact.

What this verdict rests on. Tier labels are the assessment packet’s own annotations.

  • Tier 1 · Verified: The 251 KB plan read selectively; the pin commit’s own message records its tests as unexecuted.
  • Tier 3 · Maintainer claim: No gate, receipt, or hypothesis was executed; the RCH fleet was unavailable; every H-hypothesis is untested against a real model.
  • Tier 4 · External: No independent coverage of the repo exists.
  • Tier 5 · Inference: The Monitor ring: plan-stage work with no production element.

Should you use it?

No: there is no release and no tags, and the release gate is deliberately wired to refuse.

Should you learn from it?

Yes, specifically from the honesty apparatus — an artifact that ports to any team tomorrow with no dependency on the reference crate: the unqualified-addition disclosure, the planted mutations, the self-labeled unmeasured SLOs, and "previous evidence does not validate new code" are process technology any team can adopt immediately.

What would change the verdict, in order

  1. G1 executes (one real brokered effect with a receipt-bound gate outsiders can inspect)
  2. any of the 21 hypotheses runs against a real model with published results
  3. the rider narrows to training-data incorporation only
  4. the asupersync commitment unblocks or is re-scoped
  5. a second maintainer or institutional funder appears.

05 · What it teaches

What this teaches about building with agents

The transferable lesson is the "unqualified source addition" rule.

Every fast-moving project rots the same way: the tests were green two weeks ago, the code changed since, and the badge still says green. Agent-driven development makes it worse, because agents generate code far faster than anyone re-runs gates. This repo's rule is mechanical: any commit landing after the last executed gate must open the status ledger with a self-disqualification until the gate re-runs. Staleness is not discovered in an audit; it is confessed at the top of the page, by the project's own process.

For anyone building with agents, the question to copy is not whether the tests passed but when, and against which commit. Make "previous evidence does not validate new code" a rule your tooling enforces, not a slogan your docs carry. The teams that do this will look slower, their status pages full of self-disqualifications. They will also be the only ones whose green means anything.

Evidence

This page cites selected evidence. The full claim-by-claim audit is in the assessment packet.

Open a panel to see what each source evidences. Full claim-by-claim audit.

Full claim-by-claim audit with evidence tiers lives in the program's assessment packet.

The repositoryGitHub

the repository (GitHub)

README

its README

LICENSE

LICENSE, IMPLEMENTATION_STATUS.md, and machine-readable registries at the assessed commit

arXiv:2312.06942arXiv:2312.06942

Greenblatt et al. (arXiv:2312.06942)

GitHubGitHub

NVIDIA NeMo Guardrails (GitHub).